Fake it till you make it - how good are AI detectors really?

The fact is, you have to make it clear when something was created by AI. But who wants to control that and how?

It's the same age-old game as counterfeiting banknotes, counterfeiting products and so on. As soon as one side gains a new level, the other side has to follow suit or at least pretend to have caught up.

In the case of AI, we have the EU AI Act or, in Germany, the KI-Verordnung, which set out the legal framework. This states in Article 50, quote: ...shall disclose that the content has been artificially generated or manipulated: https://www.euaiact.com/article/50 

ATTENTION: This is in no way a legal advice!

If the person publishing the content does not do this, however, numerous app providers are now advertising that they can do this for you:

How do these checking apps work?

AI checkers identify various characteristics, including recurring phrases, consistent sentence structures and the absence of personal aspects in the texts. By examining these patterns, an AI detector can recognize whether content was created by humans or by an AI.

Research currently classifies these three approaches:

• Machine learning: classifiers learn from sample texts, but need to be trained for many text types, which is expensive.
• Digital watermarks: Invisible watermarks in text that can be recognized by algorithms. Providers of AI tools would have to insert these watermarks, which is very unlikely.
• Statistical parameters: Requires access to probability values of the texts, which is difficult without API access.

IT journalist Melissa Heikkilä says: “The enormous speed of development in this sector means that any methods of recognizing AI-generated texts will look very old very quickly.” Source: https://www.heise.de/hintergrund/Wie-man-KI-generierte-Texte-erkennen-kann-7434812.html

OpenAI released the AI Text Classifier in early 2023 to recognize AI-generated texts. However, the recognition rate was very low at just 26%, and 9% of human texts were incorrectly classified as AI texts. Due to this insufficient accuracy, OpenAI removed the tool from the market in mid-2023. A new version of the tool is not yet available. This example shows that we should not have too high expectations of AI recognition tools.

In general, anyone can use the following aspects to decide for themselves whether a text was created by a human or by an AI:

• AI-generated texts often are not very original or varied and contain many repetitions, while human authors vary more when writing.
• A style with many keywords strung together could also indicate an AI as the author.
• AI tools often make mistakes with acronyms, technical terms and conjunctions.

How well do the AI detectors work?

There are now quite a few of these apps. From free / commercial financed or to be paid, there is everything. Here are two overviews:

• Best AI Detector | Free & Premium Tools Compared: https://www.scribbr.com/ai-tools/best-ai-detector/
• The best AI content detectors in 2024: https://zapier.com/blog/ai-content-detector/ 

I did my tests with noplagiat. The tool is rated as good to very good and recognized texts that I had created for this article by Microsoft Copilot in Edge.

Test 1:

My prompt: “List the planets in our solar system. Tell which is the largest and which is the smallest. Name the largest moons. How old is our solar system? Explain how our solar system was formed. Formulate the answer as a scientific essay.”

Result and rating:

Test 2:

BUT a simple and small adjustment to the prompt caused noplagiat to stumble:

PROMPT: "List the planets in our solar system. Tell which is the largest and which is the smallest. Name the largest moons. How old is our solar system? Explain how our solar system was formed. Formulate the answer as a scientific essay. Use the writing style of Stephen King."

Result and evaluation:

Only this small addition in the prompt reduced the rate from 47% to 6%. Well, you might not want to write an essay about our solar system in the style of Stephen King. However, the example clearly shows where the weaknesses of the current solutions are.

Test 3:

It also recognized texts that were 100% not created by an AI correctly. This is the abstract of my new book, which is just about to be finalized:

The next version of AI apps

Enriching the AI-generated text with content, reviewing the text and adding text passages/facts from other sources is what separates the next version of AI apps. Here it is the case that you do not receive the answer to your prompt immediately, but that it can sometimes take a few hours. With Studytexter.de, for example, up to 4 hours.

Here are three examples of such solutions:

• https://studytexter.de/: Quote from the homepage - Your entire term paper at the touch of a button in under 4 hours. Innovative AI text synthesis - especially for German academic papers. 1000x better than ChatGPT.
• https://neuroflash.com/de/: Quote from the homepage - Strengthen your marketing with personalized AI content. The all-in-one solution for brand-compliant content with AI, from ideation to content creation and optimization. neuroflash helps marketing teams save time, ensure a consistent message and improve creative processes.
• https://thezimmwriter.com/: ZimmWriter is the world's first AI content writing software for Microsoft Windows. It allows you to use the AI provided by OpenAI directly on your desktop! -> Note: The app advertises 10 features that are supposed to be entered and many details that are supposed to make generated text unique.

Conclusion

Considering that humans do themselves well to check the output and adapt / reformulate it if necessary, it is currently not possible to verify reliably whether a text was created by an AI or not. 

AI and the productivity and quality of consultants work

The study Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality examines the impact of artificial intelligence on the productivity and quality of life of knowledge workers. This is a field study conducted by the Harvard Business School in collaboration with the Boston Consulting Group.

Key statements:

• Experimental conditions: The study included 758 counselors who were divided into three groups: without AI access, with GPT-4 AI access, and with GPT-4 AI access plus an introduction to prompt engineering.
• Increase in productivity: Consultants who used AI were significantly more productive. On average, they completed 12.2% more tasks and required 25.1% less time.
• Quality improvement: The quality of the tasks supported by AI was more than 40% higher compared to the control group.
• Different effects: Consultants with below-average performance benefited more from AI support (43% increase in performance) than those with above-average performance (17% increase in performance).
• Limitations of AI: For tasks that were outside the capabilities of AI, consultants with AI support were 19 percentage points less successful in providing correct answers.
• Use of AI: Two main patterns of successful AI use were identified: “Centaurs”, who divide tasks between humans and AI, and “Cyborgs”, who fully integrate their workflows with AI.

The study highlights that AI can offer significant benefits for the productivity and quality of life of knowledge workers, but also names challenges and risks, especially for tasks that are outside the current capabilities of AI.

A detailed summary can be downloaded here:

Zusammenfassung über KI und die Produktivität und Qualität von Wissensarbeitern.pdf

Recap on AI and Knowledge Worker Productivity and Quality.pdf

The linke to the complete study is this one: https://www.hbs.edu/faculty/Pages/item.aspx?num=64700

Copilot in M365 & PowerPoint had some couples-therapy

UPDATES August 2024:

The article Work Smarter: Copilot Productivity Tips by Briana Taylor from August 26, 2024 is about Copilot in PowerPoint this time.

The article refers to the roadmap ID: 406170 

The article is structured as follows:

• Tip 1: Create presentations using brand templates
• Tip 2: Create presentations from Word & PDF documents
• Tip 3: Add images to your presentations

At least the function behind Tip 2 has been available for some time. Focusing Tip 1, some technical requirements are necessary in order to be able to use this feature. An Organizational Asset Library (OAL) must be set up for PowerPoint, in which the PowerPoint templates (.potx files) must then be stored and maintained centrally.

How to do this is described here: Create an organization assets library.

In order to then create a PowerPoint based on your own master with the support of Copilot, this master must first be selected:

The rest is then the same as before:

The article from which the screenshot is taken, Add a slide or image to your presentation with Copilot in PowerPoint, also goes into Tip 3: Add images to your presentations:

#genAI - has pushed business to the limit even further!

We simply took a system that was already at its limit and added another layer to it!

The Great Acceleration (Die Große Beschleunigung)

This is the title of a book by Christian Stöcker (Die Große Beschleunigung: Climate change, digitalization, economic growth - how we can hold our own in an exponentially changing world | https://amzn.eu/d/a77rREs)

A key topic of the book is the exponential growth and rapid change that we are currently seeing and have seen in recent years. For people, but even more so for entire cultures or companies, it is a challenge to understand and manage such changes. The rapid pace of change in today's world has far-reaching consequences for the economy and for companies. Christian Stöcker mentions the following in the context of artificial intelligence, for example:

• Technological disruption: Progress in technologies such as generative AI is driving change. Companies must continuously integrate new technologies in order to remain competitive.
• Lack of skilled workers: The demand for qualified employees is increasing, particularly in areas such as IT and data analysis. 
• New business models and processes: Digitalization often requires a complete realignment of business models and internal processes. This can lead to an increased workload and the need for constant reorientation.
• Regulatory requirements: New laws and regulations, such as the AI Regulation, the EU AI Act, pose challenges. Companies must ensure that they meet these requirements.
• Cultural change: The changes brought about by artificial intelligence / GenAI also require an adaptation of the corporate culture. Flexibility, a willingness to innovate and a new form of employee management are becoming increasingly important.

These facts show that in a rapidly changing world, companies not only need to work faster, but also smarter.

The article Capitalism in the mistrust trap by Michael Hüther follows the same line. The high rate of change and the constant emergence of new pseudo-innovations creates enormous pressure on companies. In the medium and long term, this can lead to an exhaustion of resources and the workforce.

Gartner, where do we stand?

The Gartner Hyper Cicle is a good indicator / orientation for innovations.

The Hyper Cycle is a graphical representation of innovative topics in five phases. It shows the maturity and acceptance of new technologies.

• Innovation trigger: A technological breakthrough arouses interest and generates media attention. Often there are no usable products yet and the commercial feasibility is unproven.
• Peak of Inflated Expectations: Early successes and exaggerated expectations lead to hype. Many companies show interest, but there are also many disappointments.
• Trough of Disillusionment: Interest declines as the technology fails to meet high expectations. Some providers fail or withdraw.
• Slope of Enlightenment: The benefits of the technology become clearer and better understood. Second and third generations of products appear and more companies begin to fund pilot projects.
• Plateau of Productivity: The technology is widely adopted and its market relevance becomes clear. It is now widely used and brings measurable benefits.

Hyper Cycle help to separate the hype from the actual drivers of a technology and make well-founded decisions about technology investments. As far as artificial intelligence / GenAI is concerned, it looks like this.

2023 - 2024 shows we are entering the field of disillusionment / the valley of disappointment:

The article Almost every third GenAI project is discontinued also fits in with this. The article lists the following reasons, among others:

• Poor data quality: Many projects fail due to inadequate and incorrect data.
• Escalating costs: The development and implementation of GenAI models is expensive.
• Unclear business value: Companies have challenges proving the business value of GenAI projects.

GenAI solutions, such as Microsoft Copilot or Google Gemini, are currently in several Hyper Cicle phases simultaneously. It is still an innovation trigger that causes amazement, especially among people who are less tech-savvy. This still leads to Peak of Inflated Expectations. Employees in this phase name use cases that GenAI solutions will not be able to achieve in the foreseeable future. Example:

• Suppliers should be regularly evaluated by an AI with regard to their quality. The AI should analyze relevant complaints and take them into account in a summarized evaluation. In doing so, the AI should be guided by the supplier's previous assessments and automatically inform the supplier of any changes to its status.
• The AI should evaluate and analyze commodity index data available online in order to make forecasts about the current and future price and supply situation.
• An AI application that comprehensively checks applications for guidelines and evaluation criteria and decides whether the application should be approved. The AI also creates a detailed and legally binding justification.

If it becomes clear that such scenarios cannot be implemented by GenAI, at least not at present, the next step is Trough of Disillusionment.

GenAI as an enabling technology

The article GenAI as an Enabling Technology: Empowering Yourself and Gaining Your Employee by Dr. Jim Walsh describes how GenAI can help people increase their skills and productivity. It can enable users to perform their tasks more efficiently and successfully.

Examples:

• GenAI supports the automation of routine tasks, the creation of content and the analysis of large amounts of data.
• Content creation: Automated generation of texts and graphics for marketing campaigns, for example.
• Personalization: Optimization of advertisements and customer segmentation for targeted marketing measures.
• Chatbots and digital assistants: Support in customer service and internal processes.
• Program code generation: Automated creation and improvement of software code.

Reality check

Unfortunately, some of the marketing promises made by the major brands are still a long way from reality. This is shown, for example, by the comparison between Microsoft Copilot and Google Gemini 

Here are two further examples that clearly show that we are still at the beginning with some of the new solutions:

• A guide to the length of the documents you provide to Copilot
• Approximately 80,000 words is the limit for Microsoft Copilot to summarize a document.
• Rewriting with Copilot works best with a document of less than 18,000 words.
• Using your organization's branding with Copilot in PowerPoint - minimum requirements for templates.
• Follow the instructions below to get the best results with Copilot...
• It is recommended to use a single slide master and rename it. If you need to use a template that contains multiple slide masters, you should ensure that each slide master complies with the guidelines in this article.

Neither of these examples are general showstoppers. However, the expectation of users and companies is that new solutions will bring benefits and simplifications rather than having to fulfill conditions. This is the kind of thing that puts people off in the first place.

But - and there is almost always a “but”

There is a silver lining on the horizon. The topic of GenAI is becoming more and more mainstream. Together with LinkedIn, Microsoft has published the 2024 Work Trend Index Annual Report.

The most important findings as to why GenAI will become established were summarized as follows:

• Employees expect AI in the workplace because they already know and use the apps from their private lives.
• AI raises the bar for employees and breaks down career barriers.
• A type of AI power user is emerging that will play a special role in the future.

For details see also: GPT - here to stay

The key point is...

... that GenAI is not a no-brainer in companies either. When Facebook came along and social intranets became a trend in companies at the same time, people often said: “Nobody needed training for Facebook either. So why should that be necessary for our social intranet?”

As many will remember, training and implementation concepts were necessary for social intranet projects to be successful. It's exactly the same with GenAI.

Ok, there are exceptions - here's one:

• AI can help to minimize less demanding tasks so that employees can focus on more important and essential activities.

This is derived from the 𝙀𝙢𝙥𝙡𝙤𝙮𝙚𝙚 𝙎𝙞𝙜𝙣𝙖𝙡𝙨 survey, which is conducted every six months to gain insights into employee wellbeing and productivity. The survey results show that access to AI can increase employee productivity and engagement. Source and further details: The Key to a Thriving Workforce? A Smart Approach to AI.

Mind you, “can” and not “must”. And without an adoption plan, this will only apply to a few committed employees.

Companies and IT departments are lagging behind the trend

IT departments, innovation drivers and strategy departments in companies are often still struggling with the switch to cloud solutions with their evergreen approach. In addition, there are the aspects that were explained at the beginning of the article. And now there is also the new topic of GenAI.

Another point is that the self-awareness of employees has changed. Until a few years ago, users used the IT solutions that were made available to them. Today, the motto is increasingly: Isn't there an app for this that we can download and use? In the context of GenAI: ChatGPT <-> Microsoft Copilot or Google Gemini etc.

When users take IT into their own hands and how to deal with it

IT departments can no longer reduce themselves to technology alone. The “strategic consulting” factor within the company is the key to success and acceptance among employees. Multi-speed IT approaches such as “Information Competence Centers” have proven their worth. However, such approaches also require IT or an “Information Competence Center” to take on a new / different role in the organizational chart.

References:

• IT and administration – a dying breed, or future-proof even in the cloud era?
• The cover term 'digital transformation' – or: When the department takes IT into its own hands

Chat with a Video

CBS News: Here are six notable passages from former President Barack Obama's keynote address Tuesday night, Day 2 of the Democratic National Convention 2024: The YouTube video isn't long. It's just 13:23 minutes long. In it, Barak Obama criticizes Donald Trump's presidency and highlights concerns about his approach to governing. Source: 6 moments from Barack Obama's speech at the 2024 DNC

Because the video is on YouTube, it would also have been an option to use the ChatGPT for YouTube app to chat with the video.

But what if it's about internal company videos or videos with confidential content that you don't want to upload to YouTube - then the combination of Microsoft Stream & Copilot in M365 is a solution for working with a video using genAI technology.

Step by step

Once the video has been uploaded to Stream, the transcription will start automatically.

Once this has been completed, prompts such as the following can be used:

• Summarize the video
• List the action items

Or, even though the video is in English, prompts like:

• Fasse zusammen, was über Donald Trump gesagt wird (eng: Summarize what is said about Donald Trump)

Of course you can also watch the whole video, which in this case is only 13 minutes long, and answer the questions yourself.

At the end of the day, the solution of uploading videos to Stream and then using Copilot to work with the videos is what the article A Smart Approach to AI means with the point “A key insight is that AI helps minimize monotonous tasks so that employees can focus on more important and essential activities”.

Retrieval-Augmented Generation - The metasearch engine in the age of AI

What is Retrieval Augmented Generation / RAG?

A nice analogy that also makes it clear what RAG is, is the concept of a metasearch engine. Here, the search query is forwarded to several other search engines. The results of all the requested services are then collected, processed and made available to the user. RAG is a technique in which an AI model is combined with other data sources in addition to the data in the LLM (Large Language Model) in order to generate more precise and contextually relevant answers. This is therefore a very similar approach to the metasearch engine. Even the two schematic diagrams of the technologys are similar:

RAG is used in this way in Microsoft 365 Copilot. To extend the capabilities of the AI, information is retrieved from various data sources and integrated into the response generation. This enables Copilot not only to access pre-trained data, but also to use current and specific information from other sources, including the data in the M365 Tenant. Access is via the Microsoft Graph. This also ensures that the underlying permission concept is always respected by the AI.

Source and further details: How Copilot for Microsoft 365 works: A deep dive

Copilot in Microsoft 365 uses RAG - this cannot be customized

In Microsoft 365 Copilot, RAG is used to improve responses to user queries. Copilot can access various data sources, such as documents, emails, Teams chats, etc., to provide well-grounded and accurate answers.

This also determines which functions / roles Copilot provides in the respective apps.

Examples:

• Word: Generate text with and without formatting in new or existing documents.
• Excel: Suggestions for formulas, chart types and insights for data in Excel sheets.
• PowerPoint: Create a presentation from a prompt or a Word file.

Complete overview:

Source: Copilot integration with Graph and Microsoft 365 Apps

Now we have GraphRAG - that can be customized

The article Unlocking LLM discovery on narrative private data describes GraphRAG, a new method from Microsoft Research that extends the capabilities of large language models (LLMs) to access and analyze your data.

GraphRAG combines LLM-generated knowledge graphs with machine learning to improve document analysis performance, for example. This method shows significant improvements in answering complex questions compared to standard approaches.

A key benefit of GraphRAG is its ability to identify and understand topics and concepts in large data sets, even if the data was not previously known to the LLM. Here are some practical use cases for this technology:

• Information extraction: GraphRAG can be used to extract specific information from large document collections or databases.
• Content generation: GraphRAG helps to create content that requires in-depth contextual knowledge.
• Customer support: GraphRAG can improve customer support by accessing a knowledge base and providing accurate answers to customer queries.
• Knowledge management: In large organizations, GraphRAG can help to make efficient use of existing knowledge by retrieving and consolidating relevant information from different departments and documents.

Quickstart

To get started with the GraphRAG system (https://github.com/microsoft/graphrag), it is recommended to use the Solution Accelerator package (https://github.com/Azure-Samples/graphrag-accelerator). This offers a user-friendly end-to-end solution based on Azure resources, quote: One-click deploy of a Knowledge Graph powered RAG (GraphRAG) in Azure

The graphic shows, for example, the following sources for own solutions and GraphRAG:

• Azure Blob Storage
• Cosmos DB
• Azure OpenAI
• Azure AI Search / Vectorstore
• Container Registry
• Application Insights

As described on GraphRAG's GitHub page, Prompt Tuning options can also be used to customize the solution to your needs and use cases:

Source: GraphRAG Prompt Tuning

Turn your spaces into places

The preview for Microsoft Places has been available for some time now. However, it took a while for all the features to work.

All in all, it is a bit fiddly to activate and set up the preview. The steps are described in the following Microsoft articles:

• Activate Microsoft Places in the tenant:https://learn.microsoft.com/en-us/deployoffice/places/deployment-guide-for-places
• Set up buildings and floors: https://learn.microsoft.com/en-us/deployoffice/places/get-started/quick-setup-buildings-floors
• Set up desk pools and workspaces: https://learn.microsoft.com/en-us/deployoffice/places/setting-up-desk-workspaces-in-places-finder

Frank Carius has put it all together in one article: https://www.msxfaq.de/cloud/funktionen/microsoft_places.htm Thank you for that!

And of course Copilot in Microsoft 365 will also be able to work with the data from Microsoft Places. The data is stored in the backend in Exchange Online and is therefore available to Copilot via the Microsoft Graph. Prompts such as the following can then be realized by users.

• I need a room in the office in Berlin to meet Oliver on August 9th. What are the possibilities?
• Show me the available desks in the office in New York at Time Square for September 17. Please also list which desk has which equipment.

Source and further details on Microsoft Places, Copilot and Teams Rooms: AI brings new life to flexible work with Microsoft Places

Microsoft Places also offers new possibilities completely independently of Copilot. The feature is also integrated into Outlook, for example. There, the Places Finder is available in the Outlook calendar to schedule meetings:

In order to activate and use the preview of Microsoft Places, one of the following license packages must be available:

• Microsoft 365 Business Basic
• Microsoft 365 Business Standard
• Microsoft 365 Business Premium
• Microsoft 365 or Office 365 (E1, E3, E5)
• Microsoft 365 or Office 365 (A1, A3, A5)
• Microsoft 365 Frontline Worker (F1, F3)