How many Copilots are there? Fewer than you think

Why does Copilot in Microsoft 365, for example, also use data that is integrated into the M365 search via a connector, but Copilot in Word does not? Or to be more general: What is the difference between Copilot in Word and Copilot for Microsoft 365? Let's just ask him ourselves.
Prompt in Copilot in Word: What makes you different from Copilot in Microsoft 365?

The prompt was used with the "Draft with Copilot" feature:

The detail "Copilot in Word is more suitable for users who need more specific and detailed suggestions and feedback on their writing" in the generated text makes the difference clear. It says that Copilot in Word is a writing assistant. The specification that Copilot should act as a writing assistant here is like telling a prompt in ChatGPT, that the AI should act as a travel guide and that the focus should be on suggestions for a weekend in Rome.

The technology behind this is called Retrieval Augmented Generation. In general, every prompt that a user enters is pre-processed by the Copilot Orchestration. This is where the Retrieval Augmented Generation architecture is used. Although this step is not mentioned by name in the architecture diagram, it is explained in this video by Mary Pasch, Principal Product Manager at Microsoft for Copilot: How Copilot for Microsoft 365 works: A deep dive.

A term that is also frequently used is "grounding". In the end, it means the same, i.e. adapting the prompt entered by the user, respecting the context of the used app an combine these factors to achieve the best possible result. Details on this are shown in this overview:

Source and further details can be found in this article from Microsoft: Microsoft Copilot for Microsoft 365 overview.

Retrieval Augmented Generation (RAG)

Depending on the app from which the prompt is sent, the Retrieval Augmented Generation technology decides which information sources are used to generate an answer with the help of the LLM. This aspect, called Information Retrieval System, can also be used with Azure AI Search. The article Retrieval Augmented Generation (RAG) in Azure AI Search describes how the technology works, which is also used by Copilot Orchestration.

How Copilot for Microsoft 365 decides which plug-in to use

There is also a similar effect when using plug-ins. How does Copilot decide which Plug-Ins to use to respond to a prompt? This also happens in the Copilot orchestration. The article How Copilot for Microsoft 365 decides which plugin to use describes what should be done when creating Plug-Ins so that Copilot Orchestration can evaluate them properly. The app description is located in the app manifest:

Roundup

Although this does not answer the initial question of how many Copilots are there. But it is now clear that there are far fewer than you might think. Copilot in Microsoft 365, Copilot in Word, Copilot in Outlook etc. are all the same engine, which is only used differently through orchestration and pre- and post-processing. Solutions such as Copilot in GitHub etc., however, are independent solutions.

Lessons learned - Copilot and his attention deficit

Not quite on the ball, impulsive, too creative - this is how Copilot is often described by users. In the following example, this is exactly how the Copilot app behaves and how you can still achieve good results.

Starting point

Dona Sarkar spoke on "This Is Why We Can't Have Nice Things" at the European Cloud Summit 2024. According to the agenda:

So all we have heard about for the past year and a half is AI , AI and more AI. How do we know if this is something that will stick around or be yet another fad, for example: Bitcoin, blockchain, NFTs, augmented reality, etc. The key to being able to identify hype vs reality around AI is to develop tech intuition. This is the way to structure your career investments around things that will pay off versus wasting your time. Let's find out together how you can build this intuition step-by-step and develop this skill of seeing around corners.

What was explained in the presentation was 100% comprehensible and reflected the hype topics of recent years. Among other things, Donna Sarkar mentioned the 100 Bad Ideas method as an approach to analyzing hype topics for their future potential. The method has a lot in common with the Design Sprint and Repertory Grid methods. Copilot should now compare the 3 approaches and clarify when which of the approaches should be used.

The initial prompt

After the title for the document, Copilot should take care of the rest. The following initial prompt was used for this: 

Create an overview of the Design Sprint, 100 Bad Ideas and Repertory Grid methods. Explain the details of each method. Compare the methods in a matrix and explain the advantages of each method.

The generated text was good. However, the comparison of the methods was a little short and the column “Advantages” in the generated table was empty:

This is why additional prompts were used.

Additional prompts

Copilot offers the choice of accepting the generated text or adding a further prompt:

This results in the entire text being recreated. However, as the text should be retained, a new prompt was used:

Create a new chapter. Explain when which of the methods Design Sprint, 100 Bad Ideas or Repertory Grid should be used. Create a matrix in which the methods are compared. The matrix should be followed by a text that uses examples of each method to explain when which of the methods should be used.

Copilot then creates the chapter " Chapter: Comparing Methods for Creative Problem Solving". However, the "Disadvantages" and "When to use" columns of the table he generates do not contain any values. And there is a formatting error in the examples he lists, as if someone had typed too fast and mixed up the keys.

Copilot aborts

Finally, there should be a chapter explaining how the three methods can be used to work out use cases for generative AI solutions. Prompt:

In a new chapter, explain which of the three methods can be used to develop use cases for the topic of generative AI. Create a matrix with the advantages and disadvantages of each method. Create a summary.

Without an error message, Copilot terminated before the prompt was completely fulfilled. In addition, the table he created was incomplete and therefore useless:

The exact same prompt was entered again and Copilot generated the chapter. It is remarkable that even though the identical prompt was repeated in both runs, a different heading was created for the chapter. In the first run, wich was aborted, the heading was " Use Cases for Generative AI". In the second run, it was then " Chapter: Developing Use Cases for Generative AI".

Conclusion

The comparison with an assistant, who is not always fully attentive and somewhat impulsive, fits very well. Copilot produces useful results if you tell him very explicitly what to do. This also means that you may have to enter a prompt repeatedly without it being clear why it was aborted or why not all aspects of the prompt were fulfilled equally. To create good prompts for Copilot, you can use this structure as a guide:

Source and further details: Learn about Copilot prompts

Further good tips can be found in the article Best practices for custom instructions. In addition to the aspects "Be specific" and "Keep it simple", the following aspects are also explained here:

• Use examples: Provide examples to illustrate your instructions and help the copilot understand your expectations. Examples help the copilot generate accurate and relevant responses.
• Give the copilot an “out”: Give copilot an alternative path for when it's unable to complete the assigned task. This alternative path helps the copilot avoid generating false responses.

Here the result in German and in English:

 Übersicht über Design Sprint 100 Bad Ideas und Repertory Grid

 Overview of Design Sprint 100 Bad Ideas and Repertory Grid

Don't make me think - The challenges with Microsoft Copilot and Azure OpenAI

The answer is: Copilot

...but what was the question?

It's true, the hype is very present and, similar to Microsoft Teams before it, it affects all areas of daily work if you work in a Microsoft-influenced environment.

The title "Dont make me think" is borrowed from Steve Krug's book with the same name about web design. If you look at the topics covered in the book, you will quickly find parallels to the current situation with generative AI. This is even more impressive considering that the book was first published over 20 years ago.

Or as Wictor Wilén - Product Leader @ Microsoft described it so well in his post on LinkedIn:

If you need to be an engineer to use ChatGPT or Microsoft Copilot - then we failed!:

https://www.linkedin.com/feed/update/urn:li:activity:7168538964692353027/

Even if Wictor's context here was the topic of prompt engineering, the conclusion also applies to the topic of AI in general.

Topics / chapters in the whitepaper:

• Metadata: The hidden heroes of AI
• Managing metadata in SharePoint
• Example with metadata in SharePoint and Copilot in Microsoft 365
• The semantic index
• In a nutshell - What is a semantic index?
• Copilot in Microsoft 365 & data from other sources
• Copilot in Microsoft 365 - Lessons Learned
• Copilot Studio
• Copilot - A field report
• Hijack Copilot in Microsoft 365

Download English version: LINK

Download German version: LINK