In this post, I will try to explain the inner workings of Copilot and how we can preprocess prompts with additional data to generate better responses from the underlying Large Language Models. I will finish it off with an example of how this theory could be manifested and used in practice.

1. Background

1.1 Dynamics Customer Service Copilot

Surly you have heard about Dynamics Customer Service Copilot by now. The product comes in the way of a chat side pane integrated within Dynamics and lets you generate email replies, summarize cases, and generate responses based on knowledge articles. This opens up new ways to interact with data!

1.2 The problem

This chat pane is a great feature and lets the user interact with the system in a chat interface as well as being able to generate text for different purposes. However, I did find the need to be able to extend and adjust the Copilot functionality.

For example, I wanted to be able to summarize activities on any table and not only specific tables (cases for Customer Service Copilot). Also since I live in Sweden there was a need to get the generated text in Swedish.

This led me on the path of digging deeper into the art of prompt engineering. And how good prompt engineering can lead to better generative responses. Furthermore, I took this one step further and leveraged the underlying Copilot API to summarize activity on any table and with the help of the translation connector also in any language.

1.3 Generative AI concepts

1.3.1 Prompt engineering

First thing first, what is prompt engineering?

McKinsey has the following definition:

“Prompt engineering is the practice of designing inputs for generative AI tools that will produce optimal outputs.”

The article goes into detail, describing exactly what it is and it can be boiled down to this simple rule, the better input you can provide to the model, the better answers you will get back.

Microsoft recommend that prompts should be:

• Clear and concise
• Specific
• Contextual
• Relevant

1.3.2 Prompt flow - How Microsoft designed Dynamics Copilot

Microsoft’s Dynamics Copilot consists of an interface that takes a user text prompt as input. Based on specific keywords it then goes out to Dataverse and fetches additional information, generates a new prompt with this additional information - a thing called grounding - and sends it into the Large Language Model to get a flowing prose response back (Actually, I believe grounding also incorporates the use of vector search to find the most relevant information based on the user prompt, but this is out of scope for this post). So the Large Language Model in itself does not contain any information about Dataverse, it just gets an input with all information and regurgitates it back in whatever format we told it to. This is a concept that we can use to our advantage.

Extract about the inner workings of Copilot from Microsoft blog: “Copilot requests an input prompt from a business user in an app, like Microsoft Dynamics 365 Sales or Microsoft Power Apps. Copilot then preprocesses the prompt through an approach called grounding, which improves the specificity of the prompt, so you get answers that are relevant and actionable to your specific task. It does this, in part, by making a call to Microsoft Graph and Dataverse and accessing the enterprise data that you consent and grant permissions to use for the retrieval of your business content and context. We also scope the grounding to documents and data which are visible to the authenticated user through role-based access controls. For instance, an intranet question about benefits would only return an answer based on documents relevant to the employee’s role.”

With this knowledge, now we are ready to formulate a plan.

1.4 The plan

I wanted to create my own way of summarizing the activities on a record, where I can also extend the response with different data and external data sources if needed.

So I found out that there is a Custom API call made in the background every time we use Dynamics Copilot. It is called msdyn_InvokeIntelligenceAction. The plan was to create my own Custom API (called dse_CustomSummarizeActivitiesAPI). This Custom API will get data from Dynamics, in my scenario, related activities. But you could also get data from external sources if you want. From this data we will generate a new prompt, this is the grounding part since the new prompt contains our related activities. Lastly, the prompt will be fed into the msdyn_InvokeIntelligenceAction API and the underlying Large Language Model.

This Custom API will then be called from a Canvas App that I will embed in the form of a Model-driven app. So we will mimic the native “summarize activities on Case”-functionality in Dynamics Copilot, the difference being that this canvas app can summarize activities on any table and not just Case.

Additionally, since Copilot does not support Swedish yet, the Canvas app also connects to a power automate flow that can translate the generated output to any language.

2. The solution - a case study

So having a plan I went along and executed that plan. I tried to inject the canvas app in the form of a custom table called Meeting Report. The record had two phone call activities.

This is what the prompt looked like after it went through the grounding process:

You know the following: 
Activity of type: phonecall completed on: 12/19/2023 4:37:35 PM subject of activity is: Call to Gus Person description of actvity is:I had a call with Gus Person and we discussed potential products that they would like to buy from us 
Activity of type: phonecall completed on: 12/19/2023 4:38:57 PM subject of activity is: Recieved a call From Gus Person description of actvity is:Gus Person reached out to let me know he was not gonna buy our products and he will go with a competitor

Could you please generate a short summary of this information in english.

This processed prompt is then sent to msdyn_InvokeIntelligenceAction and we get a nicely summarized version of the same text. See the below gif:

The custom Copilot in action on a custom table called Meeting Report (press for bigger image)

I also tried to inject the app into the Case form and compared it to the standard summarize activity functionality. The output was not that far away.

The custom Copilot in action on the standard Case table (press for bigger image)

The out of box Copilot on the standard Case table

3. Summary

With more and more generative AI functionalities getting incorporated into the platform, there will also come a need for businesses to adapt and extend the functionality. This is also actually why Microsoft has released Copilot Studio. But if you have a Customer Service Enterprise license and do not want to spend additional money on generative AI this could be a good enough solution. I have shown how you can wrap the native copilot functionality in your own Custom API and hence extend its functionality. Furthermore, if needed you could chain this API with other data sources using Power Automate and hence make Power Automate a light version of Copilot Studio.

Feel free to leave a comment! Merry Christmas and a Happy New Year

• Dataverse
• Dynamics
• Copilot
• Custom API