@alvesfabi OBO Support (#803)

### Motivation and Context

<!-- Thank you for your contribution to the chat-copilot repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

@alvesfabi's work to implement the OBO flow for OpenAI Plugin Specs

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Fabian Alves <falves@microsoft.com>
Co-authored-by: Fabian Alves <50325768+alvesfabi@users.noreply.github.com>
Co-authored-by: Tao Chen <taochen@microsoft.com>

.vscode/tasks.json

@@ -131,4 +131,4 @@
             "password": true
         }
     ]
-}
+}

README.md

@@ -282,6 +282,28 @@ By default, Chat Copilot runs locally without authentication, using a guest user
    ./start.sh
    ```
 
+## Optional Configuration: [Ms Graph API Plugin with On-Behalf-Of Flow](./plugins/OBO/README.md)
+
+This native plugin enables the execution of Microsoft Graph APIs using the On-Behalf-Of (OBO) flow with delegated permissions.
+
+The OBO flows is used to ensure that the backend APIs are consumed with the identity of the user, not the managed identity or service principal of the middle-tier application (in this case the WebApi).
+
+Also, this ensures that consent is given, so that the client app (WebApp) can call the middle-tier app (WebApi), and the middle-tier app has permission to call the back-end resource (MSGraph).
+
+This sample does not implement incremental consent in the UI so all the Graph scopes to be used need to have "Administrator Consent" given in the middle-tier app registration.
+
+More information in the [OBO readme.md](./plugins/OBO/README.md).
+
+### Requirements
+
+Backend authentication via Azure AD must be enabled. Detailed instructions for enabling backend authentication are provided below.
+
+### Limitations
+
+- Currently, the plugin only supports GET operations. Future updates may add support for other types of operations.
+- Graph queries that return large results, may reach the token limit for the AI model, producing an error.
+- Incremental consent is not implemented in this sample.
+
 # Troubleshooting
 
 1. **_Issue:_** Unable to load chats.

plugins/OBO/README.md

@@ -0,0 +1,103 @@
+# Ms Graph plugin using On-Behalf-Of Flow for Ms Graph APIs
+
+This repository contains a sample Plugin that uses the On-Behalf-Of (OBO) flow to call Microsoft Graph APIs.
+
+In this document we will refer to the client app as the WebApp (src/webapp), the middle-tier app as the WebApi (src/webapi) and the backend resource as the Ms Graph Api.
+
+> **IMPORTANT:** This sample is for educational purposes only and is not recommended for production deployments.
+
+> **NOTE:** This plugin was implemented as a native Kernel function, in the WebAPI code. This is not an implementation of the OpenAI plugin spec.
+
+> **NOTE:** This plugin works better GTP-4 or GTP-4-Turbo as these models works better with the function model.
+
+## Prerequisites
+
+- Enable backend authentication via Azure AD as described in the main [`README.md`](../../README.md) file.
+
+## Setup Instructions
+
+1. **Add the WebApp to the "known client application list" in the WebApi app registration.**
+
+   - Go to the WebApp app registration in your tenant and copy the Application Id (Client ID).
+   - Go to the WebAPI app registration in your tenant.
+   - Click on "Manifest" option and add an entry for the `knownClientApplications` attribute using the Application Id (Client ID) of the WebApp registration as described in this [document](https://learn.microsoft.com/en-us/entra/identity-platform/reference-app-manifest#knownclientapplications-attribute)
+
+   - Save the manifest.
+
+2. **Give the WebApi the delegated permissions.**
+
+   - Go to the WebApi API app registration.
+   - Select the "API permissions" option.
+   - Click on "+ Add Permission" option and choose the "Microsoft Graph" option.
+   - Select "Delegated permission" and choose all the delegated permissions needed.
+   - Click on "Add Permissions".
+   - As the UI does not implement incremental consent, you need to grant "Admin Consent" to the new permissions added.
+
+3. **Create a Client Secret for the WebAPI app registration OBO Configuration.**
+
+   - In the WebAPI app registration click on "Certificates & Secrets".
+   - Create a new secret by clicking in the "+ New client secret", enter a description and the expiration days.
+   - Copy the Client Secret and the Application Id (Client ID) to use in the WebAPI appsetting configuration.
+
+4. **Change the WebAPI `appsettings.json` file.**
+   - Add your OBO configuration values in the OnBehalfOfAuth section as shown below. The ClientId must be the WebAPI Application Id (Client ID).
+
+```json
+    // OBO Configuration for Plugins
+    "OnBehalfOfAuth": {
+      "Authority": "https://login.microsoftonline.com",
+      "TenantId": "[ENTER YOUR TENANT ID]",
+      "ClientId": "[ENTER YOUR CLIENT ID]",
+      "ClientSecret": "[ENTER YOUR CLIENT SECRET]"
+    }
+```
+
+5. Change the scope for the Ms Graph Obo plugin in the WebApp code
+
+   - As the UI does not implement incremental consent, you need to configure the WebApp to use the [.default scope](https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-on-behalf-of-flow#default-and-combined-consent). The scope name is formed by the Application ID of the WebAPI app registration so you need to update it with the WebApi Application ID (Client ID).
+
+   - Change the Constants.ts file located in the webapp/src folder, add the msGraphOboScopes entry with the WebApi Application Id, as shown below:
+
+   ```typescript
+   plugins: {
+      msGraphOboScopes: ['api://[ENTER THE WE API APPLICATION ID]/.default'],
+    }
+   ```
+
+## Test Instructions
+
+1. Login to the app
+
+![Login Step](./test-step-1.png)
+
+2. Enable Ms Graph OBO Plugin
+
+![Plugin Step](./test-step-2.png)
+
+![Plugin Step 2](./test-step-3.png)
+
+![Plugin Step 3](./test-step-4.png)
+
+3. Update the Persona Meta Prompt with the following text:
+
+   ```text
+   This is a chat between an intelligent AI bot named Copilot and one or more participants. SK stands for Semantic Kernel, the AI platform used to build the bot. The AI was trained on data through 2021 and is not aware of events that have occurred since then. The bot has the ability to call Graph APIs using the MS Graph OBO tool to fetch real-time data. The user must first enable the plugin. To call a Graph API, the bot would call the \\"CallGraphApiTasksAsync\\" function, and provide the Graph API URL with the ODATA query and its required scopes as a list as arguments. The plugin will automatically handle authentication. Otherwise, the bot has no ability to access data on the Internet, so it should not claim that it can or say that it will go and look things up. Try to be concise with your answers, though it is not required. Knowledge cutoff: {{$knowledgeCutoff}} / Current date: {{TimePlugin.Now}}.
+   ```
+
+![Persona Step 1](./test-step-5.png)
+
+4. Run a prompt to check if the bot understands that can can a graph API and then ask to run a query by providing a sample
+
+- Hi! Can you call a graph API for me?
+
+- Please get the list of applications in my tenant.
+  You can call the Graph API: `https://graph.microsoft.com/v1.0/applications$select=appId,identifierUris,displayName,publisherDomain,signInAudience`
+  Required scope: Application.Read.All
+
+![Check Step 1](./test-step-6.png)
+
+5. After the sample prompt the bot will execute any graph api query without the need of indicating the graph api, odata query or scopes
+
+- Please get the ObjectID of my user
+
+![Check Step 2](./test-step-7.png)

scripts/Start.ps1

@@ -9,8 +9,8 @@ $cmd = get-command 'pwsh'
 $ErrorActionPreference = 'Continue'
 
 if (!$cmd) {
-    Write-Warning "Please update your powershell installation: https://aka.ms/powershell"
-    return;
+  Write-Warning "Please update your powershell installation: https://aka.ms/powershell"
+  return;
 }
 
 $BackendScript = Join-Path "$PSScriptRoot" 'Start-Backend.ps1'
@@ -41,12 +41,13 @@ while ($backendRunning -eq $false -and $retryCount -lt $maxRetries) {
 if ($backendRunning -eq $true) {
   # Start frontend (in current PS process)
   & $FrontendScript
-} else { 
+}
+else { 
   # otherwise, write to the console that the backend is not running and we have exceeded the number of retries and we are exiting
   Write-Host "*************************************************"
   Write-Host "Backend is not running and we have exceeded "
   Write-Host "the maximum number of retries."
   Write-Host ""
   Write-Host "Therefore, we are exiting."
   Write-Host "*************************************************"
-}
+}

webapi/Controllers/ChatController.cs

@@ -47,6 +47,8 @@ public class ChatController : ControllerBase, IDisposable
     private readonly List<IDisposable> _disposables;
     private readonly ITelemetryService _telemetryService;
     private readonly ServiceOptions _serviceOptions;
+    private readonly MsGraphOboPluginOptions _msGraphOboPluginOptions;
+    private readonly PromptsOptions _promptsOptions;
     private readonly IDictionary<string, Plugin> _plugins;
 
     private const string ChatPluginName = nameof(ChatPlugin);
@@ -58,13 +60,17 @@ public ChatController(
         IHttpClientFactory httpClientFactory,
         ITelemetryService telemetryService,
         IOptions<ServiceOptions> serviceOptions,
+        IOptions<MsGraphOboPluginOptions> msGraphOboPluginOptions,
+        IOptions<PromptsOptions> promptsOptions,
         IDictionary<string, Plugin> plugins)
     {
         this._logger = logger;
         this._httpClientFactory = httpClientFactory;
         this._telemetryService = telemetryService;
         this._disposables = new List<IDisposable>();
         this._serviceOptions = serviceOptions.Value;
+        this._msGraphOboPluginOptions = msGraphOboPluginOptions.Value;
+        this._promptsOptions = promptsOptions.Value;
         this._plugins = plugins;
     }
 
@@ -214,6 +220,12 @@ private async Task RegisterFunctionsAsync(Kernel kernel, Dictionary<string, stri
             tasks.Add(this.RegisterMicrosoftGraphPlugins(kernel, GraphAuthHeader));
         }
 
+        // Microsoft Graph OBO
+        if (authHeaders.TryGetValue("MSGRAPHOBO", out string? GraphOboAuthHeader))
+        {
+            tasks.Add(this.RegisterMicrosoftGraphOBOPlugins(kernel, GraphOboAuthHeader));
+        }
+
         if (variables.TryGetValue("customPlugins", out object? customPluginsString))
         {
             tasks.AddRange(this.RegisterCustomPlugins(kernel, customPluginsString, authHeaders));
@@ -263,6 +275,15 @@ private Task RegisterMicrosoftGraphPlugins(Kernel kernel, string GraphAuthHeader
         return Task.CompletedTask;
     }
 
+    private Task RegisterMicrosoftGraphOBOPlugins(Kernel kernel, string GraphOboAuthHeader)
+    {
+        this._logger.LogInformation("Enabling Microsoft Graph OBO plugin(s).");
+        kernel.ImportPluginFromObject(
+            new MsGraphOboPlugin(GraphOboAuthHeader, this._httpClientFactory, this._msGraphOboPluginOptions, this._promptsOptions.FunctionCallingTokenLimit, this._logger),
+            "msGraphObo");
+        return Task.CompletedTask;
+    }
+
     private IEnumerable<Task> RegisterCustomPlugins(Kernel kernel, object? customPluginsString, Dictionary<string, string> authHeaders)
     {
         CustomPlugin[]? customPlugins = JsonSerializer.Deserialize<CustomPlugin[]>(customPluginsString!.ToString()!);

webapi/Extensions/ServiceExtensions.cs

@@ -59,6 +59,8 @@ public static IServiceCollection AddOptions(this IServiceCollection services, Co
 
         AddOptions<FrontendOptions>(FrontendOptions.PropertyName);
 
+        AddOptions<MsGraphOboPluginOptions>(MsGraphOboPluginOptions.PropertyName);
+
         return services;
 
         void AddOptions<TOptions>(string propertyName)

webapi/Options/MsGraphOboPluginOptions.cs

@@ -0,0 +1,24 @@
+// Copyright (c) Microsoft. All rights reserved.
+
+namespace CopilotChat.WebApi.Options;
+
+public class MsGraphOboPluginOptions
+{
+    public const string PropertyName = "OnBehalfOf";
+    /// <summary>
+    /// The authority to use for OBO Auth.
+    /// </summary>
+    public string? Authority { get; set; }
+    /// <summary>
+    /// The Tenant Id to use for OBO Auth.
+    /// </summary>
+    public string? TenantId { get; set; }
+    /// <summary>
+    /// The Client Id to use for OBO Auth.
+    /// </summary>
+    public string? ClientId { get; set; }
+    /// <summary>
+    /// The Client Secret to use for OBO Auth.
+    /// </summary>
+    public string? ClientSecret { get; set; }
+}

webapi/Options/PromptsOptions.cs

@@ -25,6 +25,11 @@ public class PromptsOptions
     /// </summary>
     [Required, Range(0, int.MaxValue)] public int ResponseTokenLimit { get; set; }
 
+    /// <summary>
+    /// The token count allowed for function calling responses.
+    /// </summary>
+    [Required, Range(0, int.MaxValue)] public int FunctionCallingTokenLimit { get; set; }
+
     /// <summary>
     /// Weight of memories in the contextual part of the final prompt.
     /// Contextual prompt excludes all the system commands and user intent.

webapi/Plugins/Chat/ChatPlugin.cs

@@ -582,10 +582,12 @@ private int GetMaxRequestTokenBudget()
         // "content": "Assistant is a large language model.","role": "system"
         // This burns just under 20 tokens which need to be accounted for.
         const int ExtraOpenAiMessageTokens = 20;
-
         return this._promptOptions.CompletionTokenLimit // Total token limit
             - ExtraOpenAiMessageTokens
-            - this._promptOptions.ResponseTokenLimit;   // Token count reserved for model to generate a response
+            // Token count reserved for model to generate a response
+            - this._promptOptions.ResponseTokenLimit
+            // Buffer for Tool Calls
+            - this._promptOptions.FunctionCallingTokenLimit;
     }
 
     /// <summary>

webapi/Plugins/Chat/CopilotChatPlanner.cs

@@ -0,0 +1 @@
+