feat: add collaboration and marketplace sections (#198)

* add docs * docs update * docs update * fixing broken links * fix links * fix broken links * fix broken link * add docs for apps * docs updates * docs update * add sub-section for resources * review * review * review * docs update * docs update * docs review * update images * review
epam · Oct 15, 2024 · 7a347f7 · 7a347f7
1 parent 003375e
commit 7a347f7
Show file tree

Hide file tree

Showing 43 changed files with 456 additions and 101 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -46,7 +46,7 @@ We aim to complement these services, streamlining the development, deployment, a
 
 - [Unified API](https://epam-rail.com/dial_api) to access multimodal models, assistants, applications, and embedding models.
 - Essential services such as resilient file/chat/prompt//model [storage](https://docs.epam-rail.com/architecture#persistent-layer) with user-based [access control](https://docs.epam-rail.com/Roles%20and%20Access%20Control/overview).
-- Important operational features, including [sharing](https://epam-rail.com/dial_api#tag/Sharing), organization-wide [publications](https://epam-rail.com/dial_api#tag/Publications), [review processes](https://docs.epam-rail.com/tutorials/enable-publications#admin-flow), and [rate limits](https://docs.epam-rail.com/architecture#rate-limits--cost-control).
+- Important operational features, including [sharing](https://epam-rail.com/dial_api#tag/Sharing), organization-wide [publications](https://epam-rail.com/dial_api#tag/Publications), [review processes](https://docs.epam-rail.com/tutorials/Collaboration/enable-publications#admin-flow), and [rate limits](https://docs.epam-rail.com/architecture#rate-limits--cost-control).
 
 ## Feature Validation Checklist
 

diff --git a/docs/architecture.md b/docs/architecture.md
@@ -48,6 +48,13 @@ The diagram below demonstrates the most complete landscape of AI DIAL ecosystem:
 
 ![max-zoom](./img/full-landscape3.svg)
 
+## Resources
+
+In AI DIAL, we call *resources* applications, conversations, prompts and files. When added to the system, they are stored in a dedicated folder within a blob store account bucket assigned to your user account. You can handle resources according to the similar pattern (CRUD operations, publications, sharing and other).
+
+> * In [DIAL API](https://epam-rail.com/dial_api) you can find dedicated sections for working with resources: [applications](https://epam-rail.com/dial_api#tag/Applications), [conversations](https://epam-rail.com/dial_api#tag/Conversations), [prompts](https://epam-rail.com/dial_api#tag/Prompts) and [files](https://epam-rail.com/dial_api#tag/Files).
+> * Refer to [Collaboration](./tutorials/collaboration/overview) to learn about collaboration features such as sharing and publication that can be applied to resources.
+
 ## AI DIAL Components
 
 ### AI DIAL Core
@@ -123,7 +130,7 @@ In AI DIAL Core, user roles are defined and configured in the application config
 
 ### Persistent Layer
 
-AI DIAL architecture includes a persistent layer, that relies on a resilient and scalable cloud blob storage (you can configure either AWS S3, Google Cloud Storage, Azure Blob Storage or a local file storage) where all conversations, prompts, and user files will be stored. Redis Cache (either cluster or a standalone) is deployed on top of it to enhance performance.
+AI DIAL architecture includes a persistent layer, that relies on a resilient and scalable cloud blob storage (you can configure either AWS S3, Google Cloud Storage, Azure Blob Storage or a local file storage) where all conversations, prompts, custom applications and user files will be stored. Redis Cache (either cluster or a standalone) is deployed on top of it to enhance performance.
 
 ![](./img/redis.svg)
 

diff --git a/docs/img/add-app.png b/docs/img/add-app.png
diff --git a/docs/img/addons2.png b/docs/img/addons2.png
diff --git a/docs/img/all-applications.png b/docs/img/all-applications.png
diff --git a/docs/img/create-pt.png b/docs/img/create-pt.png
diff --git a/docs/img/dial-marketplace.png b/docs/img/dial-marketplace.png
diff --git a/docs/img/dial-marketplace1.png b/docs/img/dial-marketplace1.png
diff --git a/docs/img/dial-marketplace2.png b/docs/img/dial-marketplace2.png
diff --git a/docs/img/dial-marketplace3.png b/docs/img/dial-marketplace3.png
diff --git a/docs/img/dial-marketplace5.png b/docs/img/dial-marketplace5.png
diff --git a/docs/img/home.png b/docs/img/home.png
diff --git a/docs/img/marketplace-home-select.png b/docs/img/marketplace-home-select.png
diff --git a/docs/img/math-prompt.png b/docs/img/math-prompt.png
diff --git a/docs/img/math-prompt1.png b/docs/img/math-prompt1.png
diff --git a/docs/img/model-version.png b/docs/img/model-version.png
diff --git a/docs/img/register-app.png b/docs/img/register-app.png
diff --git a/docs/img/register-app2.png b/docs/img/register-app2.png
diff --git a/docs/img/register-app3.png b/docs/img/register-app3.png
diff --git a/docs/img/register-app4.png b/docs/img/register-app4.png
diff --git a/docs/img/remove-app.png b/docs/img/remove-app.png
diff --git a/docs/img/select-app.png b/docs/img/select-app.png
diff --git a/docs/img/select-app2.png b/docs/img/select-app2.png
diff --git a/docs/img/select-item.png b/docs/img/select-item.png
diff --git a/docs/img/system_prompt.png b/docs/img/system_prompt.png
diff --git a/docs/img/talk-to-apps.png b/docs/img/talk-to-apps.png
diff --git a/docs/img/talk-to-assistants.png b/docs/img/talk-to-assistants.png
diff --git a/docs/img/talk-to-models.png b/docs/img/talk-to-models.png
diff --git a/docs/img/talk-to-models2.png b/docs/img/talk-to-models2.png
diff --git a/docs/img/talk-to.png b/docs/img/talk-to.png
diff --git a/docs/img/temperature.png b/docs/img/temperature.png
diff --git a/docs/tutorials/collaboration/1.overview.md b/docs/tutorials/collaboration/1.overview.md
@@ -0,0 +1,12 @@
+# Overview
+
+AI DIAL can serve as a collaborative and knowledge-sharing platform for the entire organization. To facilitate this, we provide a variety of features accessible through both the API and the chat user interface.
+
+> In AI DIAL, [resources](../../architecture#resources) refers to conversations, prompts, files, and applications. 
+
+* [Publication](enable-publications): you can publish resources making them accessible to the selected audience in your organization.
+* [Sharing](sharing): you can share your resources with a specific person.
+* [Notifications](notifications): you can receive notifications when specific events occur.
+
+
+> Watch a [demo video](../../video%20demos/demos/dial-collaboration) to see collaboration features in action.
diff --git a/docs/tutorials/collaboration/2.enable-publications.md b/docs/tutorials/collaboration/2.enable-publications.md
@@ -0,0 +1,70 @@
+# Publications
+
+You can publish/unpublish conversations, prompts (or even a collection of those) and applications, making them accessible within your organization. Additionally, you can manage access to the published [resources](../../architecture#resources), ensuring that the right information is accessible to the right individuals or teams.
+
+> Watch a [demo video](https://www.youtube.com/watch?v=GxdhfWLspa0) to see publication feature in action.
+
+## DIAL API
+
+Refer to [AI DIAL API](https://epam-rail.com/dial_api#tag/Publications) to view publications endpoints.
+
+### User Flow
+
+You can publish resources such as conversations, prompts, files or applications by calling [/v1/ops/publication/create](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1create/post) to create a publication request. You can also create a mixed publication request, in which you can pass various resources in the request body with different action types. For example, you can pass a collection of new prompts with action type `ADD` and another collection of prompts with action type `DELETE`. This way you can **publish** one set of prompts and **unpublish** the other. Similarly, you can handle other [resource types](../../architecture#resources). 
+
+In the response, you get an object with the `PENDING` status, which is awaiting the action from the admin: approve or reject. While your request is pending, you can delete it by calling [/v1/ops/publication/delete](#) endpoint.
+
+### Admin Flow
+
+As an administrator, you can get a list of publication requests awaiting your decision by calling [/v1/ops/publication/list ](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1list/post) endpoint. Further, you can call [/v1/ops/publication/get](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1get/post) endpoint to get a specific publication request and then execute [/v1/ops/publication/approve](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1approve/post) or [/v1/ops/publication/reject](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1reject/post) to change its status.
+
+### Working with Rules
+
+Use rules in publication requests to define or change access rules for specific folder(s). **Note**, that all resources (conversations, prompts, files, applications) placed in the root folder are always accessible to everyone. To apply access rules, it is necessary to create folders and apply rules to them.
+
+You can call [/v1/ops/publication/rules/list](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1rules~1list/post) endpoint to get a list of all rules for the provided path (folder sequence). Then, call a [/v1/ops/publication/create](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1create/post) endpoint providing a path to a desired folder (`targetFolder`in the request body) and a list of rules to be changed. For instance, if you create a request with `"targetFolder":"public/folder1/folder2/"`, it will only overwrite the rules for `folder2`, while the rules for `folder1` will stay unchanged.
+
+You can create a [/v1/ops/publication/create](https://epam-rail.com/dial_api#tag/Publications/paths/~1v1~1ops~1publication~1create/post) request without the `rules` object to left the rules intact.
+
+The effective access rules for a folder structure can be described as follows:
+
+1. Within a single folder, the effective access rule is determined by the logical OR operation applied to all the access rules assigned to that folder. For example, in folder A with access rules a, b, and c, the effective rule is "a OR b OR c".
+2. Between nested folders, the effective access rule is determined by the logical AND operation applied between the effective rules of the parent folder and its subfolder. For example, if folder B with access rules d, e, and f is nested under folder A, the effective rule for accessing folder B is "(a OR b OR c) AND (d OR e OR f)".
+
+![](../img/rules.svg)
+
+## DIAL Chat
+
+Refer to [user guide](../../user-guide#flow) to learn how chat users can use publications for conversations, prompts and applications.
+
+To enable the publication feature in AI DIAL Chat:
+
+### Step 1: Users
+
+Create user hierarchy (including claims) in your identity service provider (IDP) and configure AI DIAL to work with it. Refer to [Web Auth](../../Auth/Web/overview) to view the supported IDPs and configuration guidelines.
+
+### Step 2: AI DIAL Core
+
+Configure AI DIAL Core to pass claims from your IDP with the corresponding rules for users. Refer to [configuration](https://github.com/epam/ai-dial-core) to view the description of parameters.
+
+You can create your own rules according to access policies in your organization. 
+
+The following is the default configuration. In it, for admins, the value `admin` in the claim `roles` must be present. 
+
+```json
+"access": {
+  "admin": {
+    "rules": [
+      {
+        "function": "CONTAIN",
+        "source": "roles",
+        "targets": ["admin"]
+      }
+    ]
+  }
+}
+```
+
+### Step 3: AI DIAL Chat
+
+Configure AI DIAL Chat by including `ConversationsPublishing` and `PromptsPublishing` in the `ENABLED_FEATURES` variable. Refer to [configuration](https://github.com/epam/ai-dial-chat/blob/development/apps/chat/README.md) to view the description of parameters and [examples](https://github.com/epam/ai-dial-chat/blob/development/libs/shared/src/types/features.ts).
diff --git a/docs/tutorials/collaboration/3.sharing.md b/docs/tutorials/collaboration/3.sharing.md
@@ -0,0 +1,52 @@
+# Sharing
+
+## Introduction
+
+In AI DIAL, [resources](../../architecture#resources) refers to conversations, prompts, files, and applications. When a resource is created or uploaded to DIAL, it is stored in a dedicated [storage bucket](../../architecture#persistent-layer). You can share these resources with other users or applications. The sharing functionality is available through both the DIAL API and the AI DIAL Chat UI.
+
+## DIAL API
+
+There are two sides in the sharing process: the resource owner and the resource recipient. AI DIAL [Sharing API](https://epam-rail.com/dial_api#tag/Sharing) has endpoints to facilitate workflows for either of them.
+
+### Endpoints for resource owners
+
+* `/create`: resource owners can use this endpoint to share a resource by creating an `invitationLink` which includes an `invitationId`. On top of that, you can grant **read** or **write** access to your resource. **Note**: If an app/user changes the shared resource, HTTP **etag** is used by AI DIAL Core to determine versioning and resolve conflicts. You can use [Subscription API](/tutorials/collaboration/notifications#subscriptions) to get notifications about changes made by other users or applications. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1ops~1resource~1share~1create/post) to view examples. **Limitations**: the system does not support getting all the recipients of a shared resource.
+* `/copy`: this endpoint is used you need to add permissions to users for a new resource. For example, if you have a conversation with attachments that has been shared, and you decide to add another attachment, this endpoint ensures that other users can access the updated shared resource with the new attachment included.
+* `/list`: use this endpoint to query the details of all resources that were shared by you. In the request body, provide `others` as `with` parameter value to do that. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1ops~1resource~1share~1list/post) to view examples. **Note**, that in case you share a conversation with attachments, attachments are shared as well, but they will appear in the response as a separate shared resources.
+* `/revoke`: this endpoint is used when you need to stop sharing a resource. By using this endpoint to unshare a resource, the shared content becomes inaccessible to all recipients who previously had access, and any invitation link associated with it will no longer be valid for future use. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1ops~1resource~1share~1revoke/post) to view examples.
+* `/invitations`: invitations are created together with sharing resources using `/create` endpoint. Use `/invitations` endpoint to query a list of all active invitations you have created. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1invitations/get) to view examples.
+* `/invitations/{Invitation ID}`: use this endpoint with the **DELETE** method to revoke invitations. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1invitations~1%7BInvitation%20ID%7D/delete) to view examples.
+
+### Endpoints for resource recipients
+
+* `/discard`: recipients of a shared resource can use this endpoint to stop using shared resources. In this case, related resources will stop being marked as shared for the resource owner. Refer to  [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1ops~1resource~1share~1discard/post) to view examples.
+* `/list`: use this endpoint to query the details of all resources that were shared with you. In the request body, provide `me` as `with` parameter value to do that. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1ops~1resource~1share~1list/post) to view examples.
+* `/invitations/{Invitation ID}`: use this endpoint with the **GET** method to view invitation details and to **accept** an invitation. Refer to [DIAL API](https://epam-rail.com/dial_api#tag/Sharing/paths/~1v1~1invitations~1%7BInvitation%20ID%7D/get) to view examples.
+
+### Workflow Examples
+
+#### Basic workflow
+
+1. Resource owner shares a resource using the `/create` and provides the invitation link for the resource recipient.
+2. Resource recipient extracts the invitation id from the invitation link calls the `/invitations/{Invitation ID}` endpoint to accept the resource.
+3. Resource recipient can now get the list of shared resources with their details via `/list`.
+
+#### Revoke workflow
+
+1. Resource owner shares a resource via `/create` and provides the invitation link for the resource recipient.
+2. Resource recipient extracts the invitation id from the invitation link and calls the `/invitations/{Invitation ID}` to accept the resource.
+3. Resource owner revokes access to the shared resource via `/revoke`.
+4. Resource recipient cannot access the shared resource any longer.
+
+## DIAL Chat
+
+Chat users can share their resources with others, accept sharing invitations, and view resources that have been shared by or with them. When conversations that include attachments are shared, all attachments are also shared and can be accessed through the chat file manager.
+
+Refer to AI DIAL Chat user guide for details: 
+
+* [Conversations/Share](../../user-guide#share)
+* [Conversations/Unshare](../../user-guide#unshare)
+* [Prompts/Share](../../user-guide#share-1)
+* [Prompts/Unshare](../../user-guide#unshare-1)
+* [Attachments Manager](../../user-guide#attachments-manager)
+
diff --git a/docs/tutorials/collaboration/4.notifications.md b/docs/tutorials/collaboration/4.notifications.md
@@ -0,0 +1,21 @@
+# Notifications
+
+## Introduction
+
+AI DIAL API support two slightly different types of notifications: notifications and subscribe.
+
+> Note, that currently this functionality is available only via API. We are working on supporting it in AI DIAL Chat UI.
+
+## Notification API
+
+You can use [Notification API](https://epam-rail.com/dial_api#tag/Notifications/paths/~1v1~1ops~1notification~1list/post) to receive notifications about changes in specific **workflows** such as [publications](enable-publications). For example, when a publication request has been approved or rejected. 
+
+Notification API allows [getting all your notifications](https://epam-rail.com/dial_api#tag/Notifications/paths/~1v1~1ops~1notification~1list/post) and [deleting](https://epam-rail.com/dial_api#tag/Notifications/paths/~1v1~1ops~1notification~1delete/post) them.
+
+This API has a **pull-based** approach: you need to actively make a request to retrieve the notifications.
+
+## Subscribe API
+
+[Subscribe API](https://epam-rail.com/dial_api#tag/Notifications/paths/~1v1~1ops~1resource~1subscribe/post) allows receiving notifications about changes such as DELETE, CREATE, UPDATE made to DIAL resources such as applications, files, conversations and prompts.
+
+This API has a **push-based** approach: once you subscribe, the API will automatically send notifications whenever a relevant event occurs.
diff --git a/docs/tutorials/enable-publications.md b/docs/tutorials/enable-publications.md
@@ -31,7 +31,7 @@ The effective access rules for a folder structure can be described as follows:
 1. Within a single folder, the effective access rule is determined by the logical OR operation applied to all the access rules assigned to that folder. For example, in folder A with access rules a, b, and c, the effective rule is "a OR b OR c".
 2. Between nested folders, the effective access rule is determined by the logical AND operation applied between the effective rules of the parent folder and its subfolder. For example, if folder B with access rules d, e, and f is nested under folder A, the effective rule for accessing folder B is "(a OR b OR c) AND (d OR e OR f)".
 
-![](img/rules.svg)
+![](./img/rules.svg)
 
 ## DIAL Chat