Skip to content

Commit

Permalink
WMS ID 11631 (#158)
Browse files Browse the repository at this point in the history
* Apex-RAG

This lab is in production for testing purposes.

* update

* updts

* Update manifest.json

* Update apex.md

* Update apex.md

* Update apex.md

* updates

* Update ask.md

* Update tables.md

* updts

* updts

* .

* TEST

* Update download.md

* Update download.md

* f

* .

* Update apex-app.md

* updates

* Update download.md

* updates

* updates

* Delete setup.md

* Update download.md

* updates

* updates

* Update setup.md

* Update setup.md

* Update download.md
  • Loading branch information
bhenndricks authored Aug 7, 2024
1 parent 8d31c88 commit 9eecdf3
Show file tree
Hide file tree
Showing 124 changed files with 167 additions and 67 deletions.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,20 @@ Before we dive into the procedure, make sure you have the following:

## Task 1: Create Credential Object in Oracle ADB

First, create a credential object in your Oracle Autonomous Database that will store your Object Storage credentials. This is required for authenticating with Oracle Object Storage. Please set up your [secret keys](https://medium.com/@bhenndricks/secure-access-to-oracle-buckets-in-object-storage-a-step-by-step-guide-32f3242f35e2)

1. First, create a credential object in your Oracle Autonomous Database that will store your Object Storage credentials. This is required for authenticating with Oracle Object Storage.

Next head back to your ADB console, and select Database Actions and then SQL. Log in as ADMIN. This will open up an editor for us to perform statements.
![alt text](images/sqldev.png)

Copy this statement and replace with your user and password for Oracle Cloud.
Copy this statement and replace with your username and password for Oracle Cloud.

```sql
<copy>
BEGIN
DBMS_CLOUD.CREATE_CREDENTIAL(
credential_name => 'OBJ_STORE_CRED',
username => '<your_oci_user_name>',
password => '<your_oci_passwordy>'
password => '<your_oci_password>'
);
END;
/
Expand All @@ -64,17 +63,14 @@ GRANT CREATE ANY DIRECTORY TO VECTOR;
</copy>
```

## Task 3: Create the credentials for ADB to access the LLM service

To enable ADB to access an LLM service, API authentication is required.
## Task 3: Option 1 - Create the credential for ADB to access OCI GenAI Service

1. From ADB Database Actions SQL Worksheet, as ADMIN user enter and run the SQL below and replace the following with your ocid and key information you got from the previous lab.
### OCI GenAI Service
The OCI GenAI service provides access to several LLMs including Cohere and Llama.
API authentication is required.

Oracle's GenAI service is an LLM service from Oracle Cloud Infrastructure (OCI). The GenAI service provides access to several LLMs that you can pick from.
1. From ADB Database Actions SQL Worksheet, as ADMIN user enter and run the SQL below and replace the following with your ocid and key information you got from the previous lab. Important Note: Open your private key and copy the private key all onto a single line.

## OCI GenAI Service

For OCI GenAI Service, run the following procedure. Important Note: Put the private key all on a single line.
```sql
<copy>
declare
Expand All @@ -93,8 +89,27 @@ end;
/
</copy>
```
For example:
```
declare
jo json_object_t;
begin
jo := json_object_t();
jo.put('user_ocid','ocid1.user.oc1..aaaaaaaawfpzqgzsrvb4mh6hcld2hrckadyae5y...cvza');
jo.put('tenancy_ocid','ocid1.tenancy.oc1..aaaaaaaafj37mytx22oquorcznlfuh77...zrq');
jo.put('compartment_ocid','ocid1.compartment.oc1..aaaaaaaaqdp7dblf6tb3gpzbuknvgfgkedtio...yfa');
jo.put('private_key','MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCP1QXxJxzVj4SXozdfrfIr...A4Iw=');
jo.put('fingerprint','e3:e5:ab:61:99:51:29:1f:60:2a:ad...5b:a5');
dbms_vector.create_credential(
credential_name => 'GENAI_CRED',
params => json(jo.to_string));
end;
/
```

## Task 3: Option 2 - Create the credential for ADB to access OpenAI

## OpenAI
### OpenAI

For OpenAI, run the following procedure:
```sql
Expand All @@ -115,7 +130,7 @@ end;

## Task 4: Download ONNX embedding models Using DBMS\_CLOUD.GET\_OBJECTS

Now log in as `<your_database_user>`, use the DBMS\_CLOUD.GET\_OBJECTS procedure to download the ONNX embedding model files from your Oracle Object Storage bucket into Oracle ADB. You will download two different models.
Now log in as VECTOR or `<your_database_user>`, use the DBMS\_CLOUD.GET\_OBJECTS procedure to download the ONNX embedding model files from the Oracle Object Storage bucket into Oracle ADB. You will download two different models.

Run to create the staging directory.

Expand All @@ -125,12 +140,14 @@ CREATE DIRECTORY staging AS 'stage';
</copy>
```

Run to get the onnx models.

```sql
<copy>
BEGIN
DBMS_CLOUD.GET_OBJECT(
credential_name => 'OBJ_STORE_CRED',
object_uri => 'https://objectstorage.<region>.oraclecloud.com/n/<namespace>/b/<bucket>/o/<file>',
object_uri => '<URL to onnx model>',
directory_name => 'staging',
file_name => '<file_name_in_adb>'
);
Expand All @@ -139,14 +156,22 @@ END;
</copy>
```

For example, if your object URI is 'https://oraclepartnersas.objectstorage.us-ashburn-1.oci.customer-oci.com/p/HP5q2dfCzDstMprLYpR5x0LbhJb_SyxGNgHj985fd8GELKb9j2aLcEwUUpKmV7zW/n/oraclepartnersas/b/onnx/o/tinybert.onnx', and you want to download it to ADB, the command will look like this:
URL to all-MiniLM-L6-v2.onnx is:
https://oraclepartnersas.objectstorage.us-ashburn-1.oci.customer-oci.com/p/CjS1gGPZaCZE2PoRWS5c6xmGNXK0v6ny6tNwoiVIOvqQrHux9NJ5oYo0dgLc6gOG/n/oraclepartnersas/b/onnx/o/all-MiniLM-L6-v2.onnx


URL to tinybert.onnx is:
https://oraclepartnersas.objectstorage.us-ashburn-1.oci.customer-oci.com/p/m5o31C0ol_8B_OzCLOLvqc2rWYNqz0M7kZZpMZHEaOyX7GQkhEw8_UNKoKBtcQYC/n/oraclepartnersas/b/onnx/o/tinybert.onnx


For example, to get tinybert.onnx and download it to ADB, the command will look like this:

```sql
<copy>
BEGIN
DBMS_CLOUD.GET_OBJECT(
credential_name => 'OBJ_STORE_CRED',
object_uri => 'https://oraclepartnersas.objectstorage.us-ashburn-1.oci.customer-oci.com/p/HP5q2dfCzDstMprLYpR5x0LbhJb_SyxGNgHj985fd8GELKb9j2aLcEwUUpKmV7zW/n/oraclepartnersas/b/onnx/o/tinybert.onnx',
object_uri => 'https://oraclepartnersas.objectstorage.us-ashburn-1.oci.customer-oci.com/p/m5o31C0ol_8B_OzCLOLvqc2rWYNqz0M7kZZpMZHEaOyX7GQkhEw8_UNKoKBtcQYC/n/oraclepartnersas/b/onnx/o/tinybert.onnx',
directory_name => 'staging',
file_name => 'tinybert.onnx'
);
Expand Down
Binary file added ai-vector-search-apex-adb/setup/images/ocid.png
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,36 @@

## Introduction

Welcome to Lab 1, before we begin the AI RAG steps we will guide you through the steps needed for APEX to access Oracle Object Storage. The lab focuses on two main areas: Identity Setup and Bucket Setup. By the end of this lab, you will have a functional compartment and policies for managing your cloud resources, as well as a properly configured bucket for storing and managing your objects in OCI.
Welcome to Lab 1. This workshop demonstrates how to set up Oracle Cloud Infrastructure (OCI) resources to enable Oracle APEX to access Oracle Object Storage. The lab focuses on two main areas: Identity Setup and Bucket Setup. By the end of this lab, you will have a functional compartment and policies for managing your cloud resources, as well as a properly configured bucket for storing and managing your objects in OCI.

In the Identity Setup section, you will learn how to create compartments, manage policies, and set up users with the necessary permissions to interact with OCI resources. Following this, the Bucket Setup section will walk you through the process of creating and configuring a storage bucket, essential for managing data and objects in your cloud environment.

Estimated Time: 10 minutes
**Estimated Time:** 10 minutes

## Objectives

By completing this lab, you will achieve the following objectives:

- Create and configure a new compartment for managing resources in OCI.
- Establish policies to control access and permissions within the compartment.
- Set up a user specifically for API key management and interaction.
- Configure API keys for secure access and operations.
- Obtain necessary OCIDs for compartments, users, and tenancies.
- Create and configure a storage bucket in OCI.
- Generate a Pre-Authenticated Request (PAR) for secure bucket access.
- **Create and configure a new compartment** for managing resources in OCI.
- This involves setting up a logical grouping for your resources to enhance management and organization.

- **Establish policies** to control access and permissions within the compartment.
- You will define and implement policies to manage who can access and perform actions on your OCI resources.

- **Set up a user specifically for API key management and interaction.**
- This step ensures that there is a dedicated user with appropriate permissions for managing and using API keys securely.

- **Configure API keys for secure access and operations.**
- You will generate and set up API keys to enable secure programmatic access to your OCI resources.

- **Obtain necessary OCIDs** for compartments, users, and tenancies.
- OCIDs (Oracle Cloud Identifiers) are essential for uniquely identifying your resources in OCI.

- **Create and configure a storage bucket in OCI.**
- This section guides you through setting up a storage bucket, a crucial component for storing and managing data in your cloud environment.

- **Generate a Pre-Authenticated Request (PAR) for secure bucket access.**
- You will create a PAR to enable secure and temporary access to your storage bucket without requiring further authentication.

### Task 1: Create an OCI Compartment

Expand All @@ -33,24 +45,6 @@ By completing this lab, you will achieve the following objectives:
6. Save the compartment OCID for later use.
![compartment save](/images/compartmentsave.png)

### Task 2: Create the Group and Policy

1. Click the hamburger icon (≡), select Identity, and click Groups.
2. Click Create Group.
3. Enter the following information in the corresponding fields:
1. **Name**: ProdObjectManagers
2. **Description**: Users who can manage objects in compartment PROD
4. Add the users that you want to the group as well you should see it listed in the configuration box.
5. Click Create to complete the process.

6. Create the ProdObjectManagementPolicies policy in the root compartment:
7. Click the hamburger icon (≡), Click Identity & Security, and then click Policies.
![Policies](/images/oci.png)
8. Enter the following information in the corresponding fields:
- **Name**: ProdObjectManagementPolicies
- **Description**: Policies for users to manage objects in compartment PROD
- **Policy Versioning**: Keep Policy Current

The members of the ProdObjectManagers group will require the ability to list the buckets in the compartment and manage any objects in these buckets.

9. Add the following statements to allow the respective policy actions:
Expand All @@ -59,7 +53,7 @@ The members of the ProdObjectManagers group will require the ability to list the
10. Click Create to complete the process.
![Policies](/images/policyconfiguration.png)

### Task 3: Create policy to enable access to OCI GenAI
### Task 2: Create policy to enable access to OCI GenAI

Oracle's GenAI service is an LLM service from Oracle Cloud Infrastructure (OCI). The GenAI service provides access to several LLMs that you can pick from.

Expand All @@ -78,22 +72,26 @@ Description: Public Gen AI Policy

Compartment: select your own compartment

Policy: allow any-user to manage generative-ai-family in compartment \<your compartment>
```
<copy>
Policy: allow user to manage generative-ai-family in compartment \<your compartment>
</copy>
```

3. Click Create.

![](images/create-policy.png " ")

Note: This policy allows any database in the specified compartment to access OCI Generative AI service. In a production environment, ensure your policy's scope is minimally inclusive.

### Task 4: Save User OCID and Create API Keys
### Task 3: Save User OCID and Create API Keys

1. Click the profile icon in the top right of your OCI page, and click My Profile.
![Users](/images/userprofile.png)
2. Save the user OCID for later use.
![alt text](/images/ocidsave.png)

### Task 5: Create OCI API Key
### Task 4: Create OCI API Key

1. Select API KEYS under the resources section and add api key.
![alt text](images/addapikey.png)
Expand All @@ -105,15 +103,15 @@ Note: This policy allows any database in the specified compartment to access OCI
5. Go ahead and close the tab after you have saved the contents.


### Task 6: Grab Your Parent Tenancy OCID
### Task 5: Grab Your Parent Tenancy OCID

1. Type tenancies in the OCI search bar, click the result, and then the tenancy name.
![alt text](images/tenancysearch.png)
2. Copy the tenancy OCID and save it in a separate document for later use.
![alt text](images/copytenancyid.png)


### Task 7: Create a Bucket
### Task 6: Create a Bucket

In Oracle Object Storage, a bucket is a container for storing objects. To access a bucket through the Oracle Cloud Infrastructure REST APIs, complete the following identity management tasks using the Oracle Cloud Infrastructure console and an account that has administrative rights within the tenancy (the root compartment).

Expand All @@ -130,7 +128,7 @@ In Oracle Object Storage, a bucket is a container for storing objects. To access
1. Click Create Bucket to complete the process.
2. To prepare for the successful creation and discovery of the WSM data profile later, upload a dummy file to the `apex_file_storage` bucket now using the Oracle Cloud Infrastructure console.

### Task 8: Create a Pre-Authenticated Request (PAR)
### Task 7: Create a Pre-Authenticated Request (PAR)

1. Click Create Pre-Auth Request on the resources section from the newly created bucket.
![alt text](images/createpreauth.png)
Expand Down
Binary file added ai-vector-search-apex-adb/tables/images/ocid.png
Loading

0 comments on commit 9eecdf3

Please sign in to comment.