diff --git a/docs/Data offloading/README.md b/docs/Data offloading/README.md index cac64a74..5bce62c8 100644 --- a/docs/Data offloading/README.md +++ b/docs/Data offloading/README.md @@ -1,24 +1,121 @@ # Data offloading -!!! Note - TODO - this section needs detailed review +There are multiple options to copy/offload data from a SkySQL DB. You can do a logical dump(i.e. output all data and DDL as SQL) to your local machine. Or, dump large data sets securely using the SkySQL Backup service to your own S3 or GCS bucket. +You can then use the offloaded data to resurrect the DB elsewhere. You can also optionally setup "outbound replication" to keep the new DB in sync with SkySQL. -## **Using Mariadb-dump utility** +## **(1) Offload your Database using `mariadb-dump`** -The simplest way to offload your database is to use the `mariadb-dump` utility: +The [`mariadb-dump`](https://mariadb.com/kb/en/mariadb-dump/) utility is a powerful command-line tool that allows you to export databases, tables, or specific data from your MariaDB instance in SkySQL. -- The `mariadb-dump` utility provides a command-line interface (CLI) -- The `mariadb-dump` utility is available for Linux and Windows -- The `mariadb-dump` utility supports [many command-line options](https://mariadb.com/docs/skysql-dbaas/ref/mdb/cli/mariadb-dump/) -- Egress charges may apply for customer-initiated backups +### **Prerequisites** +Ensure you have the mariadb-dump utility installed on your system. See [here](../Data%20loading,%20Migration/Install%20Mariadb-dump.md) +Obtain the necessary connection details for your SkySQL instance, including the host, username, and password. -`mariadb-dump` usage is fully described [here]( -'Data loading, Migration 8d6075f7123a4b59bedfeaad01d8461a'/'Install Mariadb-dump 14d92b62ce0f42cd83557aa80dd8c049'.md). +### **Exporting All Databases** -## **Using MariaDB client** +To export all databases from your SkySQL instance, use the following command: -Use [MariaDB Client](https://mariadb.com/docs/skysql-previous-release/connect/clients/mariadb-client/) with the connection information to export your schema from your MariaDB SkySQL database service. Here is an example to export all rows from a single table: +```sql +mariadb-dump -h your_skysql_host -u your_username -p --all-databases > all_databases_backup.sql + +``` +- `-h your_skysql_host`: Specifies the host of your SkySQL instance. +- `-u your_username`: Specifies the username to connect to the SkySQL instance. +- `-p`: Prompts for the password for the specified username. +- `--all-databases`: Exports all databases in the SkySQL instance. +- `> all_databases_backup.sql`: Redirects the output to a file named all_databases_backup.sql. + + +### **Exporting Selected Databases** + +To export specific databases, list the database names after the connection details: + +```sql +mariadb-dump -h your_skysql_host -u your_username -p database1 database2 > selected_databases_backup.sql +``` + +- `database1 database2`: Replace with the names of the databases you want to export. +- ``> selected_databases_backup.sql`: Redirects the output to a file named selected_databases_backup.sql. + + +### **Exporting Just the Schema** +To export only the schema (structure) of a database without the data, use the --no-data option: + +```sql +mariadb-dump -h your_skysql_host -u your_username -p --no-data your_database > schema_backup.sql + +``` +- `--no-data`: Ensures that only the schema is exported, not the data. +- `your_database`: Replace with the name of the database whose schema you want to export. +- `> schema_backup.sql`: Redirects the output to a file named schema_backup.sql. + + +### **File Format of Exported Data** +The files exported by mariadb-dump are in plain text format and contain SQL statements. These files can be used to recreate the databases, tables, and data by executing the SQL statements in a MariaDB instance. + +- Database Creation: The file begins with statements to create the databases. +- Table Creation: For each table, the file includes CREATE TABLE statements that define the table structure. +- Data Insertion: If data is included, the file contains INSERT INTO statements to populate the tables with data. +- Comments: The file may include comments that provide additional information about the export process. + +**Example of Exported File Content** + +Here is a snippet of what an exported file might look like: +```sql +-- MariaDB dump 10.16 Distrib 10.5.9-MariaDB, for debian-linux-gnu (x86_64) +-- +-- Host: your_skysql_host Database: +-- ------------------------------------------------------ +-- Server version 10.5.9-MariaDB-1:10.5.9+maria~focal + +/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; +/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; +/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; +/*!40101 SET NAMES utf8mb4 */; + +-- +-- Database: `your_database` +-- + +-- -------------------------------------------------------- + +-- +-- Table structure for table `your_table` +-- + +DROP TABLE IF EXISTS `your_table`; +/*!40101 SET @saved_cs_client = @@character_set_client */; +/*!40101 SET character_set_client = utf8 */; +CREATE TABLE `your_table` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `name` varchar(255) DEFAULT NULL, + `created_at` datetime DEFAULT NULL, + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; +/*!40101 SET character_set_client = @saved_cs_client */; + +-- +-- Dumping data for table `your_table` +-- + +LOCK TABLES `your_table` WRITE; +/*!40000 ALTER TABLE `your_table` DISABLE KEYS */; +INSERT INTO `your_table` VALUES (1,'Example Name','2023-01-01 00:00:00'),(2,'Another Name','2023-01-02 00:00:00'); +/*!40000 ALTER TABLE `your_table` ENABLE KEYS */; +UNLOCK TABLES; +``` + +Finally, here is the reference to the utility where you will find all the [many command-line options](https://mariadb.com/docs/skysql-dbaas/ref/mdb/cli/mariadb-dump/) + +> **Note** +> +> Egress charges may apply when data is exported + + +## **(2) Using MariaDB client** + +Use [MariaDB Client](../Connecting%20to%20Sky%20DBs/README.md) with the connection information to export your schema from your MariaDB SkySQL database service. Here is an example to export all rows from a single table: ```sql mariadb --host FULLY_QUALIFIED_DOMAIN_NAME --port TCP_PORT \ @@ -34,22 +131,89 @@ mariadb --host FULLY_QUALIFIED_DOMAIN_NAME --port TCP_PORT \ - Replace `FULLY_QUALIFIED_DOMAIN_NAME` with the Fully Qualified Domain Name of your service. - Replace `TCP_PORT` with the read-write or read-only port of your service. - Replace `DATABASE_USER` with the default username for your service, or the username you created. -- Replace `~/PATH_TO_PEM_FILE` with the path to the certificate authority chain (.pem) file. -- Optionally, for large tables, specify the [-quick command-line option](https://mariadb.com/docs/skysql-previous-release/ref/mdb/cli/mariadb/quick/) to disable result caching and reduce memory usage. - -## **Select into Outfile** +- Optionally, for large tables, specify the `-quick command-line option` to disable result caching and reduce memory usage. +- You can customize the SQL along with providing multiple SQL statements to `-execute`. + + +## **(3) Exporting Data Using SkySQL Backup Service API to S3 or GCS Bucket** +The [SkySQL Backup service API](../Backup%20and%20Restore/README.md) allows you to perform logical and physical dumps of your SkySQL databases to external storage buckets such as Amazon S3 or Google Cloud Storage (GCS). + +### **Prerequisites** +- Obtain the necessary credentials for your S3 bucket. +- Ensure you have access to the SkySQL Backup service API. You need to generate the API Key from the portal. +- Obtain the service ID for your SkySQL instance. +- Base64 encode your S3 credentials. + +### **Performing a Logical Dump to an S3 Bucket** +To perform a logical dump of a SkySQL database to an S3 bucket, you need to make an API call to the SkySQL Backup service. Below is an example of how to do this. + +Example API Call for Logical Dump (The output is all SQL statements) +```shell +curl --location 'https://api.skysql.com/skybackup/v1/backups/schedules' \ +--header 'Content-Type: application/json' \ +--header 'Accept: application/json' \ +--header 'X-API-Key: ${API_KEY}' \ +--data '{ + "backup_type": "logical", + "schedule": "once", + "service_id": "your_service_id", + "external_storage": { + "bucket": { + "path": "s3://your_s3_bucket_name/path/to/backup", + "credentials": "your_base64_encoded_credentials" + } + } +}' +``` -you can use the **`SELECT INTO OUTFILE`** statement to export query results directly to a file. This is useful for exporting specific data from tables. +- backup_type: Set to "logical" for a logical dump. +- schedule: Set to "once" to schedule the backup immediately. +- service_id: The ID of your SkySQL service. +- external_storage.bucket.path: The S3 bucket path where the backup will be stored. +- external_storage.bucket.credentials: Base64 encoded S3 credentials. + +### **Performing a Physical Dump to an S3 Bucket** + +**When databases are large and you want to move the data around securely this is likely the best option.** To perform a physical dump of a SkySQL database to an S3 bucket, you need to make a similar API call but specify the backup type as "physical". + +Example API Call for Physical Dump +```shell +curl --location 'https://api.skysql.com/skybackup/v1/backups/schedules' \ +--header 'Content-Type: application/json' \ +--header 'Accept: application/json' \ +--header 'X-API-Key: ${API_KEY}' \ +--data '{ + "backup_type": "physical", + "schedule": "once", + "service_id": "your_service_id", + "external_storage": { + "bucket": { + "path": "s3://your_s3_bucket_name/path/to/backup", + "credentials": "your_base64_encoded_credentials" + } + } +}' +``` +- backup_type: Set to "physical" for a physical dump. +- schedule: Set to "once" to schedule the backup immediately. + +### **Checking the Status of Initiated Backups** +Backups are always scheduled as jobs and the time taken will depend on the size of yourDB. To check the status of the initiated backups, you can use the following API call: + +```shell +Example API Call to Check Backup Status +curl --location 'https://api.skysql.com/skybackup/v1/backups/status' \ +--header 'Content-Type: application/json' \ +--header 'Accept: application/json' \ +--header 'X-API-Key: ${API_KEY}' \ +--data '{ + "service_id": "your_service_id" +}' +``` +- service_id: The ID of your SkySQL service. +This API call will return the status of the backups, including whether they are in progress, completed, or failed. -- **Usage:** - - ```sql - SELECT * INTO OUTFILE '/path/to/file.csv' FIELDS TERMINATED BY ',' FROM your_table; - ``` - -!!! Note - Note that this file system export is not directly accessible to SkySQL users. To generate CSV files, you should contact SkySQL support. +## [**(4) Replicating changes from SkySQL to a compatible external DB**](./Replicating%20data%20from%20SkySQL%20to%20external%20database.md) -## [**Replicating data from SkySQL to a compatible external DB**](Replicating data from SkySQL to external database 65a9d8aa0dc040c09548c042ba6582ae.md) diff --git a/docs/Scaling horizontally, HA.md b/docs/Scaling horizontally, HA.md index e146b0b0..2452cdf3 100644 --- a/docs/Scaling horizontally, HA.md +++ b/docs/Scaling horizontally, HA.md @@ -58,7 +58,7 @@ In SkySQL you can control routing using 2 strategies: - Using the ['hint filter'](https://mariadb.com/kb/en/mariadb-maxscale-24-hintfilter/) (TODO: provide detailed example using SkySQL node names) -## **Level 3 Resiliency - Disaster Recovery – Across Regions, Cloud Providers, or “Self-managed” Environments ** +## **Level 3 Resiliency - Disaster Recovery – Across Regions, Cloud Providers, or “Self-managed” Environments** The major cloud providers tout disaster recover across regions, ensuring resilience against natural disasters impacting an entire geographical region. But in reality, such disasters are exceedingly rare. Whatʼs far more common are technical issues impacting an entire region for a specific cloud provider. For instance, we’ve encountered DNS-level failures in GCP regions, rendering all services dependent on DNS, including SkySQL, inaccessible. One effective strategy to mitigate such risks is to replicate data to a data center owned by a different cloud provider within the same geographical area, minimizing network latencies. Disaster recovery across cloud providers is of course something an individual provider such as AWS or GCP simply donʼt support. Alternatively, customers can maintain their own “standby” database for emergencies—an environment entirely under their control, ensuring a near-real time copy of the data at all times. diff --git a/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md b/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md index eb2d3b49..f5f07a54 100644 --- a/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md +++ b/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md @@ -1,118 +1,105 @@ # Setting up AWS Private Link -!!! Note - AWS PrivateLink is used for private connections within the same AWS region. - - Prior to configuring AWS PrivateLink on a SkySQL service, you must have created a VPC with a private subnet that will be used to communicate with private IP addresses. - - For detailed information about AWS PrivateLink, see ["AWS PrivateLink" (Amazon documentation)](https://aws.amazon.com/privatelink/). +AWS PrivateLink is an AWS service that enables secure and private connectivity between Virtual Private Clouds (VPCs) and third-party services. By using PrivateLink with SkySQL services, traffic does not traverse the public internet, which enhances security and reduces exposure to potential threats. -The default endpoint mechanism is "`nlb`", which supports accessing the SkySQL service via the public internet. Use of AWS PrivateLink occurs when the endpoint mechanism is changed to "`privateconnect`". +For detailed information about AWS PrivateLink, see ["AWS PrivateLink" (Amazon documentation)](https://aws.amazon.com/privatelink/). -AWS PrivateLink can be enabled using the SkySQL Portal, SkySQL DBaaS API, or SkySQL Terraform Provider. -## **Note the following** +## **Considerations** -- When a SkySQL service has the `privateconnect` endpoint mechanism, all connections occur through private endpoints. SkySQL does offer users to setup a “secondary” public IP endpoint also (available in the Portal UI when you provision a new service). -- Client connections can originate from private IP addresses in the linked VPC. -- AWS PrivateLink supports DNS propagation which can be configured to resolve the Database FQDN to the private IP for an internal DNS request. The private DNS names feature must be enabled on the customer's VPC. - -- The SkySQL IP Allowlist is not used with the `privateconnect` endpoint mechanism. -- With AWS PrivateLink, connections are restricted to the list of allowed accounts that were specified when configuring the SkySQL endpoint. -- Further restrictions are controlled through the customer's AWS account. VPC security group policies apply. For detailed information, see ["Control traffic to resources using security groups"](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) (Amazon documentation). -- Some customers using AWS PrivateLink may choose to disable TLS, but MariaDB recommends keeping TLS enabled for extra security. -- Endpoint changes can be destructive, resulting in downtime. When you change the connection type from private to public, or public to private, the endpoint must be destroyed and recreated. Changing the SkySQL endpoint between `nlb` and `privateconnect` will result in a service interruption. +- AWS PrivateLink is used for private connections within the same AWS region. The SkySQL service and the connection VPC must be in the same region. +- When using SkySQL with AWS PrivateLink, all connections occur through private endpoints. If you need to connect to the service from outside your VPC, you will need to use a VPN or other mechanism to go through the connected VPC. Alternatively, SkySQL can be configured to provide a second, public endpoint for an additional fee. +- A list of AWS Account IDs that will be allowed to connect to the SkySQL service must be provided when enabling AWS PrivateLink. This list can be updated at any time. +- The SkySQL IP Allowlist is not used with AWS PrivateLink connections. Access to the SkySQL service will be controlled by Security Groups in the connecting VPC. For detailed information, see ["Control traffic to resources using security groups"](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) (Amazon documentation). - Connections to SkySQL services by features such as SkySQL backups, and monitoring do not depend on AWS PrivateLink. - Query Editor is not supported when AWS PrivateLink is enabled. + ### **Enable AWS PrivateLink on Service Launch** -To enable AWS PrivateLink when launching a new service via the SkySQL Portal all you need to do is select the 'Enable Private link' option in the 'Security' section. +
+Enable Privatelink via the SkySQL Portal +
-For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page. +To enable AWS PrivateLink when launching a new service via the SkySQL Portal select the 'Enable Private link' option in the 'Security' section. +After the service completes provisioning, you will see a new option to "Set up Private Link" in the service's context menu. Click this option to add one or more AWS account IDs to the allowlist. -### **Enable AWS PrivateLink on Existing SkySQL Service** +
-To enable AWS PrivateLink for an existing service via the SkySQL Portal: +
+Enable Privatelink via the SkySQL DBaaS API +
-1. Log in to the SkySQL Portal -2. Click the "MANAGE" button (at right) for the desired service. -3. In the context menu, choose the "Set up Private Link" menu item. -4. In the popup window, add one or more AWS account IDs. -5. Click the "OK" button to confirm this operation. +To enable AWS PrivateLink when launching a new service via the SkySQL DBaaS API, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes to service creation JSON payload. -For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page. +``` +{ + "name": "my-skysql-service", + ... + "endpoint_mechanism": "privateconnect", + "allowed_accounts": [ + "AWS-ACCOUNT-ID-1", + "AWS-ACCOUNT-ID-2" + ] +} +``` +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a JSON array of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service. -### **Disabling AWS PrivateLink** +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/post_provisioning_v1_services). +
-To disable AWS PrivateLink via the SkySQL Portal: +
+Enable Privatelink via the SkySQL Terraform Provider +
-1. Visit the [SkySQL Portal](https://mariadb.com/docs/skysql-dbaas/working/nr-portal/) -2. Find the service that you would like to modify. -3. Click "MANAGE" on the far right side of the service listing. -4. In the context menu, select "Manage PrivateLink". -5. In the popup window, click "I want to disconnect my Private Link". -6. In the popup window, select "Disconnect". -7. After the service restarts, PrivateLink is disabled. -8. Since the service's allowlist was cleared when AWS PrivateLink was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling PrivateLink. +To enable AWS PrivateLink when launching a new service via the SkySQL DBaaS API, set the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes on the `skysql_service` resource. -## AWS Endpoint Setup +```hcl +resource "skysql_service" "example" { + name = "my-skysql-service" + ... + endpoint_mechanism = "privateconnect" + endpoint_allowed_accounts = ["123456789012"] +} +``` -In addition to switching the SkySQL service to the `privateconnect` endpoint mechanism, an AWS Endpoint must be created using the customer's AWS account in order for the SkySQL service to become accessible. +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a list of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service. -1. Log in to the AWS console. -2. Confirm the correct region is selected. -3. Navigate to the "VPC" page, then the "Endpoint" section. -4. Click the "Create Endpoint" button. -5. Fill in the name to help you to identify it. (This is optional.) -6. Set the Service category to "Other endpoint services". -7. The value for the "Service name" field must be set to the value of the "Fully Qualified Domain Name" in the "Connect" window from SkySQL portal. -8. Click "Verify service". AWS should find the service and auto-populate the rest of the form. -9. In the VPC search field, find the VPC that you want to use for the interconnect between the clients and the SkySQL service. -10. In the Subnets section, it is suggested that you select all the Availability Zones in the list, entering the proper subnet ID for each one. If you are unsure, view the details of your running instances to see the Subnet ID that they have configured. -11. Select IPv4 for "IP address type". -12. For the "Security Groups" section, we suggest that you create a new security group that will regulate which instances can make a connection to the database. - - In a new browser tab, use the AWS console to visit the "Security Groups" settings (under Network & Security). - - Click on the "Create security group" button. - - Fill in the group's name and (optionally) its description. - - Under "Inbound rules" click the "Add rule" button. - - Set the value for the "Port range" to be the port number of the "Read-write port" in the "Connect" window of the SkySQL portal. - - Set the Source to either a list of private (internal) IPv4 addresses that you want to authorize (adding a "/32" suffix to each one), or set it to an existing security group name that can be used to authorize all instances that have that security group in their configuration. - - Press the "Create security group" button. -13. Back on the endpoint tab, click the refresh button on the "Security Groups" section and choose the newly created security group. -14. Optionally add a "Name" tag for easy identification. -15. Press the "Create endpoint" button. Endpoint creation may take several minutes. When complete, status will change from "Pending" to "Available". -16. In the details of the new endpoint will be a list of DNS names. Copy the first name in the list and use that as the hostname for your clients to use when they connect. +A complete example Terraform template that creates a new SkySQL service with AWS PrivateLink enabled can be found in the [terraform provider examples](https://github.com/skysqlinc/terraform-provider-skysql/tree/main/examples/privateconnect). -The newly created endpoint now authorizes the internal IPs or security groups that you specified in the Source values to access the SkySQL service's connection port. When testing a client connection, ensure that the client host is authorized by the security group's Source settings and that you're using the "`readwrite`" port plus the appropriate username and password (either the default values or the value for any user you have created). -## **Setting up AWS Private link using SkySQL REST API** +For more information on using the SkySQL Terraform Provider, see ["SkySQL Terraform Provider"](https://registry.terraform.io/providers/skysqlinc/skysql/latest/docs). -### **Enable AWS PrivateLink on Service Launch** +
-To enable AWS PrivateLink when launching a new service via the SkySQL DBaaS API: - -1. Initiate service launch using the procedure at "[DBaaS API Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20REST%20API.md). -2. When you are creating the request, add the `"endpoint_mechanism"` and `"endpoint_allowed_accounts"` attributes to the JSON payload: - 1. - - ```json - "endpoint_mechanism": "privateconnect", - "endpoint_allowed_accounts": [ - "AWS-ACCOUNT-ID-1", - "AWS-ACCOUNT-ID-2" - ] - ``` - - - Set `"endpoint_mechanism"` to `"privateconnect"` - - Set `"endpoint_allowed_accounts"` to a JSON array of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service +**For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page.** -For the next step, go through the AWS Endpoint setup section. -### **Enable AWS PrivateLink on Existing SkySQL Service** +### **Enable AWS PrivateLink on an Existing SkySQL Service** -To enable AWS PrivateLink on an existing service via the SkySQL DBaaS API, create a JSON payload such as: +> [!CAUTION] +> Enabling PrivateLink on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the public endpoint is replaced with the new PrivateLink endpoint. + +
+Enable AWS PrivateLink on an existing service via the SkySQL Portal: +
+ +1. Log in to the SkySQL Portal +2. Click the "MANAGE" button (on the right) for the desired service. +3. In the context menu, choose the "Set up AWS PrivateLink" menu item. +4. In the popup window, add one or more AWS account IDs. +5. Click the "OK" button to confirm this operation. + +
+ +
+Enable AWS PrivateLink on an existing service via the SkySQL DBaaS API: +
+ +To enable AWS PrivateLink on an existing service, you will need to update the service endpoints with a payload similar to the following: ```json [ @@ -126,201 +113,117 @@ To enable AWS PrivateLink on an existing service via the SkySQL DBaaS API, creat ] ``` -- Set `"mechanism"` to `"privateconnect"` -- Set `"allowed_accounts"` to a JSON array of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service. +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +
-```json -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` +**For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page.** -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": -```json -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` +## AWS Endpoint Setup -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): +To connect to a SkySQL service using AWS PrivateLink, you must create an endpoint in your VPC that connects to the SkySQL service. The endpoint will be used by clients in your VPC to connect to the SkySQL service. + +### Pre-requisites +- You must have a VPC in the same region as the SkySQL service. +- You must create a security group that will be used to control access to the SkySQL service endpoint. + - This security group should contain rules to allow traffic from your client instances to the port of the SkySQL service (usually 3306). + - You must create a rule in your security group for each IP range or other security group that will be allowed to connect to the SkySQL service. + - The security group must be associated with the VPC that you will use to connect to the SkySQL service. +- You will need to lookup the Endpoint Service ID that SkySQL provisioned for you when you created your SkySQL Service. + - This ID can be found in the "Connect" window of the SkySQL portal. + - If using the SkySQL DBaaS API, the ID can be found in the response of the service details API call. + ``` + curl https://api.skysql.com/provisioning/v1/services/{SERVICE_ID} | jq ".endpoints[0].endpoint_service" + ``` -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the PrivateLink endpoint setup is complete. - - A status of "`pending_upgrade`" is reflected while PrivateLink endpoint setup is in-progress. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. +### VPC Endpoint Creation Steps -The output will look something like this, though your values will vary: +1. Log in to the AWS console. +2. Confirm the correct region is selected. +3. Navigate to the "VPC" page, then the "Endpoints" section. +4. Click the "Create Endpoint" button. +5. In the "Name tag" field, enter a name for the new endpoint. This name can be anything you like. +6. Set the Service category to "Other endpoint services". +7. The value for the "Service name" field must be set to the value of the Endpoint Service ID provided to you by SkySQL. See [Pre-requisites](#pre-requisites) for more information on how to find this ID. +8. Click "Verify service". AWS should find the service and auto-populate the rest of the form. +9. In the VPC search field, find the VPC that you want to use for the interconnect between the clients and the SkySQL service. +10. In the Subnets section, it is suggested that you select all the Availability Zones in the list, entering the proper subnet ID for each one. If you are unsure, view the details of your running instances to see the Subnet ID that they have configured. +11. Select IPv4 for "IP address type". +12. For the "Security Groups" section, assign the security groups that will allow your client instance to connect to your VPC endpoint. See [Pre-requisites](#pre-requisites) for more information on setting up security groups. +13. Press the "Create endpoint" button. Endpoint creation may take several minutes. When complete, status will change from "Pending" to "Available". -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"com.amazonaws.vpce.AWS_REGION.vpce-svc-AWS_SERVICE_ID" -``` +After creation, the Endpoint will be in `Pending` status while AWS provisions the new endpoint. Once the endpoint is `Available`, you can connect to your SkySQL service using the new endpoint. +The newly created endpoint now authorizes the internal IPs or security groups that you specified in the Source values to access the SkySQL service's connection port. When testing a client connection, ensure that the client host is authorized by the security group's Source settings and that you're using the "`readwrite`" port plus the appropriate username and password (either the default values or the value for any user you have created). -If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. -For the next step, go through the AWS Endpoint Setup section. +### Connecting to your SkySQL Service -### **Disable AWS PrivateLink** +After creating your VPC endpoint, AWS will create a number of DNS records that will resolve to the private IP addresses of your Privatelink Endpoint. +- The first DNS name in the list can be used from any availability zone in your VPC and will resolve to the private IP address of the endpoint in the same availability zone. +- The following DNS names provided are availability zone specific and rely on the user to match the correct DNS name to the availability zone of the client instance. +- If connecting via these DNS names, we recommend using the first DNS name in the list to ensure that the connection is routed to the correct availability zone. -To disable AWS PrivateLink via the SkySQL DBaaS API, create a JSON payload such as: -```json -[ - { - "mechanism": "nlb" - } -] -``` +> [!NOTE] +> The DNS names provided by AWS will always be in the domain `amazonaws.com`. If connecting to your SkySQL service using SSL/TLS, the database certificate will not match the VPC endpoint name. Due to this, we recommend [Enabling Private DNS for AWS PrivateLink](#enabling-private-dns-for-aws-privatelink). -Set `"mechanism"` to `"nlb"` -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +### Enabling Private DNS for AWS PrivateLink -```json -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` +In order to connect to your SkySQL service using the skysql.com service name provided in the SkySQL portal, you must enable Private DNS for the VPC endpoint. This will allow the service name to resolve to the private IP address of the SkySQL service. -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": +The following requirements must be met to enable Private DNS for the VPC endpoint: +- Private DNS must be enabled for the VPC. +- Your client instances must use the default AWS DNS server provided by the VPC (this is usually turned on by default). -```json -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` +To enable Private DNS for the VPC endpoint: +1. Log in to the AWS console. +2. Confirm the correct region is selected. +3. Navigate to the "VPC" page, then the "Endpoints" section. +4. Select the VPC endpoint that you created for your SkySQL service. +5. Click the "Actions" button, then select "Modify Private DNS Name". +6. In the popup window, select the checkbox to "Enable Private DNS Name". +7. Click the "Save changes" button. -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): +After a short period of time, the service name provided in the SkySQL portal should resolve to the private IP address of the SkySQL service via PrivateLink. You can test this by connecting to the service using the service name provided in the portal. -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the endpoint modification is complete. - - A status of "`pending_upgrade`" is reflected while PrivateLink is being disabled. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. -The output will look something like this, though your values will vary: +### **Disabling AWS PrivateLink** -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"com.amazonaws.vpce.AWS_REGION.vpce-svc-AWS_SERVICE_ID" -``` +> [!CAUTION] +> Disabling PrivateLink on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the private endpoint is replaced with the new public endpoint. -If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. +
+Disable AWS PrivateLink via the SkySQL Portal +
-Since the service's allowlist was cleared when AWS PrivateLink was previously enabled, you will need to update the allowlist to allow clients to connect after disabling PrivateLink. +1. Visit the [SkySQL Portal](https://app.skysql.com/) +2. Find the service that you would like to modify. +3. Click "MANAGE" on the far right side of the service listing. +4. In the context menu, select "Manage PrivateLink". +5. In the popup window, click "I want to disconnect my Private Link". +6. In the popup window, select "Disconnect". +7. Since the service's allowlist was cleared when AWS PrivateLink was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling PrivateLink. -To add an address to the allowlist via the SkySQL DBaaS API, create a JSON payload such as: +
-```json -**{** **"ip_address":** "192.0.2.10/32"**}** -``` +
+Disable AWS PrivateLink via the SkySQL DBaaS API +
-And then use the API to update the `allowlist` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +To disable AWS PrivateLink on an existing service, you will need to update the service endpoints with a payload similar to the following: -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X POST -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/security/allowlist \ - | jq . +```json +[ + { + "mechanism": "nlb" + } +] ``` -## Setup AWS Private link using Terraform Provider - -For general instructions on using the SkySQL Terraform Provider, see "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) - -For an example Terraform configuration that enables AWS PrivateLink, see Resources section [here](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md). - - -### **Enable AWS PrivateLink on Service Launch** +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). -To enable AWS PrivateLink when launching a new service via the SkySQL Terraform provider: - -1. Initiate service launch using the procedure at "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) . -2. When you are configuring the `skysql_service` resource, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes. - - For example, the attributes can be placed after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["AWS-ACCOUNT-ID-1","AWS-ACCOUNT-ID-2"] - ``` - - - Set `endpoint_mechanism` to `"privateconnect"` - - Set `endpoint_allowed_accounts` to a comma-separated list of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service -3. Continue the rest of the procedure. - -For the next step, see the AWS Endpoint Setup section on this page. - -### **Enable AWS PrivateLink on Existing SkySQL Service** - -To enable AWS PrivateLink for an existing service via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by adding the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes. - - Follow the same instructions as above. - -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): - - ```bash - $ terraform apply -var-file="skysql-nr-quickstart.tfvars" - ``` - - -For the next step, see the AWS Endpoint Setup section on this page. - -### **Disable AWS PrivateLink** - -To disable AWS PrivateLink via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by doing the following: - - Changing the `endpoint_mechanism` attribute to `"nlb"` - - Removing the `endpoint_allowed_accounts` attribute - - Adding the `allow_list` attribute - - For example, if you started with the following attributes after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["AWS-ACCOUNT-ID-1","AWS-ACCOUNT-ID-2"] - ``` - - You could disable AWS PrivateLink by changing the attributes to the following: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "nlb" - allow_list = [ - { - "ip" : var.ip_address, - "comment" : var.ip_address_comment - } - ] - ``` - -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): +
diff --git a/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md b/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md index 3a49025e..a7fbdba3 100644 --- a/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md +++ b/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md @@ -1,70 +1,142 @@ # Setting up GCP Private Service Connect -!!! Note: - GCP Private Service Connect (PSC) is used for private connections within the same GCP region. - - Prior to configuring PSC on a SkySQL service, you must have created a VPC with a private subnet that will be used to communicate with private IP addresses. - - For detailed information about GCP PSC, see ["Private Service Connect" (Google documentation)](https://cloud.google.com/vpc/docs/private-service-connect). +Google Private Service Connect (PSC) is a Google Cloud service that enables secure and private connectivity between Virtual Private Clouds (VPCs) and third-party services. By using PSC with SkySQL services, traffic does not traverse the public internet, which enhances security and reduces exposure to potential threats. -The default endpoint mechanism is "`nlb`", which supports accessing the SkySQL service via the public internet. Use of GCP PSC occurs when the endpoint mechanism is changed to "`privateconnect`". +For detailed information about Google PSC, see ["Private Service Connect" (Google documentation)](https://cloud.google.com/vpc/docs/private-service-connect). -GCP PSC can be enabled using the SkySQL Portal, SkySQL DBaaS API, or SkySQL Terraform Provider. -## **Note the following** +## **Considerations** -- When a SkySQL service has the `privateconnect` endpoint mechanism, all connections occur through private endpoints. SkySQL does offer users to setup a “secondary” public IP endpoint also (available in the Portal UI when you provision a new service). -- With Private Service Connect, projects are allowlisted and auto-accept of connections from any account is disabled. -- Access restrictions are controlled through the customer's GCP account. VPC firewall rules apply. For detailed information, see ["Cloud Firewall overview" (Google documentation)](https://cloud.google.com/vpc/docs/firewalls). -- Some customers using Private Service Connect may choose to disable TLS, but MariaDB recommends keeping TLS enabled for extra security. -- Database connection limits are independent from PSC connection limits. The limit for PSC connections is 10. -- Endpoint changes can be destructive, resulting in downtime. When you change the connection type from private to public, or public to private, the endpoint must be destroyed and recreated. Changing the SkySQL endpoint between `nlb` and `privateconnect` will result in a service interruption. -- Connections to SkySQL services by features such as SkySQL backups, and monitoring do not depend on Private Service Connect. -- Query Editor is not supported when Private Service Connect is enabled. +- PSC is used for private connections within the same Google Cloud region. The SkySQL service and the connection VPC must be in the same region. +- When using SkySQL with PSC, all connections occur through private endpoints. If you need to connect to the service from outside your VPC, you will need to use a VPN or other mechanism to go through the connected VPC. Alternatively, SkySQL can be configured to provide a second, public endpoint for an additional fee. +- A list of Google Cloud project IDs that will be allowed to connect to the SkySQL service must be provided when enabling PSC. This list can be updated at any time. +- The SkySQL IP Allowlist is not used with PSC connections. Access to the SkySQL service can be controlled by setting up firewall rules inside the connecting VPC. +- Connections to SkySQL services by features such as SkySQL backups, and monitoring do not depend on PSC. +- Query Editor is not supported when PSC is enabled. +- PSC has connection limits which refer to the number of endpoints that can be created to a single PSC service within Google Cloud. Database connection limits are independent from PSC connection limits. The limit for PSC connections is 10. -### **Enable GCP Private Service Connect on Service Launch** -To enable GCP Private Service Connect when launching a new service via the SkySQL Portal: +### **Enable Private Service Connect on Service Launch** -- When you get to the final "Security" section, select "Enable Private Service Connect". +
+Enable Google PSC via the SkySQL Portal +
-For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. +To enable PSC when launching a new service via the SkySQL Portal select the 'Google Private Service Connect' option in the 'Security' section. +After the service completes provisioning, you will see a new option to "Manage Google Private Service Connect" in the service's context menu. Click this option to add one or more Google project IDs to the allowlist. +
-### **Enable GCP Private Service Connect on Existing SkySQL Service** +
+Enable Google PSC via the SkySQL DBaaS API +
-To enable GCP Private Service Connect for an existing service via the SkySQL Portal: +To enable Google PSC when launching a new service via the SkySQL DBaaS API, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes to service creation JSON payload. -1. Log in to the SkySQL Portal. -2. Click the "MANAGE" button (at right) for the desired service. -3. In the context menu, choose the "Set up Private Service Connect" menu item. +``` +{ + "name": "my-skysql-service", + ... + "endpoint_mechanism": "privateconnect", + "allowed_accounts": [ + "GCP-PROJECT-ID-1", + "GCP-PROJECT-ID-2" + ] +} +``` +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a JSON array of one or more client project IDs in Google Cloud that will be allowed to establish a private connection to the SkySQL service. + +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/post_provisioning_v1_services). +
+ +
+Enable Google PSC via the SkySQL Terraform Provider +
+ +To enable Google PSC when launching a new service via the SkySQL DBaaS API, set the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes on the `skysql_service` resource. + +```hcl +resource "skysql_service" "example" { + name = "my-skysql-service" + ... + endpoint_mechanism = "privateconnect" + endpoint_allowed_accounts = ["GCP-PROJECT-ID-1", "GCP-PROJECT-ID-2"] +} +``` + +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a list of one or more customer project IDs in Google Cloud that will be allowed to establish a private connection to the SkySQL service. + +A complete example Terraform template that creates a new SkySQL service with Google PSC enabled can be found in the [terraform provider examples](https://github.com/skysqlinc/terraform-provider-skysql/tree/main/examples/private-service-connect). + + +For more information on using the SkySQL Terraform Provider, see ["SkySQL Terraform Provider"](https://registry.terraform.io/providers/skysqlinc/skysql/latest/docs). + +
+ +**For the next step, see the [PSC Endpoint Setup](#private-service-connect-endpoint-setup) section on this page.** + + +### **Enable Google PSC on an Existing SkySQL Service** + +> [!CAUTION] +> Enabling PSC on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the public endpoint is replaced with the new PSC endpoint. + +
+Enable Google PSC on an existing service via the SkySQL Portal: +
+ +1. Log in to the SkySQL Portal +2. Click the "MANAGE" button (on the right) for the desired service. +3. In the context menu, choose the "Set up Google Private Service Connect" menu item. 4. In the popup window, add one or more GCP project IDs. 5. Click the "OK" button to confirm this operation. -For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. +
-### **Disable GCP Private Service Connect** +
+Enable Google PSC on an existing service via the SkySQL DBaaS API: +
-To disable GCP Private Service Connect via the SkySQL Portal: +To enable Google PSC on an existing service, you will need to update the service endpoints with a payload similar to the following: -!!! Note: - Adding a new PSC will fail unless existing PSC to deleted projects are deleted. +```json +[ + { + "mechanism": "privateconnect", + "allowed_accounts": [ + "GOOGLE-PROJECT-ID-1", + "GOOGLE-PROJECT-ID-2" + ] + } +] +``` + +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). + +
+ +**For the next step, see the [PSC Endpoint Setup](#private-service-connect-endpoint-setup) section on this page.** -1. Visit the SkySQL Portal. -2. Find the service that you would like to modify. -3. Click "MANAGE" on the far right side of the service listing. -4. In the context menu, select "Manage Private Service Connect". -5. In the popup window, click "I want to disconnect my Private Service Connect". -6. In the popup window, select "Disconnect". -7. After the service restarts, Private Service Connect is disabled. -8. Since the service's allowlist was cleared when GCP Private Service Connect was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling Private Service Connect. -## GCP Endpoint Setup +## Private Service Connect Endpoint Setup -Endpoint setup is performed using the customer's GCP account. +To connect to a SkySQL service using Google PSC, you must create an endpoint in your VPC that connects to the SkySQL service. The endpoint will be used by clients in your VPC to connect to the SkySQL service. + + +### Pre-requisites +- You must have a VPC in the same region as the SkySQL service. +- You will need to lookup the Endpoint Service ID that SkySQL provisioned for you when you created your SkySQL Service. + - This ID can be found in the "Connect" window of the SkySQL portal. + - If using the SkySQL DBaaS API, the ID can be found in the response of the service details API call. + ``` + curl https://api.skysql.com/provisioning/v1/services/{SERVICE_ID} | jq ".endpoints[0].endpoint_service" + ``` -### **Create a Subnet (optional)** +### Create a Subnet (optional) We recommend use of a subnet dedicated to Private Service Connect endpoints in the same VPC where the application is running. @@ -79,7 +151,8 @@ We recommend use of a subnet dedicated to Private Service Connect endpoints in t - Optionally configure Flow logs - Click "ADD". -### **Create a Static Internal IP Address** + +### Create a Static Internal IP Address 1. In the GCP console, navigate VPC network → VPC networks → → STATIC INTERNAL IP ADDRESSES → RESERVE STATIC ADDRESS. - Replace with the name of the VPC where the application is running. @@ -90,108 +163,59 @@ We recommend use of a subnet dedicated to Private Service Connect endpoints in t - Purpose: Non-shared - Click "RESERVE". -### ****Connect to the Published Service**** + +### VPC Endpoint Creation Steps + 1. In the GCP console, navigate Network services → Private Service Connect → CONNECTED ENDPOINTS → CONNECT ENDPOINT. 2. Configure the endpoint connection: - Target: Published service - - Target service: the value of the "Fully Qualified Domain Name" in the "Connect" window from SkySQL portal. + - Target service: the value of the Endpoint Service ID. See [Pre-requisites](#pre-requisites) for more information on how to find this ID. - Endpoint name: set to the Database ID from SkySQL (dbxxxxxxxx) - Network: select the VPC network where the application is running - Subnetwork: select the subnet where the static internal IP address is reserved - IP address: select the reserved internal IP address from the prior step - Click "ADD ENDPOINT". -## Setting up GCP PSC using SkySQL REST API - -### **Enable GCP PSC on Service Launch** - -To enable GCP Private Service Connect when launching a new service via the SkySQL DBaaS API: - -1. Initiate service launch using the procedure at "[DBaaS API Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20REST%20API.md)". -2. When you are creating the request, add the `"endpoint_mechanism"` and `"endpoint_allowed_accounts"` attributes to the JSON payload: - - ```json - "endpoint_mechanism": "privateconnect", - "endpoint_allowed_accounts": [ - "GCP-PROJECT-ID-1", - "GCP-PROJECT-ID-2" - ] - ``` - - - Set `"endpoint_mechanism"` to `"privateconnect"` - - Set `"endpoint_allowed_accounts"` to a JSON array of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service - -For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. - -### **Enable GCP Private Service Connect on Existing SkySQL Service** - -To enable GCP Private Service Connect on an existing service via the SkySQL DBaaS API, create a JSON payload such as: - -```json -[ - { - "mechanism": "privateconnect", - "allowed_accounts": [ - "GCP-PROJECT-ID-1", - "GCP-PROJECT-ID-2" - ] - } -] -``` +After creation, the Endpoint should have a status of `Accepted`. If this status is not present, please ensure your Google Project ID is added to the list of allowed accounts in the SkySQL portal for this service. -- Set `"mechanism"` to `"privateconnect"` -- Set `"allowed_accounts"` to a JSON array of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +### Connecting to your SkySQL Service -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` +After creating your PSC endpoint, your service should be available within your VPC at the Private IP Address you assigned to the endpoint. +- DNS propagation from SkySQL to the Private IP address is not supported when using PSC. +- The hostname when connecting to your SkySQL service should always be the Private IP address of the PSC endpoint. -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": -```bash -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` +> [!NOTE] +> When using PSC with SSL/TLS, there will be a hostname mismatch since the hostname provisioned by SkySQL will not match your internal IP Address. This can be ignored as the connection is still secure. -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the Private Service Connect endpoint setup is complete. - - A status of "`pending_upgrade`" is reflected while Private Service Connect endpoint setup is in-progress. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. +### **Disabling Google PSC** -The output will look something like this, though your values will vary: +> [!CAUTION] +> Disabling PSC on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the private endpoint is replaced with the new public endpoint. -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"GCP_ENDPOINT_HOSTNAME" -``` +
+Disable Google PSC via the SkySQL Portal +
-If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. +1. Visit the [SkySQL Portal](https://app.skysql.com/) +2. Find the service that you would like to modify. +3. Click "MANAGE" on the far right side of the service listing. +4. In the context menu, select "Manage your Private Service Connect". +5. In the popup window, click "I want to disconnect my Private Service Connect". +6. In the popup window, select "Disconnect". +7. Since the service's allowlist was cleared when Goolge PSC was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling PSC. -For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. +
-### **Disable GCP PSC** +
+Disable Google PSC via the SkySQL DBaaS API +
-To disable GCP Private Service Connect via the SkySQL DBaaS API, create a JSON payload such as: +To disable Google PSC on an existing service, you will need to update the service endpoints with a payload similar to the following: ```json [ @@ -201,157 +225,7 @@ To disable GCP Private Service Connect via the SkySQL DBaaS API, create a JSON p ] ``` -- Set `"mechanism"` to `"nlb"` - -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: - -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` - -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": - -```bash -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` - -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): - -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the endpoint modification is complete. - - A status of "`pending_upgrade`" is reflected while Private Service Connect is being disabled. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. - -The output will look something like this, though your values will vary: - -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"GCP_ENDPOINT_HOSTNAME" -``` - -If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. - -Since the service's allowlist was cleared when GCP Private Service Connect was previously enabled, you will need to update the allowlist to allow clients to connect after disabling Private Service Connect. - -To add an address to the allowlist via the SkySQL DBaaS API, create a JSON payload such as: - -```json -**{** **"ip_address":** "192.0.2.10/32"**}** -``` - -And then use the API to update the `allowlist` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: - -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X POST -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/security/allowlist \ - | jq . -``` - -## **Terraform Provider** - -GCP Private Service Connect can be enabled with Terraform using the SkySQL Terraform provider. - -For general instructions on using the SkySQL Terraform Provider, see "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) . - -For an example Terraform configuration that enables GCP Private Service Connect, see Resources section [here](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) . - -### **Enable GCP Private Service Connect on Service Launch** - -To enable GCP Private Service Connect when launching a new service via the SkySQL Terraform provider: - -1. Initiate service launch using the procedure at "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md). -2. When you are configuring the `skysql_service` resource, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes: - - For example, the attributes can be placed after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["GCP-PROJECT-ID-1","GCP-PROJECT-ID-2"] - ``` - - - Set `endpoint_mechanism` to `"privateconnect"` - - Set `endpoint_allowed_accounts` to a comma-separated list of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service -3. Continue the rest of the procedure. - -For the next step, see the [Connectivity](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#Connectivity), [Controls](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#Controls), & [GCP Endpoint Setup](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#GCP_Endpoint_Setup) sections on this page. - -### **Enable GCP Private Service Connect on Existing SkySQL Service** - -To enable GCP Private Service Connect for an existing service via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by adding the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes. - - For example, the attributes can be placed after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["GCP-PROJECT-ID-1","GCP-PROJECT-ID-2"] - ``` - - - Set `endpoint_mechanism` to `"privateconnect"` - - Set `endpoint_allowed_accounts` to a comma-separated list of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): - - ```bash - $ terraform apply -var-file="skysql-nr-quickstart.tfvars" - ``` - - -For the next step, see the [GCP Endpoint Setup](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#GCP_Endpoint_Setup) sections on this page. - -### **Disable GCP Private Service Connect** - -To disable GCP Private Service Connect via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by doing the following: - - Changing the `endpoint_mechanism` attribute to `"nlb"` - - Removing the `endpoint_allowed_accounts` attribute - - Adding the `allow_list` attribute - - For example, if you started with the following attributes after `ssl_enabled`: - - ```json - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["GCP-PROJECT-ID-1","GCP-PROJECT-ID-2"] - ``` - - You could disable GCP Private Service Connect by changing the attributes to the following: - - ```json - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "nlb" - allow_list = [ - { - "ip" : var.ip_address, - "comment" : var.ip_address_comment - } - ] - ``` - -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): - - ```json - $ terraform apply -var-file="skysql-nr-quickstart.tfvars" - ``` - +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). ---- +
diff --git a/docs/index.md b/docs/index.md index 73dc23a8..e3952a82 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,11 +1,7 @@ # Introduction to SkySQL -!!! Note - **SkySQL documentation is work-in-progress. Many of the links from the pages below still reference the older version of SkySQL documentation on the [Mariadb](http://Mariadb.com) site. Please bear with us during this transition.** - - -MariaDB is one of the most popular, mature open source relational databases in the world. +MariaDB is one of the most popular, mature open source relational databases in the world. MariaDB is designed to be highly compatible with MySQL, as it was originally forked from MySQL. SkySQL is a multi-cloud, fully managed Database as a Service. It is specifically designed to manage MariaDB across data centers, regions and even across cloud providers. SkySQL was originally developed at [MariaDB](http://mariadb.com) with the goal to be the most comprehensive cloud platform for MariaDB. Its practical feature set was based on many years of insights gleamed from hundreds of customers running mission critical workloads. The core team that built SkySQL is now part of a new independent company called SkySQL (Established in late 2023).