Docc (#506)

* docc scripts

* SotoSignerV4 main page

* AWSSigner docs

* gen-docs uses docc

* Setup docc environment in GH action

* html template folder

* Update gen-docs.yml

* Sort SotoCore symbols

* Minor comment changes

* Credential Provider factory cleanup

* AWSClient functions

* AWSService

* Update AWSRequest.swift

* Articles

* Doc changes

* GH action

* Add links between SotoCore and SotoSignerV4

* Use apple/swift-docc@main

To get navigation bar

* Update SotoCore.docc/SotoCore/AWSClient.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoCore/AWSClient.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoCore/AWSClient.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update Sources/SotoCore/AWSClient.swift

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoSignerV4/SotoSignerV4.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoCore/SotoCore.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoCore/Articles/CredentialProviders.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoCore/Articles/ServiceObjects.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

* Update SotoCore.docc/SotoCore/Articles/ServiceObjects.md

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>

.github/workflows/gen-docs.yml

@@ -12,14 +12,19 @@ jobs:
 
     steps:
     - name: Checkout
-      uses: actions/checkout@v1
+      uses: actions/checkout@v3
+      with:
+        fetch-depth: 0
     - name: Install Dependencies
       run: |
-        brew install sourcekitten
-        gem install jazzy
+        brew install mint
+        mint install apple/swift-docc@main
+        echo "$HOME/.mint/bin" >> $GITHUB_PATH
     - name: Build
+      env:
+        DOCC: docc
       run: |
-        ./scripts/build-docs.sh
+        ./scripts/build-docc.sh
     - name: Commit
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -29,9 +34,9 @@ jobs:
         git config user.name "${GITHUB_ACTOR}"
         git config user.email "${GITHUB_ACTOR}@users.noreply.github.com"
         git checkout ${REMOTE_BRANCH}
-        rm -rf 5.x.x/
-        mv docs/ 5.x.x/
-        git add --all 5.x.x
+        rm -rf 6.x.x/
+        mv docs/soto-core/6.x.x/ 6.x.x/
+        git add --all 6.x.x
         git status       
         
         git commit -m "Documentation for https://github.com/${GITHUB_REPOSITORY}/tree/${GITHUB_SHA}" -m "Generated by gen-docs.yml"

SotoCore.docc/SotoCore/AWSClient.md

@@ -0,0 +1,151 @@
+# ``SotoCore/AWSClient``
+
+@Metadata {
+    @DocumentationExtension(mergeBehavior: override)
+}
+
+Client managing communication with AWS services
+
+## Overview
+
+The `AWSClient` is the core of Soto. This is the object that manages your communication with AWS. It manages credential acquisition, takes your request, encodes it, signs it, sends it to AWS and then decodes the response for you. In most situations your application should only require one `AWSClient`. Create this at startup and use it throughout.
+
+When creating an `AWSClient` you need to provide how you are going to acquire AWS credentials, what your policy is on retrying failed requests, a list of middleware you would apply to requests to AWS and responses from AWS, client options, where you get your `HTTPClient` and a `Logger` to log any output not directly linked to a request. There are defaults for most of these parameters. The only one required is the `httpClientProvider`.
+
+```swift
+let awsClient = AWSClient(
+    credentialProvider: .default,
+    retryPolicy: .default,
+    middlewares: [],
+    options: .init(),
+    httpClientProvider: .createNew,
+    logger: AWSClient.loggingDisabled
+)
+```
+
+### Credential Provider
+
+The `credentialProvider` defines how the client acquires its AWS credentials. You provide `AWSClient.init` with a factory function that will create the credential provider once the `AWSClient` has been initialised. This allows for the credential provider to use the `EventLoopGroup`, `Logger` and `HTTPClient` that the `AWSClient` uses. You can find factory functions for all the standard credential providers in ``CredentialProviderFactory``.
+
+```swift
+let awsClient = AWSClient(
+    credentialProvider: .environment, 
+    ...
+)
+```
+
+If no credential provider is provided the default is to try environment variables, ECS container credentials, EC2 instance metadata and then the `~/.aws/credentials` file in this order. The first method that is successful will be used.
+
+An alternative is to provide credentials in code. You can do this as follows
+
+```swift
+let client = AWSClient(
+    credentialProvider: .static(
+        accessKeyId: "MY_AWS_ACCESS_KEY_ID",
+        secretAccessKey: "MY_AWS_SECRET_ACCESS_KEY"
+    ),
+    ...
+)
+```
+The article <doc:CredentialProviders> gives more information on credential providers.
+
+### Retry policy
+
+The `retryPolicy` defines how the client reacts to a failed request. There are three retry policies supplied. `.noRetry` doesn't retry the request if it fails. The other two will retry if the response is a 5xx (server error) or a connection error. They differ in how long they wait before performing the retry. `.exponential` doubles the wait time after each retry and `.jitter` is the same as exponential except it adds a random element to the wait time. `.jitter` is the recommended method from AWS so it is the default.
+
+### Middleware
+
+Middleware allows you to insert your own code just as a request has been constructed or a response has been received. You can use this to edit the request/response or just to view it. SotoCore supplies one middleware — `AWSLoggingMiddleware` — which outputs your request to the console once constructed and the response is received from AWS.
+
+### HTTP Client provider
+
+The `HTTPClientProvider` defines where you get your HTTP client from. You have three options:
+
+- Pass `.createNew` which indicates the `AWSClient` should create its own HTTP client. This creates an instance of `HTTPClient` using [`AsyncHTTPClient`](https://github.com/swift-server/async-http.client).
+- Supply your own `EventLoopGroup` with `.createNewWithEventLoopGroup(EventLoopGroup`). This creates a new `HTTPClient` but has it use the supplied `EventLoopGroup`.
+- Supply your own HTTP client with `.shared(AWSHTTPClient)` as long as it conforms to the protocol `AWSHTTPClient`. `AsyncHTTPClient.HTTPClient` already conforms to this protocol.
+
+There are a number of reasons you might want to provide your own client, such as:
+
+- You have one HTTP client you want to use across all your systems.
+- You want to change the configuration for the HTTP client used, perhaps you are running behind a proxy or want to enable response decompression.
+
+## AWSClient Shutdown
+
+The AWSClient requires you shut it down manually before it is deinitialized. The manual shutdown is required to ensure any internal processes are finished before the `AWSClient` is freed and Soto's event loops and client are shutdown properly. You can either do this asynchronously with `AWSClient.shutdown()` or do this synchronously with `AWSClient.syncShutdown()`.
+
+## Topics
+
+### Initializers
+
+- ``init(credentialProvider:retryPolicy:middlewares:options:httpClientProvider:logger:)``
+- ``HTTPClientProvider``
+- ``Options``
+- ``loggingDisabled``
+
+### Instance Properties
+
+- ``credentialProvider``
+- ``middlewares``
+- ``retryPolicy``
+- ``httpClient``
+- ``eventLoopGroup``
+
+### Shutdown
+
+- ``shutdown()``
+- ``shutdown(queue:_:)``
+- ``syncShutdown()``
+
+### Credentials
+
+- ``getCredential(on:logger:)-96cyr``
+- ``getCredential(on:logger:)-5dlty``
+- ``signHeaders(url:httpMethod:headers:body:serviceConfig:logger:)-2uyw9``
+- ``signHeaders(url:httpMethod:headers:body:serviceConfig:logger:)-8h5yq``
+- ``signURL(url:httpMethod:headers:expires:serviceConfig:logger:)-8d5k0``
+- ``signURL(url:httpMethod:headers:expires:serviceConfig:logger:)-49aa3``
+
+### Errors
+
+- ``ClientError``
+
+### Request Execution
+
+- ``execute(operation:path:httpMethod:serviceConfig:logger:on:)-6jc01``
+- ``execute(operation:path:httpMethod:serviceConfig:logger:on:)-3mu6q``
+- ``execute(operation:path:httpMethod:serviceConfig:logger:on:)-6hhlh``
+- ``execute(operation:path:httpMethod:serviceConfig:logger:on:)-6klm4``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:logger:on:)-4iuwj``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:logger:on:)-1upt6``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:logger:on:)-3ttl7``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:logger:on:)-3dlpq``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:logger:on:stream:)-3c73e``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:logger:on:stream:)-3yiw5``
+- ``execute(operation:path:httpMethod:serviceConfig:endpointDiscovery:logger:on:)-3ek2x``
+- ``execute(operation:path:httpMethod:serviceConfig:endpointDiscovery:logger:on:)-4uver``
+- ``execute(operation:path:httpMethod:serviceConfig:endpointDiscovery:logger:on:)-9pukf``
+- ``execute(operation:path:httpMethod:serviceConfig:endpointDiscovery:logger:on:)-7gnt3``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:endpointDiscovery:logger:on:)-3hzuw``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:endpointDiscovery:logger:on:)-2l7fr``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:endpointDiscovery:logger:on:)-1vx0e``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:endpointDiscovery:logger:on:)-3mdnx``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:endpointDiscovery:logger:on:stream:)-30x0f``
+- ``execute(operation:path:httpMethod:serviceConfig:input:hostPrefix:endpointDiscovery:logger:on:stream:)-1m1h2``
+
+### Pagination
+
+- ``PaginatorSequence``
+- ``paginate(input:command:inputKey:outputKey:logger:on:onPage:)``
+- ``paginate(input:initialValue:command:inputKey:outputKey:logger:on:onPage:)``
+- ``paginate(input:command:tokenKey:logger:on:onPage:)``
+- ``paginate(input:initialValue:command:tokenKey:logger:on:onPage:)``
+- ``paginate(input:command:tokenKey:moreResultsKey:logger:on:onPage:)``
+- ``paginate(input:initialValue:command:tokenKey:moreResultsKey:logger:on:onPage:)``
+
+### Waiters
+
+- ``waitUntil(_:waiter:maxWaitTime:logger:on:)-3ccn1``
+- ``waitUntil(_:waiter:maxWaitTime:logger:on:)-385eg``
+- ``Waiter``
+- ``WaiterState``

SotoCore.docc/SotoCore/Articles/CredentialProviders.md

@@ -0,0 +1,102 @@
+# Credential Providers
+
+Providing Credentials for signing AWS requests
+
+Before using Soto, you will need AWS credentials to sign all your requests. The main client object, `AWSClient`, accepts a `credentialProvider` parameter in its `init`. With this you can specify how the client should find AWS credentials. The default if you don't set the `credentialProvider` parameter is to select a method from the four methods listed below. Each method is tested in the order they are listed below and the first that is successful is chosen. If you are running on a Mac it ignores the ECS or EC2 methods as they would obviously fail.
+
+### Load Credentials from Environment Variable
+
+You can set the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` and `AWSClient` will automatically pick up the credentials from these variables.
+
+```bash
+AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID
+AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEY
+```
+
+### Via ECS Container credentials
+
+If you are running your code as an AWS ECS container task, you can [setup an IAM role](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html#create_task_iam_policy_and_role) for your container task to automatically grant credentials via the metadata service.
+
+### Via EC2 Instance Profile
+
+If you are running your code on an AWS EC2 instance, you can [setup an IAM role](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html) as the server's Instance Profile to automatically grant credentials via the metadata service.
+
+### Load Credentials from shared credential file.
+
+You can [set shared credentials](https://docs.aws.amazon.com/ses/latest/DeveloperGuide/create-shared-credentials-file.html) in the home directory for the user running the app, in the file `~/.aws/credentials`.
+
+```ini
+[default]
+aws_access_key_id = YOUR_AWS_ACCESS_KEY_ID
+aws_secret_access_key = YOUR_AWS_SECRET_ACCESS_KEY
+```
+
+## Pass Credentials to AWSClient directly
+
+If you would prefer to pass the credentials to the `AWSClient` directly you can specify credentials with the `.static` credential provider as follows.
+
+```swift
+let client = AWSClient(
+    credentialProvider: .static(
+        accessKeyId: "MY_AWS_ACCESS_KEY_ID",
+        secretAccessKey: "MY_AWS_SECRET_ACCESS_KEY"
+    )
+)
+```
+## Without Credentials
+
+Some services like CognitoIdentityProvider don't require credentials to access some of their functionality. In this case you should use the `.empty` credential provider. This will disable all other credential access functions and send requests unsigned.
+```swift
+let client = AWSClient(credentialProvider: .empty)
+```
+
+## Selector Credential Providers
+
+You can supply a list of credential providers you would like your `AWSClient` to use with the `.selector` credential provider. Each provider in the list is tested, until it finds a provider that successfully provides credentials. The following would test if credentials are available via environment variables, and then in the shared config file `~/.aws/credentials`.
+```swift
+let client = AWSClient(credentialProvider: .selector(.environment, .configfile()))
+```
+
+The default credential provider is implemented as a selector.
+```swift
+.selector(.environment, .ecs, .ec2, .configfile())
+```
+
+## STS and Cognito Identity
+
+The `CredentialProviders` protocol allows you to define credential providers external to the core library. The STS(Security Token Service) and Cognito Identity libraries both provide credential providers.
+
+STS extends `CredentialProviderFactory` with five new `CredentialProviders`.
+
+- `stsAssumeRole` for returning temporary credentials for a different role.
+- `stsSAML` for users authenticated via a SAML authentication response.
+- `stsWebIdentity` for users who have been authenticated in a mobile or web application with a web identity provider.
+- `stsWebIdentityTokenFile` for authenticating on EKS clusters. Set `AWS_WEB_IDENTITY_TOKEN_FILE`, `AWS_ROLE_ARN` and `AWS_ROLE_SESSION_NAME` environment variables.
+- `federationToken` for providing temporary credentials to federated users.
+- `sessionToken` for providing temporary credentials for the provided user with possible MFA authentication.
+
+See the AWS documentation on [requesting temporary security credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html).
+
+CognitoIdentity adds two versions of `cognitoIdentity` for users in a Cognito Identity Pool. The first version takes static identity details and the second you provide an object conforming to `IdentityProvider` which will be used to provide identity details when they are required. Identities can be users in a Cognito User Pool or users who authenticate with external providers such as Facebook, Google and Apple. See the AWS documentation on [Cognito Identity Pools](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-identity.html).
+
+For example, to use `STS.AssumeRole` to acquire new credentials you provide a request structure, credential provider to access original credentials and a region to run the STS commands in:
+
+```swift
+import STS
+
+let request = STS.AssumeRoleRequest(roleArn: "arn:aws:iam::000000000000:role/Admin", roleSessionName: "session-name")
+let client = AWSClient(credentialProvider: .stsAssumeRole(request: request, credentialProvider: .ec2, region: .euwest1))
+```
+
+Similarly you can setup a Cognito Identity credential provider as follows:
+
+```swift
+import CognitoIdentity
+
+let credentialProvider: CredentialProviderFactory = .cognitoIdentity(
+    identityPoolId: poolId,
+    logins: ["appleid.apple.com": "APPLETOKEN"],
+    region: .useast1
+)
+let client = AWSClient(credentialProvider: credentialProvider)
+```

SotoCore.docc/SotoCore/Articles/ServiceObjects.md

@@ -0,0 +1,75 @@
+# AWS Service Objects
+
+Defining an AWS service.
+
+In Soto each AWS Service has a service object. This object brings together an `AWSClient` and a service configuration, `AWSServiceConfig`, and provides methods for accessing all the operations available from that service.
+
+## Initialisation
+
+The `init` for each service is as follows. Details about each parameter are available below.
+
+```swift
+public init(
+    client: AWSClient,
+    region: SotoCore.Region? = nil,
+    partition: AWSPartition = .aws,
+    endpoint: String? = nil,
+    timeout: TimeAmount? = nil,
+    byteBufferAllocator: ByteBufferAllocator = ByteBufferAllocator(),
+    options: AWSServiceConfig.Options = []
+)
+```
+
+#### Client
+
+The client is the `AWSClient` this service object will use when communicating with AWS.
+
+#### Region and Partition
+
+The `region` defines which AWS region you want to use for that service. The `partition` defines which set of AWS server regions you want to work with. Partitions include the standard `.aws`, US government `.awsusgov` and China `.awscn`. If you provide a `region`, the `partition` parameter is ignored. If you don't supply a `region` then the `region` will be set as the default region for the specified `partition`, if that is not defined it will check the `AWS_DEFAULT_REGION` environment variable or default to `us-east-1`.
+
+Some services do not have a `region` parameter in their initializer, such as IAM. These services require you to communicate with one global region which is defined by the service. You can still control which partition you connect to though.
+
+#### Endpoint
+
+If you want to communicate with non-AWS servers you can provide an endpoint which replaces the `amazonaws.com` web address. You may want to do this if you are using an AWS mocking service for debugging purposes for example, or you are communicating with a non-AWS service that replicates AWS functionality.
+
+#### Time out
+
+Time out defines how long the HTTP client will wait until it cancels a request. This value defaults to 20 seconds. If you are planning on downloading/uploading large objects you should probably increase this value. `AsyncHTTPClient` allows you to set an additional connection timeout value. If you are extending your general timeout, use an `HTTPClient` configured with a shorter connection timeout to avoid waiting for long periods when a connection fails.
+
+#### ByteBufferAllocator
+
+During request processing the `AWSClient` will most likely be required to allocate space for `ByteBuffer`s. You can define how these are allocated with the `byteBufferAllocator` parameter.
+
+#### Options
+
+A series of flags, that can affect how requests are constructed. The only option available at the moment is `s3ForceVirtualHost`. S3 uses virtual host addressing by default except if you use a custom endpoint. `s3ForceVirtualHost` will force virtual host addressing even when you specify a custom endpoint.
+
+## AWSService
+
+All service objects conform to the `AWSService` protocol. This protocol brings along a couple of extra bits of functionality.
+
+### Presigned URLs
+
+When a request is made to AWS it has to be signed. This uses your AWS credentials and the contents of the request to create a signature. When the request is sent to the AWS server, the server also creates a version of this signature. If these two signatures match then AWS knows who is making the request and that it can be trusted. If you want to allow your clients to access AWS resources, creating a presigned request to send to your client is a common way to do this. Alternatively you would have to send AWS credentials to the client and these could be abused.
+
+One of the most common operations where this is used is for uploading an object to S3. Below creates a presigned URL which someone could use to upload a file to S3.
+```swift
+let signedURL = s3.signURL(
+    url: URL(string: "https://<bucketname>.s3.us-east-1.amazonaws.com/<key>")!,
+    httpMethod: .PUT,
+    expires: .minutes(60)
+).wait()
+```
+
+The function `signURL` returns an `EventLoopFuture<URL>` as it is dependent on a credential provider that may not have been resolved yet. In most cases though you are safe to just `wait` on the result as the credentials will be available.
+
+### Creating new service objects from existing
+
+It is possible to create a new version of a service object from an already existing one with additional `AWSServiceMiddleware`, an edited `timeOut`, `byteBufferAllocator` or `options` using the `AWSService.with(middlewares:timeout:byteBufferAllocator:options)` function.
+
+If you are loading a much larger object then usual into S3 and want to extend the `timeout` value for this one operation you can do it as follows.
+```swift
+s3.with(timeout: .minutes(10)).putObject(request)
+```