Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update devguide for endpointsv2 #2205

Merged
merged 2 commits into from
Jul 31, 2023
Merged

update devguide for endpointsv2 #2205

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c4522e0
update devguide for endpointsv2
lucix-aws Jul 24, 2023
161a3d0
update content
lucix-aws Jul 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
364 changes: 326 additions & 38 deletions content/en/docs/configuring-sdk/endpoints.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,39 +5,251 @@ date: "2020-11-12"
description: Customizing service client endpoints.
---

The {{% alias sdk-go %}} provides the ability to configure a custom endpoint to be used for a service. In most cases you use a pre-configured
endpoint for a service. Configuring custom endpoints, lets you do more, such as working with pre-release versions
of a service.

A [EndpointResolver]({{< apiref "aws#EndpointResolver" >}}) can be configured to provide custom endpoint resolution
logic for service clients. You can use a custom endpoint resolver to override a service's endpoint resolution logic
for all endpoints, or a just specific regional endpoint. Custom endpoint resolver can trigger the service's endpoint
resolution logic to fallback if a custom resolver does not wish to resolve a requested endpoint.
[EndpointResolverWithOptionsFunc]({{< apiref "aws#EndpointResolverWithOptionsFunc" >}}) can be used to easily wrap functions to satisfy the
`EndpointResolverWithOptions` interface.

A `EndpointResolver` can be easily configured by passing the resolver wrapped with
[WithEndpointResolverWithOptions]({{< apiref "config#WithEndpointResolverWithOptions" >}}) to
[LoadDefaultConfig]({{< apiref "config#LoadDefaultConfig" >}}), allowing for the ability to override endpoints when
loading credentials, as well as configuring the resulting `aws.Config` with your custom endpoint resolver.

The endpoint resolver is given the service and region as a string, allowing for the resolver to dynamically drive its
behavior. Each service client package has an exported `ServiceID` constant which can be used to determine which service
{{% pageinfo color="warning" %}}
Endpoint resolution is an advanced SDK topic. By changing these settings
you risk breaking your code. The default settings should be applicable to
most users in production environments.
{{% /pageinfo %}}

The {{% alias sdk-go %}} provides the ability to configure a custom
endpoint to be used for a service. In most cases, the default
configuration will suffice. Configuring custom endpoints allows for
additional behavior, such as working with pre-release versions of a
service.

## Customization

There are two "versions" of endpoint resolution config within the SDK.
* v2, released in Q3 of 2023, configured via:
* `EndpointResolverV2`
* `BaseEndpoint`
* v1, released alongside the SDK, configured via:
* `EndpointResolver`

We recommend users of v1 endpoint resolution migrate to v2 to obtain access to
newer endpoint-related service features.

## V2: `EndpointResolverV2` + `BaseEndpoint`

In resolution v2, `EndpointResolverV2` is the definitive mechanism through
which endpoint resolution occurs. The resolver's `ResolveEndpoint` method is
invoked as part of the workflow for every request you make in the SDK. The
hostname of the `Endpoint` returned by the resolver is used **as-is** when
making the request (operation serializers can still append to the HTTP path,
however).

Resolution v2 includes an additional client-level config, `BaseEndpoint`, which
is used to specify a "base" hostname for the instance of your service. The
value set here is not definitive-- it is ultimately passed as a parameter to
the client's `EndpointResolverV2` when final resolution occurs (read on for
more information about `EndpointResolverV2` parameters). The resolver
implementation then has the opportunity to inspect and potentially modify that
value to determine the final endpoint.

For example, if you perform an S3 `GetObject` request against a given bucket
with a client where you've specified a `BaseEndpoint`, the default resolver
will inject the bucket into the hostname if it is virtual-host compatible
(assuming you haven't disabled virtual-hosting in client config).

In practice, `BaseEndpoint` will most likely be used to point your client at a
development or preview instance of a service.

### `EndpointResolverV2` parameters

Each service takes a specific set of inputs which are passed to its resolution
function, defined in each service package as `EndpointParameters`.

Every service includes the following base parameters, which are used to
facilitate general endpoint resolution within AWS:

| name | type | description |
|----------------|----------|------------------------------------------------------------|
| `Region` | `string` | The client's AWS region |
| `Endpoint` | `string` | The value set for `BaseEndpoint` in client config |
| `UseFips` | `bool` | Whether FIPS endpoints are enabled in client config |
| `UseDualStack` | `bool` | Whether dual-stack endpoints are enabled in client config |

Services can specify additional parameters required for resolution. For
example, S3's `EndpointParameters` include the bucket name, as well as several
S3-specific feature settings such as whether virtual host addressing is
enabled.

If you are implementing your own `EndpointResolverV2`, you should never need to
construct your own instance of `EndpointParameters`. The SDK will source the
values per-request and pass them to your implementation.

### A note about Amazon S3
Amazon S3 is a complex service with many of its features modeled through
complex endpoint customizations, such as bucket virtual hosting, S3 MRAP, and
more.

Because of this, we recommend that you don't replace the `EndpointResolverV2`
implementation in your S3 client. If you need to extend its resolution
behavior, perhaps by sending requests to a local development stack with
additional endpoint considerations, we recommend wrapping the default
implementation such that it delegates back to the default as a fallback (shown
in examples below).

### Examples

#### With `BaseEndpoint`

The following code snippet shows how to point your S3 client at a local
instance of a service, which in this example is hosted on the loopback device
at port 8080.

```go
client := s3.NewFromConfig(cfg, func (o *svc.Options) {
o.BaseEndpoint = aws.String("https://localhost:8080/")
})
```

#### With `EndpointResolverV2`

The following code snippet shows how to inject custom behavior into S3's
endpoint resolution using `EndpointResolverV2`.

```go
import (
"context"
"net/url"

"github.com/aws/aws-sdk-go-v2/service/s3"
smithyendpoints "github.com/aws/smithy-go/endpoints"
)

type resolverV2 struct {
// you could inject additional application context here as well
}

func (*resolverV2) ResolveEndpoint(ctx context.Context, params s3.EndpointParameters) (
smithyendpoints.Endpoint, error,
) {
if /* input params or caller context indicate we must route somewhere */ {
return smithyEndpoints.Endpoint{
URI: url.Parse("https://custom.service.endpoint/"),
}
}

// delegate back to the default v2 resolver otherwise
return s3.NewDefaultEndpointResolverV2().ResolveEndpoint(ctx, params)
}

func main() {
// load config...

client := s3.NewFromConfig(cfg, func (o *s3.Options) {
o.EndpointResolverV2 = &resolverV2{
// ...
}
})
}
```

#### With both

The following sample program demonstrates the interaction between
`BaseEndpoint` and `EndpointResolverV2`. **This is an advanced use case:**

```kotlin
import (
"context"
"fmt"
"log"
"net/url"

"github.com/aws/aws-sdk-go-v2"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
smithyendpoints "github.com/aws/smithy-go/endpoints"
)

type resolverV2 struct {}

func (*resolverV2) ResolveEndpoint(ctx context.Context, params s3.EndpointParameters) (
smithyendpoints.Endpoint, error,
) {
// s3.Options.BaseEndpoint is accessible here:
fmt.Printf("The endpoint provided in config is %s\n", *params.Endpoint)

// fallback to default
return s3.NewDefaultEndpointResolverV2().ResolveEndpoint(ctx, params)
}

func main() {
cfg, err := config.LoadDefaultConfig(context.Background()
if (err != nil) {
log.Fatal(err)
}

client := s3.NewFromConfig(cfg, func (o *s3.Options) {
o.BaseEndpoint = aws.String("https://endpoint.dev/")
o.EndpointResolverV2 = &resolverV2{}
})

// ignore the output, this is just for demonstration
client.ListBuckets(context.Background(), nil)
}
```

When run, the above program outputs the following:

```
The endpoint provided in config is https://endpoint.dev/
```


## V1: `EndpointResolver`

{{% pageinfo color="warning" %}}
Endpoint resolution v1 is retained for backwards compatibility and is isolated
from the modern behavior in endpoint resolution v2. It will only be used if the
`EndpointResolver` field is set by the caller.

Use of v1 will most likely prevent you from accessing endpoint-related service
features introduced with or after the release of v2 resolution. See "Migration"
for instructions on how to upgrade.
{{% /pageinfo %}}

A [EndpointResolver]({{< apiref "aws#EndpointResolver" >}}) can be configured
to provide custom endpoint resolution logic for service clients. You can use a
custom endpoint resolver to override a service's endpoint resolution logic for
all endpoints, or a just specific regional endpoint. Custom endpoint resolver
can trigger the service's endpoint resolution logic to fallback if a custom
resolver does not wish to resolve a requested endpoint.
[EndpointResolverWithOptionsFunc]({{< apiref
"aws#EndpointResolverWithOptionsFunc" >}}) can be used to easily wrap functions
to satisfy the `EndpointResolverWithOptions` interface.

A `EndpointResolver` can be easily configured by passing the resolver wrapped
with [WithEndpointResolverWithOptions]({{< apiref
"config#WithEndpointResolverWithOptions" >}}) to [LoadDefaultConfig]({{< apiref
"config#LoadDefaultConfig" >}}), allowing for the ability to override endpoints
when loading credentials, as well as configuring the resulting `aws.Config`
with your custom endpoint resolver.

The endpoint resolver is given the service and region as a string, allowing for
the resolver to dynamically drive its behavior. Each service client package has
an exported `ServiceID` constant which can be used to determine which service
client is invoking your endpoint resolver.

An endpoint resolver can use the [EndpointNotFoundError]({{< apiref "aws#EndpointNotFoundError" >}}) sentinel error
value to trigger fallback resolution to the service clients default resolution logic. This allows you to selectively
override one or more endpoints seamlessly without having to handle fallback logic.
An endpoint resolver can use the [EndpointNotFoundError]({{< apiref
"aws#EndpointNotFoundError" >}}) sentinel error value to trigger fallback
resolution to the service clients default resolution logic. This allows you to
selectively override one or more endpoints seamlessly without having to handle
fallback logic.

If your endpoint resolver implementation returns an error other than `EndpointNotFoundError`, endpoint resolution will
stop and the service operation returns an error to your application.
If your endpoint resolver implementation returns an error other than
`EndpointNotFoundError`, endpoint resolution will stop and the service
operation returns an error to your application.

## Examples
### Examples

### Overriding Endpoint with Fallback
#### With fallback

The following code snippet shows how a single service endpoint can be overridden for {{% alias service=DDB %}} with
fallback behavior for other endpoints:
The following code snippet shows how a single service endpoint can be
overridden for {{% alias service=DDB %}} with fallback behavior for other
endpoints:

```go
customResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
Expand All @@ -55,10 +267,11 @@ customResolver := aws.EndpointResolverWithOptionsFunc(func(service, region strin
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithEndpointResolverWithOptions(customResolver))
```

### Overriding Endpoint Without Fallback
#### Without fallback

The following code snippet shows how a single service endpoint can be overridden for {{% alias service=DDB %}} without
fallback behavior for other endpoints:
The following code snippet shows how a single service endpoint can be
overridden for {{% alias service=DDB %}} without fallback behavior for other
endpoints:

```go
customResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
Expand All @@ -75,18 +288,21 @@ customResolver := aws.EndpointResolverWithOptionsFunc(func(service, region strin
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithEndpointResolverWithOptions(customResolver))
```

### Immutable Endpoints
### Immutable endpoints

{{% pageinfo color="warning" %}}
Setting an endpoint as immutable may prevent some service client features from functioning correctly, and could result
in undefined behavior. Caution should be taken when defining an endpoint as immutable.
Setting an endpoint as immutable may prevent some service client features from
functioning correctly, and could result in undefined behavior. Caution should
be taken when defining an endpoint as immutable.
{{% /pageinfo %}}

Some service clients, such as {{% alias service=S3 %}}, may modify the endpoint returned by the resolver for certain
service operations. For example, the {{% alias service=S3 %}} will automatically handle
[Virtual Bucket Addressing](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html) by mutating the
resolved endpoint. You can prevent the SDK from mutating your custom endpoints by setting
[HostnameImmutable]({{< apiref "aws#Endpoint.HostnameImmutable" >}}) to`true`. For example:
Some service clients, such as {{% alias service=S3 %}}, may modify the endpoint
returned by the resolver for certain service operations. For example, {{% alias
service=S3 %}} will automatically handle [Virtual Bucket
Addressing](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html)
by mutating the resolved endpoint. You can prevent the SDK from mutating your
custom endpoints by setting [HostnameImmutable]({{< apiref
"aws#Endpoint.HostnameImmutable" >}}) to `true`. For example:

```go
customResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
Expand All @@ -104,3 +320,75 @@ customResolver := aws.EndpointResolverWithOptionsFunc(func(service, region strin
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithEndpointResolverWithOptions(customResolver))
```

## Migration

When migrating from v1 to v2 of endpoint resolution, the following general principles apply:
* Returning an [Endpoint]({{< apiref "aws#Endpoint" >}}) with [HostnameImmutable]({{<
apiref "aws#Endpoint.HostnameImmutable" >}}) set to `false` is roughly
equivalent to setting `BaseEndpoint` to the originally returned URL from v1 and
leaving `EndpointResolverV2` as the default.
* Returning an Endpoint with HostnameImmutable set to `true` is roughly
equivalent to implementing an `EndpointResolverV2` which returns the
originally returned URL from v1.

Examples for these cases are provided below.

{{% pageinfo color="warning" %}}
V1 immutable endpoints and V2 resolution are not equivalent in behavior. For
example, signing overrides for custom features like S3 Object Lambda would
still be set for immutable endpoints returned via v1 code, but the same will
not be done for v2.
{{% /pageinfo %}}

### Examples

#### Mutable endpoint

The following code sample demonstrates how to migrate a basic v1 endpoint
resolver that returns a modifiable endpoint:

```go
// v1
client := svc.NewFromConfig(cfg, func (o *svc.Options) {
o.EndpointResolver = svc.EndpointResolverFromURL("https://custom.endpoint.api/")
})

// v2
client := svc.NewFromConfig(cfg, func (o *svc.Options) {
// the value of BaseEndpoint is passed to the default EndpointResolverV2
// implementation, which will handle routing for features such as S3 accelerate,
// MRAP, etc.
o.BaseEndpoint = aws.String("https://custom.endpoint.api/")
})
```

#### Immutable endpoint
```go
// v1
client := svc.NewFromConfig(cfg, func (o *svc.Options) {
o.EndpointResolver = svc.EndpointResolverFromURL("https://custom.endpoint.api/", func (e *aws.Endpoint) {
e.HostnameImmutable = true
})
})

// v2
import (
smithyendpoints "github.com/aws/smithy-go/endpoints"
)

type staticResolver struct {}

func (*staticResolver) ResolveEndpoint(ctx context.Context, params svc.EndpointParameters) (
smithyendpoints.Endpoint, error,
) {
// This value will be used as-is when making the request.
return smithyEndpoints.Endpoint{
URI: url.Parse("https://custom.endpoint.api/"),
}
}

client := svc.NewFromConfig(cfg, func (o *svc.Options) {
o.EndpointResolverV2 = &staticResolver{}
})
```

Loading