Invalid presigned S3 URL

[sluga]

Hi,

I'm attempting to generate a presigned URL. I've created an s3 object & verified that it works:

s3_object <- paws.storage::s3(
    credentials = list(
        cred = list(
            access_key_id     = creds$Credentials$AccessKeyId,
            secret_access_key = creds$Credentials$SecretAccessKey,
            session_token     = creds$Credentials$SessionToken
        ),
        profile = 'my-profile'
    ),
    region = 'my-region'
)

I then generated a URL via:

s3_url <- s3_object$generate_presigned_url(
    client_method = 'get_object',
    params = list(
        Bucket = "my-bucket",
        Key    = "my-file.json"
    ),
    expires_in = 600
)

This warning was emitted:

Warning message:
In charToRaw(object) : argument should be a character vector of length 1
all but the first element will be ignored

The URL I got was much longer compared to the URL I got from AWS CLI, about twice the number of characters. And while the AWS CLI URL works, I get the following message when I visit the URL generated by the package:

The request signature we calculated does not match the signature you provided. Check your key and signing method.

paws.storage version: 0.6.0

Am I doing something wrong or is this a bug?

[DyfanJones]

What version of paws.common do you have? Plus what region are you using? For Aws cli what signer did you use (v1 or V4)?

[sluga]

• 0.7.4
• eu-west-1
• the default, I believe that's v4

[DyfanJones]

Sadly I am unable to replicate this error even when I use eu-west-1 s3 bucket:

By default paws and boto3 use v1 signer to generate urls. However it looks like aws-cli uses v4 as default. I believe this is causing the main issue when comparing aws-cli and paws.

aws --profile paws s3 presign "s3://remove-paws-bucket/demo.txt"
#> [1] "https://remove-paws-bucket.s3.us-east-1.amazonaws.com/demo.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=DUMMY-V4%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240719T135640Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=e3b8867f45875a60579e237b7def98b415812a4dac1bddc9f197a3a46a0de27a"

library(paws.storage)

client = s3(config(credentials(profile = "paws"), region = "eu-west-1"))

client$generate_presigned_url(
  "get_object",
  params = list(
    Bucket = "remove-paws-bucket", Key = "demo.txt"
  ),
  expires_in = 600
)
#> [1] "https://remove-paws-bucket.s3.eu-west-1.amazonaws.com/demo.txt?AWSAccessKeyId=DUMMY-V1&Expires=1721397953&Signature=14WJMcSJ%2FGOlMGrz%2BgDEG7YC4Jk%3D"

^{Created on 2024-07-19 with reprex v2.1.1}

>>> import boto3
... 
... client = boto3.Session(profile_name = "paws", region_name = "eu-west-1").client("s3")
... 
... client.generate_presigned_url(
...   "get_object",
...   Params = {
...     "Bucket": "remove-paws-bucket", "Key": "demo.txt"
...   },
...   ExpiresIn = 600
... )
'https://remove-paws-bucket.s3.amazonaws.com/demo.txt?AWSAccessKeyId=DUMMY-V1&Signature=58tc50k1ZTwPYWjb%2FSOV2r1Rc4g%3D&Expires=1721398332'

To get the v4 signer you need to specify it in the client, this is similar to boto3.

library(paws.storage)

client = s3(config(credentials(profile = "paws"), region = "eu-west-1", signature_version = "s3v4"))

client$generate_presigned_url(
  "get_object",
  params = list(
    Bucket= "remove-paws-bucket", Key = "demo.txt"
  ),
  expires_in = 600
)
#> [1] "https://remove-paws-bucket.s3.eu-west-1.amazonaws.com/demo.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=DUMMY-V4%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20240719T135553Z&X-Amz-Expires=600&X-Amz-Signature=e835f78aef84924d536612c616f2bfb0b672b09b932dd11aabc3630408fc42df&X-Amz-SignedHeaders=host"

^{Created on 2024-07-19 with reprex v2.1.1}

>>> import boto3
>>> import boto3
... from botocore.config import Config
... 
... client = boto3.Session(profile_name = "paws", region_name = "eu-west-1").client("s3", config=Config(signature_version='s3v4'))
... 
... client.generate_presigned_url(
...   "get_object",
...   Params = {
...     "Bucket": "remove-paws-bucket", "Key": "demo.txt"
...   },
...   ExpiresIn = 600
... )
'https://remove-paws-bucket.s3.amazonaws.com/demo.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=DUMMY-V4%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20240719T141010Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=e40a0ff560bfa5f31629ca11fa7657fc93311efcd648428d3ae0f662c910b9e9'

From this you should see that paws and boto3 v4 signer looks alot similar to aws-cli presign url.
I hope this helps :)

[sluga]

Thanks @DyfanJones, that worked!

(I did look for signature settings before but I was looking in the wrong place (?s3_generate_presigned_url))