Invalid copy source encoding

[jwijffels]

I have a file which is on S3 which basically looks like this 202408050803-lum_v2_gs_nl-%13668%-1002120667.wav. If I want to copy it to another folder, I'm getting 'invalid source encoding' due to that '%' character.

>        s3_file_copy(path = info$file[i], new_path = info$new_path[i], overwrite = TRUE)
Error: InvalidArgument (HTTP 400). Invalid copy source encoding

[DyfanJones]

Sorry about that. I will have to see if it is an aws thing, a paws sdk issue or an s3fs issue

[DyfanJones]

From what I can see aws says to avoid % characters in s3 uris (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html):

Characters to avoid
We recommend that you don't use the following characters in a key name because of significant special character handling, which isn't consistent across all applications.

• Backslash ("")
• Left curly brace ("{")
• Non-printable ASCII characters (128–255 decimal characters)
• Caret ("^")
• Right curly brace ("}")
• Percent character ("%")
• Grave accent / back tick ("`")
• Right square bracket ("]")
• Quotation marks
• 'Greater Than' symbol (">")
• Left square bracket ("[")
• Tilde ("~")
• 'Less Than' symbol ("<")
• 'Pound' character ("#")
• Vertical bar / pipe ("|")

[jwijffels]

Thanks for the reply.
I understand that that symbol should not have been used but unfortunately I don't have possibility to change the file name. It's coming from an external source.

[DyfanJones]

No worries it looks like an issues with s3fs

library(paws)
library(s3fs)
client = s3(config(credentials(profile = "paws")))

resp <- client$put_object(
  Bucket = "paws-rm",
  Key = "rm/%01file%/output.txt",
  Body = charToRaw("helloworld")
)

s3_file_system(profile_name = "paws")
s3_dir_ls("s3://paws-rm", recurse = T)
#> [1] "s3://paws-rm/rm/%01file%/output.txt"

^{Created on 2024-08-06 with reprex v2.1.1}

[jwijffels]

yes, probably s3_file_copy("s3://paws-rm/rm/%01file%/output.txt", "s3://paws-rm/rm2/%01file%/output.txt") will give that error

[jwijffels]

This does seem to work

s3_file_download("s3://paws-rm/rm/%01file%/output.txt", "output.txt")
s3_file_download("output.txt", "s3://paws-rm/rm3/%01file%/output.txt")

[jwijffels]

FYI. I noticed the same issue occurs if you use s3_dir_copy and in that folder on S3 there is a file with that '%' symbol

s3_dir_copy("s3://paws-rm/rm", "s3://paws-rm/rm2")

[DyfanJones]

It looks like this issue is coming from the paws sdk paws-r/paws#819

[DyfanJones]

Try the latest paws.common development version:

remotes::install_github("dyfanjones/paws/paws.common")