cannot recognize global credential and s3 arn

[joyfangcheng]

hello all,

i'm following this guidance https://github.com/awslabs/data-transfer-hub/blob/main/docs/S3_PLUGIN.md#data-transfer-hub---s3-plugin to deploy s3 plugin in AWS China region, to transfer objects from China to global. Source bucket is located in AWS China, destination bucket is located in AWS Ireland.

After deployment has finished, i notice that in FinderStackFinderRoleA666A5A8, destCredential and destBucket arn cannot be recognized as global arn, I tried to change it to global arn, but obviously it failed :( Would like to know if here's anyone getting same requirement and how to solve it?

Thanks a lot,
Joy FANG

[YikaiHu]

Hi @joyfangcheng , could you provide your Cloudformation Parameters?
And please follow our latest documentation: https://awslabs.github.io/data-transfer-hub/en/user-guide/tutorial-s3/#using-the-s3-plugin

[joyfangcheng]

Hello @YikaiHu,

I tried with latest stack again, but got the same issue, destCredential could not be recognized as global arn

error in cloudwatch:
2024/05/03 03:43:53 Unable to list objects in / - operation error S3: ListObjectsV2, https response error StatusCode: 400, RequestID: 2X2J1S6VEHGE66CX, HostID: c5uXnKsGp5PP4sHYs72ml4GjlgegOu5Wih35j14xvpyun4GjwwZBqyLiaubF6AXWpKhmXyC/FzM=, api error InvalidToken: The provided token is malformed or otherwise invalid.

Any comments would be really apprciated.

B.R.
Joy FANG

[YikaiHu]

Hi @joyfangcheng ,
The information for the region‘s ARN in the credentials provided in your DefaultPolicyBB6BAB25 is correct. This is because it is the ARN of the Secrets where your Access Key and Secrets Key are stored, which are both located in the same region where you deployed your DTH.

So I have two questions may be can find out the root cause:

1. It appears that both your S3 source and S3 destination are not in the same account as your DTH deployment. Is this correct? So, the actual setup is: Source S3 (in Account A - cn-northwest-1) -> DTH (in Account B - cn-northwest-1) -> Destination S3 (in Account C - eu-west-1), correct?
2. If the answer of question 1 is correct, please check the format of your secret stored in AWS Secrets Manager. Your secret must adhere to the following format:

{
  "access_key_id": "<Your Access Key ID>",
  "secret_access_key": "<Your Access Key Secret>"
}

[YikaiHu]

Oh I see the root cause.
If you do not need to use source prefixlist feature, you can just leave SourcePrefixListBucket empty.

[YikaiHu]

@joyfangcheng

It's wired, I leave it blank and the creation is successful.

Below is my parameters:

And the template link is : https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template

[YikaiHu]

@joyfangcheng , we have finded out the root cause.

This failure is due to changes in the underlying IAM logic implemented by AWS. As a result, this issue may occur in new accounts but not in my old account.

Please using the following patch template to create the Transfer Task, and please leave SourcePrefixListBucket blank.

DataTransferS3Stack-fix-0509.json

[YikaiHu]

We will plan a patch in next version

[joyfangcheng]

[YikaiHu]

Hi @joyfangcheng , we noticed that you have posted an potential AWS AKSK in github.
This credential is no longer secure.
Please deactivate it as soon as possible!!