How to config Finder Job memory for S3 transfer task?

[YikaiHu]

Finder nodes will load the filenames and sizes of all objects from the destination bucket or destination prefix (if you have specified a specific prefix).
Then, they will scan the objects in the source bucket or source prefix (if you have specified a specific prefix) to determine which data needs to be transferred.

Therefore, the amount of memory required for a Finder Job depends on the total number of objects in your dataset. Please refer to the table below for recommendations:

Total Object #*	Recommended Memory
<20 Million	8GB
40 Million	16GB
80 Million	32GB
160 Million	64GB
320 Million	128GB
640 Million	256GB
960 Million	384GB
1280 Million	512GB
>1280 Million	Doesn't support for now. Recommend dividing into multiple tasks. Or using Finder Depth and Finder Number to tuning the task.

Total Object # refers to the total number of unique objects from both the Source and Destination after removing duplicates.