- A Linux based OS (had problems using on Windows)
- Python 3.7+
- The following modules from pip3:
oauth2client
,google-api-python-client
,progress
&httplib2shim
- Figure out how many projects you will need to make. For example, a 100TB clone job will take approximately 135 SAs to make a full clone. Each project can have a maximum limit of 100 SAs. Incase of the 100TB job, we will need 2 projects.
- Head over to https://console.developers.google.com/ and sign in with your account.
- Click "Library" on the left column, then click on "Select a project" at the top. Click on
NEW PROJECT
on the top-right corner of the new window. - In the Project name section, input any project name you chose, followed by the number 1. For example,
copyjob1
. After that, pressCREATE
. - Repeat Steps 3 & 4 to create the amount of projects needed for your clone job, while keeping the first part of the project name same and increasing the number at the end for each new project needed. For example, we need to make
copyjob1
andcopyjob2
project for our 100TB clone job. - Wait till the project creation is done and then click on "Select a project" again at the top and select your first project.
- In the "Search for APIs & Services" search bar, look for "Google Drive API", click on it and press
ENABLE
. Do the same for "Identity and Access Management (IAM) API". - Click on the Google APIs logo on the top-left and then click on Credentials on the left pane. Next, press the blue
Create credentials
button and select "Service account key". - Make sure the selected Key type is JSON. Click on
Select...
and click onNew service account
. - Select the Service account name the same as the project name. For example, the Service account name for our 100TB job will be
copyjob1
. - Click on
Select a role
and scroll down toService Accounts
. From there, enableCreate Service Accounts
andService Account Key Admin
. After that, press the blueCreate
button on the main screen. You should be prompted to download a JSON file. Save it. - Repeat Steps 7 to 11 for all your projects.
- Place the JSON files in the same folder as the 3 scripts are in. Rename the first project JSON file to controller1.json, the 2nd JSON to controller2.json and so on.
- Open terminal in the scripts folder and run the following command.
python3 serviceaccountfactory.py 1 [number of projects] [accountprefix of your choice] [project prefix]
. Replace[number of projects]
with the total number of projects used,[accountprefix of your choice]
with prefix of your choice and[project prefix]
with the project name used in Step 4. - If you have set everything up correctly, you should see the script making 99 service accounts of each project.
- Once the script is done making all the accounts, open Google Drive and make a new TeamDrive to copy all the files to.
- Add
[projectname1]@[projectname1].iam.gserviceaccount.com
as a Manager to the TD. Replace[projectname1]
with the name of your first project. For example, the first project name for our example was copyjob1, so the email address will becopyjob1@copyjob1.iam.gserviceaccount.com
. - Run the following command
python3 masshare.py [TDFolderID] 1 [number of projects] [accountprefix of your choice] [project prefix]
. Replace the[TDFolderID]
withXXXXXXXXXXXXXXXXXXXXXXXXX
. The Folder ID can be obtained from the TD Folder Linkhttps://drive.google.com/drive/folders/XXXXXXXXXXXXXXXXXXXXXXXXX
. This will add all the service accounts to your TD. - After that, run the following command,
python3 folderclone.py 1 [SourceFolderID] [TDFolderID]
. Replace[SourceFolderID]
with the folder ID of the folder you are trying to copy and replace[TDFolderID]
with the same ID as used in step 18. It should start cloning the source folder to the TD at this point if everything was done correctly.