Skip to content

Latest commit

 

History

History
137 lines (109 loc) · 6.13 KB

README.md

File metadata and controls

137 lines (109 loc) · 6.13 KB
Raw

landsatlinks

Creating download urls for Landsat Collection 2 Level 1 product bundles using the USGS/EROS Machine-to-Machine API

Python Version License

landsatlinks offers a simple command line interface to retrieve download links for Landsat Collection 2 Level 1 product bundles through the USGS/EROS machine-to-machine API.

Please note

The USGS M2M login endpoint for username/password will be deprecated in February of 2025. Please make sure to create an application token and adapt your workflows by using a secrets file (see the --secret argument below).
More info on how to create an app token here

Requirements

User credentials to log in to the USGS EarthExplorer interface are required. Your user account needs to have access to the machine-to-machine API, which can be requested through the user profile page here.
Python >= 3.6 is required.
aria2 is required to download product bundles (Linux only). You can still create links and download them manually if aria2 is not available.

Installation

Install using pip:

python -m pip install git+https://github.com/ernstste/landsatlinks.git

or download the Docker image (recommended):

docker pull ernstste/landsatlinks:latest

Usage

Landsatlink provides two sub-programs available: search and download
search retrieves the download links for a given search query and can download the product bundles right away.
download will download product bundles from a list of download links that were created with search before.

landsatlinks search
There are two mandatory arguments required to run the tool and several optional arguments that allow a more detailed filtering of search results. A call may look like this:

search demo

required arguments:

  • aoi
    The area of interest. Valid input:
    a) .txt - text file containing one tile per line in the format PPPRRR (P = path, R = row)
    Keep padding zeroes! Good: 194023, bad: 19432
    b) .shp, .gpkg, .geojson - vector file containing point, line, or polygon geometries.
  • output-dir
    The directory where the file containing the download URLs or downloaded products will be stored.
    The --download option will deactivate saving of URLs.

optional arguments:

  • -s | --sensor
    Restrict results to specific sensor(s).
    choices = 'TM', 'ETM', 'OLI' (Landsat 4/5, Landsat 7, Landsat 8/9)
    Default: All sensors

  • -d | --daterange
    Start date and end date = date range to be considered.
    Format: YYYYMMDD,YYYYMMDD
    Default: full archive until today.

  • -c | --cloudcover
    Percent (land) cloud cover range to be considered.
    Default: 0,100

  • -m | --months
    Seasonal filter: define the months to be considered.
    Default: 1,2,3,4,5,6,7,8,9,10,11,12

  • -i | --ingestrange
    ingestion time filter: define the time range in which data was added to the USGS EROS archive. This can be used to exclude old L9 images which have been reprocessed.
    Format: YYYYMMDD,YYYYMMDD
    Default: beginning of the archive until today.

  • -t | --tier
    Landsat collection tier level.
    Valid tiers: T1,T2,RT
    Default: T1

  • -l | --level
    Landsat level of processing.
    Valid levels: L1TP,L1GT,L1GS
    Default: L1TP

  • --download
    Download the product bundles directly after creating the download links.

  • -n | --no-action
    Only search for product bundles and print info about search results without generating links or downloading.

  • -f | --forcelogs
    Path to FORCE log file directory (Level-2 processing logs, directory will be searched recursively)
    Links will only be generated for products that haven't been processed by FORCE yet.

  • -q | --queue-file
    Path to FORCE queue file.
    Downloaded product bundle file paths will be appended to the queue.

  • --secret
    Path to the file containing the username and password for M2MApi access (EarthExplorer login).
    Avoids having to enter credentials every time the tool is run.
    a) 1st line: user, 2nd line: password - deprecated by the USGS API from February 2025
    b) 1st line: 'app-token', 2nd line: user, 3rd line: token

Example:

landsatlinks search ~/berlin.shp ~/level1 -s OLI -d 20180101,20201231 -m 10,11 -c 0,70 --secret ~/.m2m.txt --no-action

Sensor(s): OLI
Tile(s): 192023,192024,193023,193024
Date range: 2018-01-01 to 2020-12-31
Included months: 10,11
Cloud cover: 0% to 70%

20 Landsat Level 1 scenes matching criteria found
22.13 GB data volume found

landatlinks download

  • url-file
    Path to the file containing the download links.
  • output-dir
    The directory where the product bundles will be stored.
  • -q | --queue-file
    Path to FORCE queue file. Downloaded product bundle file paths will be appended to the queue.

Example:

landsatlinks download ~/landsatlinks/urls_landsat_TM_ETM_OLI_20221001T174038.txt ~/landsatlinks/

Loading urls from ~/landsatlinks/test/urls_landsat_TM_ETM_OLI_20221001T174038.txt

6 of 116 product bundles found in filesystem, 110 left to download.

Downloading: 5%|===>                                    | 6/110 [08:36<2:29:13, 100.97s/pproduct bundle/s]

Gotchas

The output directory will be checked recursively (i.e. including all subfolders) for existing product bundles and download URLs are only created for product bundles that were not found in the filesystem. All directories, .tar files, and .tar.gz files that match the Landsat Collections Level-1 naming convention are considered. Partial downloads (product bundles that are accompanied by .aria2 files) will be continued.

The M2M API is rate limited to 15,000 requests/15min. If you exceed this limit, landsatlinks will wait for 15 minutes and continue afterwards. Checking for existing product bundles in the output directory happens before generating download URLs to reduce using unnecessary requests.

License

MIT