Export API

Table of Contents

Overview

• Overview

Exports

• Get an export of a repository version
• Create an export of a repository version
• Determine the status of an export file
• Delete an export file

Other

• Useful HTTPie commands

Overview

The API provides an export endpoint for creating, fetching, and deleting specific repository versions. Requesting an export returns a status of 303 See Other with the Location in the response header if the file already exists, 102 Processing if it is being processed, or a 204 No Content if the export does not exist or is still being processed. The filename of the export includes the lastUpdated timestamp of the export, which can be used to fetch the diff between an export and the current state of a source or collection. The Location is designed to be used only once -- to ensure that the Location works correctly, you must request another Location each time you want to download an export file.

Exports are a compressed tar of the JSON results. The JSON results are the equivalent of the following REST API requests:

GET /[:ownerType/]:owner/:repoType/:repo/:repoVersion/?includeConcepts=true&includeMappings=true&includeRetired=true&limit=0

In the above:

• :ownerType is "orgs" or "users", or it is omitted if :owner is "user"
• :owner is a username if :ownerType is "users", organization ID if :ownerType is "orgs", or "user"
• :repoType is "sources" or "collections"
• :repo is a source or collection ID, depending on the value of :repoType
• :repoVersion is a source or collection version ID

The export filename takes the following form:

[:repositoryId]_[:repoVersion].[:lastUpdated].tar

To fetch the diff between a lastUpdated timestamp and the current state of a source or collection:

GET /[:ownerType/]:owner/:repoType/:repo/:repoVersion/?includeConcepts=true&includeMappings=true&includeRetired=true&limit=0&updatedSince=:lastUpdated

Refer to sources and collections documentation for details on the above requests.

The Subscriptions documentation describes how the export functionality can be used to subscribe to a source or collection to keep a client system in synch.

Future Considerations

• Add lastUpdated into the response header of GET request (right now lastUpdated is only stored in the filename)
• The current status and progress of the creation of a repository export may be made available through the Flower package

Get an export of a repository version

• Get the export URL for the specified repository version. This has three possible results:
  • If the export exists, returns a status code of 303 See Other with the Location property in the response header set to the fully specified download URL
  • If the export file does not exist but the URL is correct, returns 204 No Content
  • If the export URL is non-existent, returns 404 Not Found

GET /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/
HEAD /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

• Notes
  • :repoVersion is required - exports can only be created for repository versions. If HEAD version is requested, the API will return 405 Not Allowed.
  • The API returns the fully specified URL of the export file in the Location attribute of the response header. The Location is designed to be used only once -- to ensure that the Location works correctly, you must request another Location each time you want to download an export file.
  • This request first performs a check as to whether the export file already exists. If it exists, it then generates an Location. These actions are performed on an Amazon Web Server and may take up to 30 seconds to process. The timeout of the client system making the request should set its timeout period accordingly.

Example

• Fetch the export URL for v2016-08-22 of the CIEL source

GET /orgs/CIEL/sources/CIEL/v2016-08-22/export/

• Fetch the export URL for v1.2 of the CIEL Starter Set

GET /orgs/CIEL/collections/StarterSet/v1.2/export/

Response

• If the export file exists:

Status: 303 See Other
Response Header:
Location: https://ocl-source-export-production.s3.amazonaws.com/CIEL/CIEL_55572f688a86f24b48d935be.20150516122820.tgz?Signature=fmBSI6hL9IhN4mu4W5x%2FPFs5uxw%3D&Expires=1431864368&AWSAccessKeyId=...

• If the export file does not exist:

Status: 204 No Content
Response Header:
Location: 

• If the URL is non-existent

Status: 404 Not Found

• If the request is otherwise invalid - return the appropriate error code

Create an export of a repository version

• Create an export file for the specified repository version; if it already exists, no action is taken

POST /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

• Notes
  • :repoVersion is required. If HEAD version is requested, the API will return 405 Not Allowed.
  • This request only triggers the creation of the export file and does not return the Location. It is necessary to follow up with a GET request after the file has been processed in order to get the Location

Example

• Create the export file for v2.2 of the CIEL source

POST /orgs/CIEL/sources/CIEL/v2.2/export/

Response

• If no export file already exists and processing is initiated:

Status: 202 Accepted

• If an export file is currently being processed:

Status: 409 Conflict

• If the export file already exists

Status: 303 See Other
Response Header:
Location: https://ocl-source-export-production.s3.amazonaws.com/CIEL/CIEL_55572f688a86f24b48d935be.20150516122820.tgz?Signature=fmBSI6hL9IhN4mu4W5x%2FPFs5uxw%3D&Expires=1431864368&AWSAccessKeyId=...

• If the request is otherwise invalid - return the appropriate error code

Delete an export file

• Deletes a specific export file

DELETE /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

• Notes
  • If HEAD version is requested, the API will return 405 Not Allowed.
  • The passed authorization token must have administrative access to the repository in order to delete the export file

Example

• Create the export file for v2.2 of the CIEL source

DELETE /orgs/CIEL/sources/CIEL/v2.2/export/

Response

• If the file exists, it is deleted with 200

Status: 200 Success

• If the file does NOT exist

Status: 204 No Content

• If the URL or request is otherwise invalid - return the appropriate error code