Skip to content

Current Status of STAC Implementation

Jump to bottom
Carmen Tawalika edited this page Sep 29, 2022 · 14 revisions

ACTINIA STAC PLUGIN

STAC plugin empowers Actinia with the possibility to organize and access STAC collections components. The plugin has two main concepts:

  • Instances: The concept refers to a virtual space where the user can store multiples STAC Collections.
  • Collections: it refers to the STAC Collections stored inside an instance. Noteworthy to mention that the collection appropriate an internal ID structured as stac.<stac_instance_id>.rastercube.<stac_collection_id>

GET Method

Instances Endpoint

  • Structure : /api/<version>/stac/instances or /api/<version>/stac/instances/<stac_instance_id>
  • Definition: Users get to know which instances are set up and theirs content. The specific instance shows the collections stored and list the collection's ID with the respective href and root of the collection.
  • Retrieval:
  • Comments: the instances are created with the name given.

Collections Endpoint

  • Structure : /api/<version>/stac/collections or /api/<version>/stac/collections/<stac_collection_id>
  • Definition: Users can get the list of all collections as well as access the content of specific collection using the right ID structure. When the user access a specific Collection, the JSON of the collection is display as the same as it is established in the root URL.
  • Retrieval:
  • Comments: The ID of the collections is change on the fly and adjusted to Actinia naming structure

POST Method

Instances Endpoint

  • Structure : /api/<version>/stac/instances
  • Body:
{
    "stac_instance_id":"STAC_Others_2"
}
  • Definition: Users are allowed to add new instances into their storage capacity. This stage of management allow the user to organize the collections by topics, thematic or any level of organization they wish.
  • Comments: For this method the only parameter required is the stac_instance_id, This parameter creates an empty space in the REDIS-DB with such ID, on this instance the different collections will be stored.

Collections Endpoint

  • Structure : /api/<version>/stac/collections
  • Body:
{
    "stac_instance_id":"STAC_Others_2",
    "stac_url": "https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a"
}
  • Definition: Users are allowed to add new Collections into one instance. This stage of management allow the user to add the collections to the instance thought the shown body in the example.
  • Retrieval:
  • Comments: The ID in the STAC Collection (JSON Document) cached is modified and assigned to a new internal structured designed for Actinia (stac.<stac_instance_id>.rastercube.<stac_collection_id>)

DELETE Method

Instances Endpoint

  • Structure : /api/<version>/stac/instances/<stac_instance_id>
  • Definition: The users are allowed to delete instances into their storage capacity. the action implies the elimination of all the collections stored inside the instance.
  • Retrieval:
  • Comments:

Collections Endpoint

  • Structure : /api/<version>/stac/collections/<stac_collection_id>
  • Definition: Users are allowed to delete a collection. The method does not use body to pass the collection id, instead the collection id is given in the URL.
  • Retrieval:
  • Comments: To use this function, the user should specify the delete method and pass the URL

STAC IMPORTER

file:src/actinia_core/core/stac_exporter_interface.py

The STAC importer functionality in Actinia CORE works as a bridge between STAC-plugin and Actinia Core. This bridge allow the user to create process chain taking advantage of the STAC collections stored in the instances.

Based on the STAC specifications, the POST search endpoint from the root of the TAC collection is used to fetch all the items required. The capabilities of this search engine allow the users to provide spatio-temporal and semantic labelling filtering, enhancing the performance of the process by reducing the items analyzed.

Additionally, the process chain (body) defined for this endpoint, provide a filter parameter. That must follow the STAC specification and will be added to the search query.\

In order to translate the STAC format into a GRASS format, the Importer follows the next procedure:

    • Create an empty STRDS (SpatioTemporal Raster Datasets) database
    • Import using r.import (GRASS function) all the datasets from the items (Based on the filtering options provided).
    • Register the semantic label provided by asset in the STAC ITEM
    • Register the dataset into the STDR - Associating the image date

PC Example:

{
    "list": [{
        "id": "importer_1",
        "module": "importer",
        "inputs": [{
            "import_descr": {
                "source": "stac.STAC_Others_2.rastercube.sentinel-s2-l2a",
                "type": "stac",
                "semantic_label": [],
                "extent": {
                    "spatial": {
                        "bbox": [[30.1918571065124,-16.3692458600817,4834039161154,-0.2636709443366629]]
                    },
                    "temporal":{
                        "interval": [["2020-02-09", "2020-09-12"]]
                    }
                },
                "filter": {}
            },
            "param": "map",
            "value": "sentinel-s2-l2a"
        }
        ]
    }],
    "version": 1
}

STAC EXPORTER

file:src/actinia_core/core/stac_exporter_interface.py

The STAC exporter functionality in Actinia Core contributes to the formats of exportation. Once the metadata STAC option is defined in the process chain, the process is initialized by creating a result STAC catalog (just the first time) in the user cache. The Catalog will contain all the STAC items created.

The STAC item is crated and fed with all the metadata from the raster file (only format supported at the moment). As a noteworthy status, all the exported sources are stored in a configurable folder. The process creates a temporal file as the export function is defined as an ephemeral process\

The export function makes a whole analysis of the resource by extracting diverse elements of metadata such as projections, BBOX, geometry, datetime, epsg, gds, transform parameters, number of no data, spatial resolution, data type. In addition, the process add more information such as the software, the level of processing, the facility and STAC extension used for the Item construction.

PC Example:

{
    "list":[
        {
            "module": "r.slope.aspect",
            "id": "r_slope_aspect_1",
            "inputs": [
                {
                    "param": "elevation",
                    "value": "elevation@PERMANENT"
                },
                {
                    "param": "format",
                    "value": "degrees"
                },
                {
                    "param": "min_slope",
                    "value": "0.0"
                }
            ],
            "outputs": [
                {
                    "param": "slope",
                    "value": "elevation",
                    "export": {
                        "format": "GTiff",
                        "type": "raster"
                    },
                    "metadata":{
                        "format": "STAC"
                    }
                }
            ],
            "flags": "a"
        }
    ],
    "version": "1"
}

STAC IN OpenEO

modified file:src/openeo_grass_gis_driver/actinia_processing/load_collection_process.py, src/openeo_grass_gis_driver/actinia_processing/base.py

Once the development part in Actinia was concluded, subsequently action was integrating the capability inside OpenEO-grass-driver. In order to fulfill the requirement, the process of development implied finding some ways around for STAC item data.\

The current logic establish in OpenEO-grass-driver is based on the management of GRASS data types. Therefore, the reading and importing of the STAC items (JSON files not part of the GRASS Datatype) inside the openEO processes, required to the registration of STAC Items into STRDS. Once the process was defined inside the load_collection_process.py file, the process was able to run.\

It is noteworthy to mention that as the STAC naming structure did not follow the same logic as the GRASS dataset naming, inside the base.py it was required to insert STAC data type as an option for GRASS datatype and settle as fixed location for it.

Comment for Improvements

POST Method

  • When a new collection is injected to the instance, it would be useful to validate the links of the ITEMS, thereupon the item fetching will be somehow warranted

  • Support relative endpoints as well as manage .json ending endpoints

STAC Importer

  • Reduce the complexity of the workflow - currently it applies 4 steps to create the STRDS database
  • When the link of a STAC ITEM downgrade is, would be great to check in the root and update the collection to make some validation to fetch the element.

STAC Exporter

  • Modify the endpoints for catalog and items following the Actinia structure, deleting the catalog.json and <item_id>.json url
Clone this wiki locally