Init dataset/layer config for seal tag data pulled from ADC #808
There are no files selected for viewing
|
There was a problem hiding this comment. Here we define a new dataset for the layer we're adding. I manually pulled this information directly from the ADC dataset landing page. For our QGreenland-Net processing, this information can be extracted programmatically from the metadata. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| from qgreenland.models.config.asset import HttpAsset | ||
| from qgreenland.models.config.dataset import Dataset | ||
|
|
||
| dataset = Dataset( | ||
| id="seal_tag_measurements", | ||
| assets=[ | ||
| HttpAsset( | ||
|
There was a problem hiding this comment. This dataset has just a single file (asset) we're interested in pulling (the |
||
| id="only", | ||
| urls=[ | ||
| "https://arcticdata.io/metacat/d1/mn/v2/object/urn%3Auuid%3A31162eb9-7e3b-4b88-948f-f4c99f13a83f", | ||
| ], | ||
| ), | ||
| ], | ||
| metadata={ | ||
| "title": "Water temperature, salinity, and position-geographic data taken from seal tags, in coastal Greenland from 2010-08-31 to 2012-05-14", | ||
| "abstract": ( | ||
| """This project involves a marine-mammal sensor-tagging | ||
| approach that will allow for sustained oceanographic observations along | ||
| the periphery of the Greenland Ice Sheet. This effort is motivated by a | ||
| successful pilot project involving ringed seals in two Greenland fjords | ||
| this past summer. The pilot proved the viability of the technique, which | ||
| makes use of ringed seals who spend the majority of their time in such | ||
| fjords and who may be appropriately equipped with integrated, | ||
| location-tracking, CTD, and satellite communication instrumentation. The | ||
| observations, which will be collected in partnership with colleagues at | ||
| the Greenland Institute of Natural Resources, will be archived in | ||
| national data bases, on a project website, and made widely available to | ||
| others in near real-time. This project will address a currently missing, | ||
| critical component of the Arctic Observing Network, namely observations | ||
| of ocean temperatures and salinities along the periphery of the | ||
| Greenland Ice Sheet. The methodology will lead to a practical, | ||
| sustainable data stream for hydrographic properties in the notoriously | ||
| difficult to access regions of the Greenland inner fjords, thus further | ||
| filling out the overall Arctic Observing Network data portfolio. Such | ||
| observations are needed to develop improved physics that can | ||
| subsequently be directly used in IPCC class, coupled climate models. The | ||
| data collection will also be of immense value to researchers studying | ||
| the behavior of the ringed seals in Greenland. Virtually no behavioral | ||
| data exists for these seals in the ice-ocean fjords, and the project | ||
| data can be used to learn about the habitat and distribution of the | ||
| seals, and ultimately of their possible vulnerability to climate | ||
| change.""" | ||
| ), | ||
| "citation": { | ||
| "text": ( | ||
| """David Holland. (2018). Water temperature, salinity, and | ||
| position-geographic data taken from seal tags, in coastal Greenland | ||
| from 2010-08-31 to 2012-05-14. Arctic Data | ||
| Center. urn:uuid:1f3f702f-9594-4293-8cbd-07932e54e8ed.""" | ||
| ), | ||
| "url": "https://arcticdata.io/catalog/view/urn%3Auuid%3A1f3f702f-9594-4293-8cbd-07932e54e8ed", | ||
| }, | ||
| }, | ||
| ) | ||
|
There was a problem hiding this comment. Beginning of the layer configuration. Each subdirectory under |
|
There was a problem hiding this comment. Here is the configuration for the layer itself. This includes the layer title (what it'll be called in the QGIS layers panel), description, style (not currently defined here, but normally we add a style file for each dataset to make it look nice by default), the dataset/asset used as input to create the layer, and the steps required to make it QGreenland-ready. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| from qgreenland.config.datasets.seal_tag_water_measurements import dataset | ||
| from qgreenland.config.helpers.steps.ogr2ogr import ogr2ogr | ||
| from qgreenland.models.config.layer import Layer, LayerInput | ||
|
|
||
| layer = Layer( | ||
| id="seal_tag_measurements", | ||
| title="Seal tag water measurements", | ||
| description=( | ||
| """Locaitons of seal tag measurements with temperature and salinity in coastal Greenland from 2010-08-31 to 2012-05-14.""" | ||
| ), | ||
| tags=[], | ||
| in_package=True, | ||
| input=LayerInput( | ||
| dataset=dataset, | ||
| asset=dataset.assets["only"], | ||
| ), | ||
| steps=[ | ||
| *ogr2ogr( | ||
|
There was a problem hiding this comment. This helper function runs |
||
| input_file="{input_dir}/*.csv", | ||
| output_file="{output_dir}/final.gpkg", | ||
| ogr2ogr_args=( | ||
| "-s_srs", | ||
| "EPSG:4326", | ||
| "-oo", | ||
| "X_POSSIBLE_NAMES=Longitude", | ||
| "-oo", | ||
| "Y_POSSIBLE_NAMES=Latitude", | ||
| ), | ||
| ), | ||
| ], | ||
| ) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This lockfile provides a json representation of the config that's defined using Python. This allows us to quickly be able to assess the results of config changes against previous state.