This repository contains the functionality to process occurrence data and create aggregated occurrence cubes. An occurrence cube is a multi-dimensional array of values. In our context we have three dimensions, N = 3:

1. taxonomic (taxon)
2. temporal (year)
3. spatial (cell code)

For each triplet the stored value represent the number of occurrences found in GBIF.

As the occurrence cubes are used as input for modelling and risk assessment, we store the smallest geographic coordinate uncertainty of the occurrences assigned to a certain cell code as value as well. The occurrences are first reassigned randomly within their uncertainty circle before assigning them to a cell. If uncertainty is not available, a default 1000m radius is assigned. Due to the random assignment, the same occurrence data could result in different occurrence cubes if iterated.

Using a tabular structure (typical of R data.frames), a cube would look like this:

where number of columns is equal to the number of dimensions, N, plus number of values. In our case we have three dimensions and two values, so five columns in total.

One of the main output of TrIAS project is delivering a global, unified checklist of alien species in Belgium. This checklist is published on GBIF as Global Register of Introduced and Invasive Species - Belgium. More information can be found here. In this repository we produce an occurrence cube of all taxa of the unified checklist that have at least one occurrence in Belgium. As we need to make an occurrence cube at class level (baseline), we harvested ALL occurrences in Belgium. This first step is documented in /src/belgium/1_download.Rmd. At the moment of writing, the GBIF download counts around 35 million occurrences. In order to handling such huge amount of data, we decided to work using a SQLite file, see second step: /src/belgium/2_create_db.Rmd. Handling occurrence geographic uncertainty and assigning the correspondent cell codes to occurrences is the third step, documented in /src/belgium/3_assign_grid.Rmd. Finally, we select the alien species belonging to the Global Register of Introduced and Invasive Species - Belgium and aggregate the occurrences by year, cell code and taxon as described in the fourth and last pipeline: /src/belgium/4_aggregate.Rmd. The resulting occurrence cube is saved in file /data/processed/be_alientaxa_cube.tsv. We also produce an occurrence cube at class level, /data/processed/be_classes_cube.tsv which can be used as baseline to tackle the research bias effort while calculating occurrence-based indicators.

The TrIAS project aims to assess risk of invasion by applying distribution modelling and other modelling techniques for a subset of taxa of the unified checklist. The list is saved in file /data/reference/modelling_species.tsv. For these species we build a specific occurrence cube which takes into account all occurrences in Europe. The region we define as Europe is described by the European Environmental Agency, see image here. Similarly to the Belgian cube, we first download the occurrences of these species following the workflow described in src/europe/1_download.Rmd. Then, we assign the occurrences randomly within their uncertainty circles in order to calculate the 1kmx1km cell they belong to, see /src/europe/2_assign_grid.Rmd and finally we aggregate as described in /src/europe/3_aggregate.Rmd in order to produce the final occurrence cube at European level, eu_modellingtaxa_cube.tsv, saved in data/processed.

If a taxon has taxonomic status ACCEPTED or DOUBTFUL, i.e. it's not a synonym, then GBIF returns not only the occurrences linked directly to it, but also the occurrences linked to its synonyms and its infraspecific taxa.

As example, consider the species Reynoutria japonica Houtt.`. If you search for its occurrrences wordwide you will get all the occurrences from the synonyms and infraspecies too.

See https://doi.org/10.15468/dl.rej1cz for more details. Note: the table above is just an example and can be outdated.

By aggregating we would loose this information, so we provide aside the cubes, be_alientaxa_cube.tsv and eu_modellingtaxa_cube.tsv, a kind of taxonomic compendium, be_alientaxa_info.tsv and eu_modellingtaxa_info.tsv respectively. They include for each taxa in the cube all the synonyms or infraspecies whose occurrences contribute to the total count. They are both saved in data/processed.

For example, Aedes japonicus (Theobald, 1901) is an accepted species present in the belgian cube: based on the information stored in be_alientaxa_info.tsv, its occurrences include occurrences linked to the following taxa:

1. Aedes japonicus (Theobald, 1901)
2. Ochlerotatus japonicus (Theobald, 1901)
3. Aedes japonicus subsp. japonicus
4. BOLD:AAC5210

The repository structure is based on Cookiecutter Data Science. Files and directories indicated with GENERATED should not be edited manually.

├── README.md            : Description of this repository
├── LICENSE              : Repository license
├── occ-processing.Rproj : RStudio project file
├── .gitignore           : Files and directories to be ignored by git
│
├── references
│   ├── Europe.png       : Map of Europe
│   ├── modelling_species.tsv: List of species whos occurrences are queried from GBIF at European level
│
├── data
│   ├── raw              : Occurrence data as downloaded from GBIF GENERATED
│   ├── interim          : big sqlite and text files, stored locally  GENERATED
│   └── processed        : occurrence data cubes and related taxa informations GENERATED
│
├── docs                 : Repository website (not implemented yet) GENERATED
│
└── src
    ├── belgium
        ├── 1_download.Rmd    : Script to trigger a download of occurrences in Belgium
        ├── 2_create_db.Rmd   : Script to genereate a sqlite file and perform basic filtering
        ├── 3_assign_grid.Rmd : Script to assign cell code to occurrences
        ├── 4_aggregate.Rmd   : Script to aggregate data and make the Belgian data cube
    ├── europe
        ├── 1_download.Rmd    : Script to trigger a download of occurrences in Belgium
        ├── 2_assign_grid.Rmd : Script to perform basic filtering and assign cell code to occurrences
        ├── 3_aggregate.Rmd   : Script to aggregate data and make the modelling data cube at European level  

Clone this repository to your computer and open the RStudio project file, occ-processing.Rproj.

You can generate the Belgian occurrence data cube by running the R Markdown files in src/belgium following the order shown here below:

1. 1_download.Rmd: trigger a GBIF download and add it to the list of triggered downloads
2. 2_create_db.Rmd: create a sqlite database and perform basic data cleaning
3. 3_assign_grid.Rmd: assign geographic cell code to occurrence data
4. 4_aggregate.Rmd: aggregate occurrences per taxon, year and cell code, the Belgian occurrence data cube

In the aggregation step, we also create a data cube at class level. The data cubes are authomatically generated in folder /data/processed/.

At European level we are interested in occurrences of a list of taxa, which will be used for modelling and risk assessment. This list is maintained in file modelling_species.tsv in folder references.

You can generate the European occurrence data cube by running the R Markdown files in src/europe following the order shown here below:

1. 1_download.Rmd: trigger a GBIF download and adding it to the list of triggered downloads
2. 2_assign_grid.Rmd: assign geographic cell code to occurrence data
3. 3_aggregate.Rmd: aggregate occurrences per taxa, year and cell code, the European occurrence data cube.

Install any required packages.

List of contributors

MIT License for the code and documentation in this repository.