Geonode notes

Integrating pycsw and GeoNode

Features of CSW used by GeoNode

Creating and updating CSW records.
Searching in free form, a set of words in any field.
Searching by bounding box.
Support for TC/211 metadata format.

pycsw notes

(including review of https://github.com/GFDRR/geonode/commit/4e8bc00202e5fd4dcc427156564ee46da87a9588)

http://pycsw.org/
supports transactions (Insert/Update/Delete) and Harvest
must be enabled; (list of) IP restricted access
supports federated search
supports ISO (must be enabled)
support INSPIRE (must be enabled)
passes 103/103 OGC CITE tests
search support
full text (csw:AnyText)
spatial and aspatial (http://pycsw.org/docs/introduction.html#supported-filters)
repository
supports sqlite3, mysql and postgresql
postgresql setup requires setting up plpythonu functions (https://sourceforge.net/apps/trac/pycsw/browser/trunk/sbin/setup_db.py#L87)
data model
data model loosely based on ISO 19115, full model can be seen at https://sourceforge.net/apps/trac/pycsw/browser/trunk/sbin/setup_db.py#L75
django integration: this would need to investigated/implemented. pycsw repository is self contained, gets setup with https://sourceforge.net/apps/trac/pycsw/browser/trunk/sbin/setup_db.py. Underlying transaction operations via sqlalchemy can be reviewed at https://sourceforge.net/apps/trac/pycsw/browser/trunk/server/repository.py
privileges/groups/roles
pycsw has no implementation of this. CSW, by nature, leaves this to the implementation. By default, everything published to pycsw is published to all
UUIDs are required for Transaction Insert'ed records. We could extend by assigning a server generated UUID

The most pertinent change would be django integration. This would affect how the repository is setup and how it is interacted with. Options:

create a django based pycsw plugin
have a configuration option in pycsw to signal django based interaction
update CRUD functions to be django based if requested (repository_model=internal|django|geonode)
if most of the interaction is CSW request based, then just create db in django-site/settings.py, and ref db connection from pycsw configuration. pycsw would still use sqlalchemy as the ORM/API. Abstraction would be required between data model specifics (column names).

Metadata Links

Background

The OGC CSW 2.0.2 standard provides a discovery mechanism of geospatial metadata. For the HTTP protocol binding, the baseline information model is the Dublin Core (DC) Metadata Element Set. Additional profiles (such as APISO 1.0) exist to support specific information models (i.e. ISO 19139:2007) and output schemas.

Both ISO and DC allow for the advertisement of HTTP links. ISO's CI_OnlineResource construct allows for a certain level of granularity to express links (linkage, protocol, applicationProfile, name, description, function). Dublin Core allows for dct:references [0..*], which has a scheme attribute. There is no further granularity within dct:references.

Implementations have used dc:URI, however dc:URI is a non-standard element of DC record model in CSW 2.0.2

A finer model is required to communicate different types of links (opaque, binding parameters, etc.).

Motivation

Geonode, as a CSW client, requires the ability to identify various online linkages and be aware of their types/model to be able to process them accordingly (i.e. opaque, binding parameters, etc.).

This is possible given the ISO model, however it would be valuable to have an lightweight approach within DC that is inspired by ISO's CI_OnlineResource as well as fit within DC's dct:references element.

Proposal

Express ISO's CI_OnlineResource model in JSON using in dct:references/@scheme.

Example: <dct:references scheme="{'protocol': 'OGC:WMS-1.1.1-http-get-map', 'name': 'base%3Adistrits', 'description': 'distrits (JPEG Format)', 'linkage': 'http://localhost:8001/geoserver/wms'}">http://localhost:8001/geoserver/wms?layers=base%3Adistrits&width=620&bbox=-86.144%2C8.041%2C-82.555%2C11.221&service=WMS&format=image%2Fjpeg&srs=EPSG%3A4326&request=GetMap&height=550</dct:references>

Where:

dct:references.text: the full URL to the resource
dct:references/@scheme: a JSON representation of ISO's CI_OnlineResource:
name: the name of the link. In the case of an OWS, this should be the resource (layer|featureType|coverage) name
description: a human-readable description.
protocol: the type assigned to the URL, from the GeoNetwork gmd:online/gmd:CI_OnlineResource/gmd:protocol enumeration *
linkage: the URL of the resource. In the case of an OWS, this should be the service endpoint with query parameters
protocol enumeration and description

protocol	description
ESRI:AIMS--http--configuration	ArcIMS Map Service Configuration File (*.AXL)
ESRI:AIMS--http-get-feature	ArcIMS Internet Feature Map Service
ESRI:AIMS--http-get-image	ArcIMS Internet Image Map Service
GLG:KML-2.0-http-get-map	Google Earth KML service (ver 2.0)
OGC:CSW	OGC-CSW Catalogue Service for the Web
OGC:KML	OGC-KML Keyhole Markup Language
OGC:GML	OGC-GML Geography Markup Language
OGC:ODS	OGC-ODS OpenLS Directory Service
OGC:OGS	OGC-ODS OpenLS Gateway Service
OGC:OUS	OGC-ODS OpenLS Utility Service
OGC:OPS	OGC-ODS OpenLS Presentation Service
OGC:ORS	OGC-ODS OpenLS Route Service
OGC:SOS	OGC-SOS Sensor Observation Service
OGC:SPS	OGC-SPS Sensor Planning Service
OGC:SAS	OGC-SAS Sensor Alert Service
OGC:WCS	OGC-WCS Web Coverage Service
OGC:WCS-1.1.0-http-get-capabilities	OGC-WCS Web Coverage Service (ver 1.1.0)
OGC:WCTS	OGC-WCTS Web Coordinate Transformation Service
OGC:WFS	OGC-WFS Web Feature Service
OGC:WFS-1.0.0-http-get-capabilities	OGC-WFS Web Feature Service (ver 1.0.0)
OGC:WFS-G	OGC-WFS-G Gazzetteer Service
OGC:WMC-1.1.0-http-get-capabilities	OGC-WMC Web Map Context (ver 1.1)
OGC:WMS	OGC-WMS Web Map Service
OGC:WMS-1.1.1-http-get-capabilities	OGC-WMS Capabilities service (ver 1.1.1)
OGC:WMS-1.3.0-http-get-capabilities	OGC-WMS Capabilities service (ver 1.3.0)
OGC:WMS-1.1.1-http-get-map	OGC Web Map Service (ver 1.1.1)
OGC:WMS-1.3.0-http-get-map	OGC Web Map Service (ver 1.3.0)
OGC:SOS-1.0.0-http-get-observation	OGC-SOS Get Observation (ver 1.0.0)
OGC:SOS-1.0.0-http-post-observation	OGC-SOS Get Observation (POST) (ver 1.0.0)
OGC:WNS	OGC-WNS Web Notification Service
OGC:WPS	OGC-WPS Web Processing Service
WWW:DOWNLOAD-1.0-ftp--download	File for download through FTP
WWW:DOWNLOAD-1.0-http--download	File for download
FILE:GEO	GIS file
FILE:RASTER	GIS RASTER file
WWW:LINK-1.0-http--ical	iCalendar (URL)
WWW:LINK-1.0-http--link	Web address (URL)
WWW:LINK-1.0-http--partners	Partner web address (URL)
WWW:LINK-1.0-http--related	Related link (URL)
WWW:LINK-1.0-http--rss	RSS News feed (URL)
WWW:LINK-1.0-http--samples	Showcase product (URL)
DB:POSTGIS	PostGIS database table
DB:ORACLE	ORACLE database table
WWW:LINK-1.0-http--opendap	OPeNDAP URL
RBNB:DATATURBINE	Data Turbine
UKST	Unknown Service Type

Extensions to protocol enumerations and descriptions

protocol	description
WWW:LINK-1.0-http--image-thumbnail	Web image thumbnail (URL)

Examples

WMS Layer with only binding information <dct:references scheme="{'protocol': 'OGC:WMS-1.1.1-http-get-map', 'name': 'base%3Adistrits', 'description': 'distrits (JPEG Format)', 'linkage': 'http://localhost:8001/geoserver/wms'}">http://localhost:8001/geoserver/wms?layers=base%3Adistrits&width=620&bbox=-86.144%2C8.041%2C-82.555%2C11.221&service=WMS&format=image%2Fjpeg&srs=EPSG%3A4326&request=GetMap&height=550</dct:references>

Thumbnail <dct:references scheme="{'protocol': 'WWW:LINK-1.0-http--image-thumbnail', 'name': 'base%3Adistrits', 'description': 'Web image thumbnail (URL)', 'linkage': 'http://localhost:8001/geoserver/wms'}">http://localhost:8001/geoserver/wms?layers=base%3Adistrits&width=20&bbox=-86.144%2C8.041%2C-82.555%2C11.221&service=WMS&format=image%2Fjpeg&srs=EPSG%3A4326&request=GetMap&height=20</dct:references>

https://pycsw.org