Merge branch '1.2-dev' into 1695-classification-codelist-tidy

open-contracting · Jun 28, 2024 · a3ff238 · a3ff238
2 parents 7245fba + 9be1ae0
commit a3ff238
Show file tree

Hide file tree

Showing 13 changed files with 7,850 additions and 859 deletions.
diff --git a/.github/workflows/spellcheck.yml b/.github/workflows/spellcheck.yml
@@ -10,4 +10,4 @@ jobs:
         with:
           python-version: '3.10'
       - run: pip install codespell
-      - run: codespell -S .git,docson,locale,examples,country.csv,currency.csv,language.csv,mediaType.csv -L fo,zar .
+      - run: codespell -S .git,docson,locale,examples,country.csv,currency.csv,language.csv,mediaType.csv -L fo,sme,zar,SME .
diff --git a/docs/examples/release_schema_reference/release_package.json b/docs/examples/release_schema_reference/release_package.json
@@ -94,6 +94,47 @@
           "projectID": "SP001",
           "uri": "https://openspending.org/uk-barnet-budget/entries/6801ad388f3a38b7740dde20108c58b35984ee91"
         },
+        "project": {
+          "id": "oc4ids-bu3kcz-0000",
+          "title": "Central Junction Cycle Lane Construction",
+          "description": "Design, construction and supervision of a new cycle lane for the Central Junction.",
+          "totalValue": {
+            "amount": 1100000,
+            "currency": "GBP"
+          },
+          "uri": "http://example.com/projects/oc4ids-bu3kcz-0000.json",
+          "sector": {
+            "id": "transport.road",
+            "description": "Road transport, including roads, highways, streets, tunnels and bridges.",
+            "scheme": "oc4idsProjectSector"
+          },
+          "additionalClassifications": [
+            {
+              "id": "03.04.05",
+              "description": "Cycle lanes for road transport.",
+              "scheme": "GB-EXAMPLE"
+            }
+          ],
+          "locations": [
+            {
+              "description": "Barnet, London, UK",
+              "geometry": {
+                "type": "Point",
+                "coordinates": [
+                  51.605,
+                  0.2076
+                ]
+              },
+              "gazetteer": {
+                "scheme": "GEONAMES",
+                "identifier": [
+                  "2656295"
+                ]
+              },
+              "uri": "https://www.geonames.org/2656295/barnet.html"
+            }
+          ]
+        },
         "documents": [
           {
             "datePublished": "2009-01-05T00:00:00Z",

diff --git a/docs/guidance/map/linked_standards.md b/docs/guidance/map/linked_standards.md
@@ -8,7 +8,7 @@ The [Open Contracting for Infrastructure Data Standards](https://standard.open-c
 
 A joint effort by the [Infrastructure Transparency Initiative](https://infrastructuretransparency.org) (CoST) and the Open Contracting Partnership, it builds on best practice in open data and openness of public infrastructure procurement globally.
 
-The [Project](https://extensions.open-contracting.org/en/extensions/project/master/) extension describes how to connect OCDS and OC4IDS datasets.
+OC4IDS datasets ought to be linked from OCDS datasets using the `planning.project` object, by referencing the OC4IDS project identifier in the `planning.project.id` field.
 
 ## Open Fiscal Data Package
 

diff --git a/docs/guidance/map/translations.md b/docs/guidance/map/translations.md
@@ -14,12 +14,12 @@ You can publish the values of these fields in any language:
 - `description`, in any location
 - `finalStatusDetails`, in any location
 - `legalName`, in any location
+- `locality`, in any location
 - `name`, in any location
 - `rationale`, in any location
+- `region`, in any location
+- `streetAddress`, in any location
 - `title`, in any location
-- `parties/address/locality`
-- `parties/address/region`
-- `parties/address/streetAddress`
 - `planning/budget/project`
 - `tender/awardCriteriaDetails`
 - `tender/procurementMethodDetails`

diff --git a/docs/history/changelog.md b/docs/history/changelog.md
@@ -72,8 +72,10 @@ Per the [normative and non-normative content and changes policy](../governance/n
 
 * Add new codelists:
   * [#1372](https://github.com/open-contracting/standard/pull/1372) `country`
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) `geometryType`
   * [#1316](https://github.com/open-contracting/standard/pull/1316) `language`
   * [#1326](https://github.com/open-contracting/standard/pull/1326) `linkRelationType`
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) `locationGazetteer`
   * [#1308](https://github.com/open-contracting/standard/pull/1308) `mediaType`
   * [#1459](https://github.com/open-contracting/standard/pull/1459) `partyScale`
   * [#1654](https://github.com/open-contracting/standard/pull/1654) `permission`
@@ -96,6 +98,7 @@ Per the [normative and non-normative content and changes policy](../governance/n
     * 'eu-buyer-legal-type'
   * [#1452](https://github.com/open-contracting/standard/pull/1452) Add 'proClass'.
   * [#1637](https://github.com/open-contracting/standard/pull/1637) Add 'sdg' and 'sdgTarget'.
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) Add 'oc4idsProjectSector' from project extension.
 
 * `documentType.csv`:
   * Add codes:
@@ -183,9 +186,10 @@ Per the [normative and non-normative content and changes policy](../governance/n
 
 * Clarify core concepts:
   * [#1216](https://github.com/open-contracting/standard/pull/1216) Define contracting process and planning process in the schema description. Update definition of release, record and ocid. Update references to contracting process so that it takes take the planning process into account.
-  * [#1513](https://github.com/open-contracting/standard/pull/1513) Update fields' use of "contracting process" and "contracting (or planning) process" to reflect the new definition. Align `budget.project` and `budget.projectID`.
+  * [#1513](https://github.com/open-contracting/standard/pull/1513) Update fields' use of "contracting process" and "contracting (or planning) process" to reflect the new definition.
   * [#1443](https://github.com/open-contracting/standard/pull/1443) `date`
   * [#1182](https://github.com/open-contracting/standard/pull/1182) `buyer`
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) `budget.project` and `budget.projectID`
   * [#1163](https://github.com/open-contracting/standard/pull/1163) `tender.procuringEntity`
   * [#1395](https://github.com/open-contracting/standard/pull/1417) `tender.procurementMethod`
   * [#1442](https://github.com/open-contracting/standard/pull/1442) `tender.submissionMethodDetails`
@@ -198,7 +202,9 @@ Per the [normative and non-normative content and changes policy](../governance/n
 * Add new fields:
   * [#1335](https://github.com/open-contracting/standard/pull/1335) `planning.id`
   * [#1642](https://github.com/open-contracting/standard/pull/1642) `planning.finalStatus`, `planning.finalStatusDetails` and `planning.finalStatusDate`
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) `planning.project`
   * [#1324](https://github.com/open-contracting/standard/pull/1324) `tender.datePublished`
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) `tender.deliveryLocations` and `tender.deliveryAddresses`
   * [#1296](https://github.com/open-contracting/standard/pull/1296) [#1674](https://github.com/open-contracting/standard/pull/1674) `tender.exclusionGrounds`
   * [#1669](https://github.com/open-contracting/standard/pull/1669) `tender.expressionOfInterestDeadline`
   * [#1490](https://github.com/open-contracting/standard/pull/1490) `tender.identifiers`
@@ -219,6 +225,7 @@ Per the [normative and non-normative content and changes policy](../governance/n
   * [#1372](https://github.com/open-contracting/standard/pull/1372) `Address.country`
   * [#1380](https://github.com/open-contracting/standard/pull/1380) [#1440](https://github.com/open-contracting/standard/pull/1440) `Document.languages`
   * [#1439](https://github.com/open-contracting/standard/pull/1439) `Document.relatedItems`
+  * [#1684](https://github.com/open-contracting/standard/pull/1684) `Item.deliveryLocations` and `Item.deliveryAddresses`
   * [#1455](https://github.com/open-contracting/standard/pull/1455) `Milestone.dueAfterDate`
   * [#1434](https://github.com/open-contracting/standard/pull/1434) `Milestone.value`
   * [#1490](https://github.com/open-contracting/standard/pull/1490) `SimpleIdentifier`

diff --git a/docs/schema/reference.md b/docs/schema/reference.md
@@ -135,7 +135,7 @@ The planning section is used in a planning process. This includes information ab
 
 ```{jsonschema} ../../build/current_lang/release-schema.json
 :pointer: /definitions/Planning
-:collapse: budget,documents,milestones
+:collapse: budget,project,documents,milestones
 ```
 
 ```{extensionlist} The following extensions are available for planning
@@ -168,6 +168,26 @@ Apart from documents, the majority of planning information is held within the bu
 :list: budget
 ```
 
+#### Project
+
+````{admonition} Example
+:class: hint
+
+```{jsoninclude} ../examples/release_schema_reference/release_package.json
+:jsonpointer: /releases/0/planning/project
+:title: project
+```
+````
+
+```{jsonschema} ../../build/current_lang/release-schema.json
+:pointer: /definitions/Project
+:collapse: totalValue,additionalClassifications,locations
+```
+
+```{extensionlist} The following extensions are available for project
+:list: 
+```
+
 ### Tender
 
 The tender section includes details of the announcement that an organization intends to source some particular goods, services or works and to establish one or more contract(s) for these.
@@ -671,7 +691,18 @@ As well as providing this machine-readable link between processes, publishers ma
 
 ### Location
 
-The [Location](https://extensions.open-contracting.org/en/extensions/location/v1.1.4/) extension can be used to provide location information.
+````{admonition} Example
+:class: hint
+
+```{jsoninclude} ../examples/release_schema_reference/release_package.json
+:jsonpointer: /releases/0/planning/project/locations/0
+:title: location
+```
+````
+
+```{jsonschema} ../../build/current_lang/release-schema.json
+:pointer: /definitions/Location
+```
 
 ### Link
 

diff --git a/manage.py b/manage.py
@@ -519,7 +519,7 @@ def pre_commit():
     """
     nonmultilingual = {
         # Identifiers.
-        'amendsReleaseID', 'id', 'identifier', 'ocid', 'relatedItems', 'releaseID',
+        'amendsReleaseID', 'id', 'identifier', 'identifiers', 'ocid', 'relatedItems', 'releaseID',
         # Missing format properties. https://github.com/open-contracting/standard/issues/881
         'email',
         # Published-defined formats.
@@ -532,14 +532,16 @@ def pre_commit():
     jsonref_release_schema = json_load('release-schema.json', jsonref, merge_props=True)
 
     counts = defaultdict(list)
+    nonstring = ('boolean', 'integer', 'number', 'object')
     for field in get_schema_fields(jsonref_release_schema):
         name = field.path_components[-1]
         # Skip definitions (output dereferenced properties only). Skip deprecated fields.
         if field.definition_pointer_components or field.deprecated:
             continue
         multilingual = (
             # If a field can be a non-string, it is not multilingual.
-            not any(t in field.schema['type'] for t in ('boolean', 'integer', 'number', 'object'))
+            not any(t in field.schema['type'] for t in nonstring)
+            and ('array' not in field.schema['type'] or not any(t in field.schema['items']['type'] for t in nonstring))
             # If a field's value is constrained to a codelist or format, it is not multilingual.
             and not any(prop in field.schema for prop in ('codelist', 'format'))
             # If an array can contain non-strings, it is not multilingual.
@@ -549,14 +551,16 @@ def pre_commit():
         )
         field.sep = '/'
         if name in counts and bool(counts[name]) ^ multilingual:
-            if multilingual:
+            if not multilingual and field.schema['type'] == 'object':
+                click.secho(f'{field.path} is an object. {" & ".join(counts[name])} is/are multilingual.', fg='yellow')
+            elif multilingual:
                 raise Exception(f'{name} is multilingual at {field.path}, but not elsewhere')
             else:
                 raise Exception(f'{name} is multilingual at {" & ".join(counts[name])}, but not at {field.path}')
         if multilingual:
             counts[name].append(field.path)
         else:
-            counts[name] = []
+            counts[name] = counts[name]
 
     bulletlist = [
         '% STARTLIST',

diff --git a/schema/codelists/classificationScheme.csv b/schema/codelists/classificationScheme.csv
@@ -21,3 +21,4 @@ TED_CE_ACTIVITY,EU Main activity of contracting entity (TED schema),The main act
 eu-buyer-contracting-type,EU Buyer contracting entity type (eForms),Whether or not the buyer is a contracting entity within the domain of public procurement according to the legislation of the European Parliament.,https://docs.ted.europa.eu/eforms/latest/reference/code-lists/buyer-contracting-type.html,organization
 eu-main-activity,EU Main activity authority list (eForms),The main activity of the contracting entity within the domain of public procurement according to the legislation of the European Parliament. This scheme's codes match those in eForms.,https://op.europa.eu/en/web/eu-vocabularies/concept-scheme/-/resource?uri=http://publications.europa.eu/resource/authority/main-activity,organization
 eu-buyer-legal-type,EU Buyer legal type (eForms),The type of the procuring authority within the domain of public procurement according to the legislation of the European Parliament. This scheme's codes match those in eForms.,https://op.europa.eu/en/web/eu-vocabularies/concept-scheme/-/resource?uri=http://publications.europa.eu/resource/authority/buyer-legal-type,organization
+oc4idsProjectSector,OC4IDS Project Sector,Open Contracting for Infrastructure Data Standards Toolkit projectSector codelist,https://standard.open-contracting.org/infrastructure/latest/en/reference/codelists/#projectsector,project
diff --git a/schema/codelists/geometryType.csv b/schema/codelists/geometryType.csv
@@ -0,0 +1,7 @@
+Code,Title,Description,Source
+Point,Point,The `coordinates` field is a single position.,https://tools.ietf.org/html/rfc7946#section-3.1
+MultiPoint,MultiPoint,The `coordinates` field is an array of positions.,https://tools.ietf.org/html/rfc7946#section-3.1
+LineString,LineString,The `coordinates` field is an array of two or more positions.,https://tools.ietf.org/html/rfc7946#section-3.1
+MultiLineString,MultiLineString,The `coordinates` field is an array of LineString coordinate arrays.,https://tools.ietf.org/html/rfc7946#section-3.1
+Polygon,Polygon,The `coordinates` field is an array of linear ring coordinate arrays.,https://tools.ietf.org/html/rfc7946#section-3.1
+MultiPolygon,MultiPolygon,The `coordinates` field is an array of Polygon coordinate arrays.,https://tools.ietf.org/html/rfc7946#section-3.1
diff --git a/schema/codelists/locationGazetteer.csv b/schema/codelists/locationGazetteer.csv
@@ -0,0 +1,7 @@
+Category,Code,Title,Description,Source,URI Pattern
+Sub-National,NUTS,EU Nomenclature of Territorial Units for Statistics,The Nomenclature of Territorial Units for Statistics (NUTS) was established by Eurostat in order to provide a single uniform breakdown of territorial units for the production of regional statistics for the European Union.,https://ec.europa.eu/eurostat/web/nuts/linked-open-data,http://data.europa.eu/nuts/code/
+National,ISO2,ISO 3166-1 alpha-2,ISO 3166-1 alpha-2 assigns two-letter codes to countries.,https://www.iso.org/iso-3166-country-codes.html,
+Universal,GEONAMES,GeoNames,GeoNames assigns numerical identifiers to geographical names.,https://www.geonames.org/,https://www.geonames.org/
+Universal,OSMN,OpenStreetMap node,An OpenStreetMap node defines a point in space.,,https://www.openstreetmap.org/node/
+Universal,OSMW,OpenStreetMap way,"An OpenStreetMap way defines linear features and area boundaries, as an ordered list of nodes.",https://wiki.openstreetmap.org/wiki/Way,https://www.openstreetmap.org/way/
+Universal,OSMR,OpenStreetMap relation,"An OpenStreetMap relation combines nodes, ways, and/or other relations.",https://wiki.openstreetmap.org/wiki/Relation,https://www.openstreetmap.org/relation/