Errors with scope code "dataset"

[dwalt]

Issue

mdEditor Version (find version under settings): V0.11.0-Dev 2

mdEditor Web Address (enter the URL): dev.mdeditor.org

Issue Description:

Scope code in Data Quality appears to be written incorrectly in mdJSON and misinterpreted in mdtranslator

Steps to reproduce (Bug reports only):

1. _____Chose data quality scope code from drop down list: "geographicDataset"
2. _____Return to data quality list shows scope code in ui report as: "dataset"
3. _____Preview of scope code in mdJSON: "dataset"
4. _____Translation to -1 and to -2 scope code: "series"
5. _____MD_ScopeCode shows code is "dataset" and is confirmed by ISO docs. However, definition is incorrect, still referencing geographic dataset.

Observed Results (Bug reports only):

Appears that geographicDataset is no longer a valid code. Appears that there is conversion handling to convert geographicDataset to dataset in mdEditor, however this same conversion is not being applied to -1 and -2 writers.

Translation to -1 or -2 is writing the wrong code.

Expected Results (Bug reports only):

mdcodes: dataset, Information applies to a dataset
mdEditor drop down list: dataset not geographicDataset
mdTranslator -1, -2: scope code of dataset

[dwalt]

@hmaier-fws @jwaspin Per conversation, "geographicDataset" was never a code. It has always been "dataset". ISO -2 code definition defined it as a geographic dataset. ISO -1 later removed the word "geographic" from the definition to read as a generic dataset. mdEditor has handling for displaying the "dataset" code as "geographicDataset" in the codelist due to a previous use case considering dataset to mean geographic dataset. This handling can be removed which will present the dataset code in a consistent manner as generic dataset. Geographic Dataset is not a current coding in Scope Code. No need for conversion handling in mdTranslator.

[hmaier-fws]

@dwalt I removed the bug label since this is actually functioning as designed. This would be a change to the existing behavior.

I also think this needs a discussion with the stakeholders to confirm that this should be the desired behavior. Without informing users of the change, they will likely incorrectly categorize their data. (that is the reason the display label of the "dataset" code was changed initially).

Some background for discussion:

• The current ISO codelist (also at standards.iso.org) contains both a "dataset" and a "nonGeographicDataset" code
• The official ISO definitions are as described above
• The underlying assumption for the change in the pick-list description was that if ISO defines "nonGeographicDataset" as "information applies to non-geographic data", then the "dataset" must apply to everything else (non non-geographic data = geographic data).
• That distinction was lost among users who are not aware of the ISO standards and they were selecting "dataset" for non-geographic data (not the nonGeographicDataset code).
• The change was originally made to support the fundamental goal of the mdEditor to "...promote the creation and use of metadata by lowering the level of technical expertise required to produce archival quality metadata"
• If the "dataset" code and be applied to both geographic and non-geographic (generic) data, then it's probably not much of an issue.
• But, if coding a generic (non-geographic) data set as "dataset" instead of "nonGeographicDataset" then that will most likely cause problems for the users as it has in the past.

[dwalt]

@hmaier-fws I agree this should be discussed by stakeholders. I would add that mdToolkit has otherwise used ISO codes verbatim and extended codelists as agreed upon by the community. This on-the-fly recasting of this code was likely done before profiles existed (are there any more recastings?). I would suggest that geographicDataset become an extended code, or part of a profile code "recasting" functionality. Of the solutions, I would favor implementation of filtered codelists in profiles, then use a combination of filtered codelists in profiles and extended codes to meet this need. In the short term, an extended code would meet the requirement without confusing new users with a recast code in the drop down list.

[hmaier-fws]

@dwalt , @jwaspin
I was reviewing PR #658 and noticed that making this change will cause some major problems for users:

• The above mentioned "profile code recasting" feature (Custom codelists in profiles #331) has not yet been implemented in profiles (at least not that I'm aware of).
• Making the change now would result in the following two entries in the code list, causing confusion among users as to the difference between a dataset and a nonGeographicDataset:
  • codeName: dataset, description: "information applies to the dataset"
  • codeName: nonGeographicDataset, description: "information applies to non-geographic data"
• Users would have no method to distinguish between a spatial and a non-spatial dataset:
  • This is used as a critical quality control component. For example, the FWS Alaska Region schema (see: line 575) requires conditional elements related to spatial data.
  • Data.gov uses the DCAT-US theme property to flag geospatial records that should be harvested by the Geoplatform. We should probably also consider if we want the DCAT-US translation to support flagging records for import to the Geoplatform.

I also notice that this change has already been merged into the iso_scope.yml file on the develop branch. We can probably keep the code list as it is since the underlying values stored in the JSON do not change. But we'll probably need to retain the current behavior of displaying "geographicDataset" in the picklist until we figure out how to deal with this.

[dwalt]

@hmaier-fws I thought we had reached agreement on this issue. The simple solution without getting into profiles is to extend the codelist for Geographic Dataset. Then mdEditor can offer a straight codelist drop down. Probably what we are missing is to trap for previous use of the Dataset code by FWS and convert it to a new extended Geographic Dataset code, preserving the categorization.