Prevent and cleanup duplicate records

[hmaier-fws]

Importing mdJSON can create duplicate metadata, contact and dictionary records.

Metadata, contact and dictionary records created by mdEditor have associated unique identifiers (UUID) assigned to each record at the time of creation ("metadataIdentifier", "contactID", "dictionaryID"). A short internal "mdEditor identifer" ("id") is also assigned to each record for use in managing the Ember datastore and is displayed in the browser as the record endpoint.

The internal "mdEditor identifier" ("id") is not included in the mdJSON exports. On import of a mdJSON file, the editor assigns the imported records a randomly generated "id". Importing a mdJSON file multiple times will result in a different "mdEditor identifier" being assigned to records that have the same "metadataIdentifier", "contactID" or "dictionaryID". The issue is restricted to importing mdJSON ("id" is embedded in mdEditorJSON).

Possible features to prevent the creation of and remove duplicate records:

• Change how ember "id" values are assigned (Assign ember record identifiers ("id") based on UUID's #481)
• Add option to check for duplicate UUID's on import.
  1. Keep existing
  2. Use new
  3. Show diff and select
  4. Others?
• Create utilities to identify and eliminate duplicate records.
• Check current metadata, contact and dictionary records for duplicated id's
• Identify possible rules/options for selecting the record to retain.

See also: #67, #341

[dwalt]

Consider contactId, dictionaryId and other components as "internal keys" that are persistent. Consider adding timeDate stamp that acts as part of a two-part key along with contactId which can be used to identify duplicates by the system. Also provides basic metadata of last update.