diff --git a/site/docs/connectors/unity-catalog/index.md b/site/docs/connectors/unity-catalog/index.md index 9f71ee3e64..eaf760e54a 100644 --- a/site/docs/connectors/unity-catalog/index.md +++ b/site/docs/connectors/unity-catalog/index.md @@ -100,6 +100,29 @@ The diagram below illustrates the mapping of the Unity Catalog metadata resource ![Type Mapping](unity-catalog-type-mapping.svg) +Below is the field mapping from Unity Catalog to these Open Metadata Types: + +=== "Catalogs" + + --8<-- "snippets/unity-catalog/catalog-mapping.md" + +=== "Schemas" + + --8<-- "snippets/unity-catalog/schema-mapping.md" + +=== "Volumes" + + --8<-- "snippets/unity-catalog/volume-mapping.md" + +=== "Tables" + + --8<-- "snippets/unity-catalog/table-mapping.md" + +=== "Functions" + + --8<-- "snippets/unity-catalog/function-mapping.md" + + The templates that implement this mapping are described in [Unity Catalog Templates](/catalog-templates/unity-catalog-templates). ### Anchor design for Unity Catalog diff --git a/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg b/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg index 16b24dbfba..348cf747c1 100644 --- a/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg +++ b/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg @@ -1,4 +1,4 @@ -SoftwareServerUnity Catalog ServerUnity Catalog CatalogDeployedSoftwareComponentVirtualRelationalTableUnity Catalog SchemaData FolderCatalog(Software Capability)Unity Catalog VolumeUnity Catalog TableRelational DB Schema TypeRelational TableRelational ColumnRelational ColumnRelational ColumnUnity Catalog FunctionAssetSchemaDataContentForDataSetAPI Schema TypeDeployed APIProcessCall(implementation)AssetSchemaServerAssetUseSupportedSoftwareCapabilityDataContentForDataSetDataContentForDataSetPropertyFacetPropertyFacetPropertyFacetExternalIdentifierExternalIdentifierExternalIdentifierExternalIdentifierExternalIdentifierDataContentForDataSetDeployedDatabaseSchemaData Folder \ No newline at end of file +SoftwareServerUnity Catalog ServerUnity Catalog CatalogDeployedSoftwareComponentVirtualRelationalTableUnity Catalog SchemaData FolderCatalog(Software Capability)Unity Catalog VolumeUnity Catalog TableRelational DB Schema TypeRelational TableRelational ColumnRelational ColumnRelational ColumnUnity Catalog FunctionAssetSchemaDataContentForDataSetAPI Schema TypeDeployed APIProcessCall(implementation)Created whenUC Instance identifiedAssetSchemaServerAssetUseSupportedSoftwareCapabilityDataContentForDataSetDataContentForDataSetPropertyFacetPropertyFacetPropertyFacetExternalIdentifierExternalIdentifierExternalIdentifierExternalIdentifierExternalIdentifierDataContentForDataSetDeployedDatabaseSchemaData Folder \ No newline at end of file diff --git a/site/snippets/unity-catalog/catalog-mapping.md b/site/snippets/unity-catalog/catalog-mapping.md new file mode 100644 index 0000000000..1570078bf4 --- /dev/null +++ b/site/snippets/unity-catalog/catalog-mapping.md @@ -0,0 +1,32 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|---------------------------------|---------------------------------------------------------|--------------------------| +| id | ExternalIdentifier.identifier | Yes | +| name | Catalog.name | Yes | +| comment | Catalog.description | Yes | +| owner | Ownership.owner | No | +| name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_type | PropertyFacet.properties.ucCatalogType | No | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| isolation_mode | PropertyFacet.properties.ucIsolationMode | No | +| accessible_in_current_workspace | PropertyFacet.properties.ucAccessibleInCurrentWorkspace | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|-----------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Catalog: Server URL : full_name | Catalog.qualifiedName | Yes | +| Unity Catalog Catalog | Catalog.deployedImplementationType | Yes | +| Databricks Unity Catalog Catalog: Server URL | Catalog.qualifiedName | No | +| Databricks Unity Catalog Catalog | Catalog.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/function-mapping.md b/site/snippets/unity-catalog/function-mapping.md new file mode 100644 index 0000000000..6570942d05 --- /dev/null +++ b/site/snippets/unity-catalog/function-mapping.md @@ -0,0 +1,35 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|----------------------------------------------------|--------------------------| +| function_id | ExternalIdentifier.identifier | Yes | +| name | DeployedAPI.name | Yes | +| full_name | DeployedAPI.resourceName | Yes | +| comment | DeployedAPI.description | Yes | +| owner | Ownership.owner | No | +| data_source_format | DataAssetEncoding.encoding | Yes | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| schema_name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| storage_location | PropertyFacet.properties.ucStorageLocation | Yes | +| table_type | PropertyFacet.properties.ucTableType | Yes | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| resource_name | PropertyFacet.properties.ucResourceName | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|-----------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Function: Server URL : full_name | DeployedAPI.qualifiedName | Yes | +| Unity Catalog Function | DeployedAPI.deployedImplementationType | Yes | +| Databricks Unity Catalog Function: Server URL : full_name | DeployedAPI.qualifiedName | No | +| Databricks Unity Catalog Function | DeployedAPI.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/schema-mapping.md b/site/snippets/unity-catalog/schema-mapping.md new file mode 100644 index 0000000000..9bf59f1832 --- /dev/null +++ b/site/snippets/unity-catalog/schema-mapping.md @@ -0,0 +1,33 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|----------------------------------------------------|--------------------------| +| schema_id | ExternalIdentifier.identifier | Yes | +| name | DeployedDatabaseSchema.name | Yes | +| full_name | DeployedDatabaseSchema.resourceName | Yes | +| comment | DeployedDatabaseSchema.description | Yes | +| owner | Ownership.owner | No | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| catalog_type | PropertyFacet.properties.ucCatalogType | No | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| properties | PropertyFacet.properties | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|---------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Schema: Server URL : full_name | DeployedDatabaseSchema.qualifiedName | Yes | +| Unity Catalog Schema | DeployedDatabaseSchema.deployedImplementationType | Yes | +| Databricks Unity Catalog Schema: Server URL : full_name | DeployedDatabaseSchema.qualifiedName | No | +| Databricks Unity Catalog Schema | DeployedDatabaseSchema.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/table-mapping.md b/site/snippets/unity-catalog/table-mapping.md new file mode 100644 index 0000000000..3e746858df --- /dev/null +++ b/site/snippets/unity-catalog/table-mapping.md @@ -0,0 +1,35 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|----------------------------------------------------|--------------------------| +| table_id | ExternalIdentifier.identifier | Yes | +| name | VirtualRelationalTable.name | Yes | +| full_name | VirtualRelationalTable.resourceName | Yes | +| comment | VirtualRelationalTable.description | Yes | +| owner | Ownership.owner | No | +| data_source_format | DataAssetEncoding.encoding | Yes | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| schema_name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| storage_location | PropertyFacet.properties.ucStorageLocation | Yes | +| table_type | PropertyFacet.properties.ucTableType | Yes | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| resource_name | PropertyFacet.properties.ucResourceName | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|--------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Table: Server URL : full_name | VirtualRelationalTable.qualifiedName | Yes | +| Unity Catalog Table | VirtualRelationalTable.deployedImplementationType | Yes | +| Databricks Unity Catalog Table: Server URL : full_name | VirtualRelationalTable.qualifiedName | No | +| Databricks Unity Catalog Table | VirtualRelationalTable.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/volume-mapping.md b/site/snippets/unity-catalog/volume-mapping.md new file mode 100644 index 0000000000..4a3619c8c4 --- /dev/null +++ b/site/snippets/unity-catalog/volume-mapping.md @@ -0,0 +1,34 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|--------------------------------------------------------------------------------------------|--------------------------| +| volume_id | ExternalIdentifier.identifier | Yes | +| name | FileFolder.name | Yes | +| full_name | FileFolder.resourceName | Yes | +| comment | FileFolder.description | Yes | +| storage_location | FileFolder.pathName + Endpoint.networkAddress + PropertyFacet.properties.ucStorageLocation | Yes | +| owner | Ownership.owner | No | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| schema_name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| volume_type | PropertyFacet.properties.ucVolumeType | Yes | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| resource_name | PropertyFacet.properties.ucResourceName | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|---------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Volume: Server URL : full_name | FileFolder.qualifiedName | Yes | +| Unity Catalog Volume | FileFolder.deployedImplementationType | Yes | +| Databricks Unity Catalog Volume: Server URL : full_name | FileFolder.qualifiedName | No | +| Databricks Unity Catalog Volume | FileFolder.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes |