diff --git a/site/docs/connectors/unity-catalog/index.md b/site/docs/connectors/unity-catalog/index.md index 9f71ee3e64..eaf760e54a 100644 --- a/site/docs/connectors/unity-catalog/index.md +++ b/site/docs/connectors/unity-catalog/index.md @@ -100,6 +100,29 @@ The diagram below illustrates the mapping of the Unity Catalog metadata resource ![Type Mapping](unity-catalog-type-mapping.svg) +Below is the field mapping from Unity Catalog to these Open Metadata Types: + +=== "Catalogs" + + --8<-- "snippets/unity-catalog/catalog-mapping.md" + +=== "Schemas" + + --8<-- "snippets/unity-catalog/schema-mapping.md" + +=== "Volumes" + + --8<-- "snippets/unity-catalog/volume-mapping.md" + +=== "Tables" + + --8<-- "snippets/unity-catalog/table-mapping.md" + +=== "Functions" + + --8<-- "snippets/unity-catalog/function-mapping.md" + + The templates that implement this mapping are described in [Unity Catalog Templates](/catalog-templates/unity-catalog-templates). ### Anchor design for Unity Catalog diff --git a/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg b/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg index 16b24dbfba..348cf747c1 100644 --- a/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg +++ b/site/docs/connectors/unity-catalog/unity-catalog-type-mapping.svg @@ -1,4 +1,4 @@ -
Software
Server
Unity Catalog Server
Unity Catalog Catalog
Deployed
Software
Component
Virtual
Relational
Table
Unity Catalog Schema
Data Folder
Catalog
(Software Capability)
Unity Catalog Volume
Unity Catalog Table
Relational DB Schema Type
Relational Table
Relational Column
Relational Column
Relational Column
Unity Catalog Function
AssetSchema
DataContentForDataSet
API Schema Type
Deployed API
ProcessCall
(implementation)
AssetSchema
ServerAssetUse
Supported
Software
Capability
DataContentForDataSet
DataContentForDataSet
PropertyFacet
PropertyFacet
PropertyFacet
ExternalIdentifier
ExternalIdentifier
ExternalIdentifier
ExternalIdentifier
ExternalIdentifier
DataContentForDataSet
Deployed
Database
Schema
Data Folder
\ No newline at end of file +
Software
Server
Unity Catalog Server
Unity Catalog Catalog
Deployed
Software
Component
Virtual
Relational
Table
Unity Catalog Schema
Data Folder
Catalog
(Software Capability)
Unity Catalog Volume
Unity Catalog Table
Relational DB Schema Type
Relational Table
Relational Column
Relational Column
Relational Column
Unity Catalog Function
AssetSchema
DataContentForDataSet
API Schema Type
Deployed API
ProcessCall
(implementation)
Created when
UC Instance identified
AssetSchema
ServerAssetUse
Supported
Software
Capability
DataContentForDataSet
DataContentForDataSet
PropertyFacet
PropertyFacet
PropertyFacet
ExternalIdentifier
ExternalIdentifier
ExternalIdentifier
ExternalIdentifier
ExternalIdentifier
DataContentForDataSet
Deployed
Database
Schema
Data Folder
\ No newline at end of file diff --git a/site/snippets/unity-catalog/catalog-mapping.md b/site/snippets/unity-catalog/catalog-mapping.md new file mode 100644 index 0000000000..1570078bf4 --- /dev/null +++ b/site/snippets/unity-catalog/catalog-mapping.md @@ -0,0 +1,32 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|---------------------------------|---------------------------------------------------------|--------------------------| +| id | ExternalIdentifier.identifier | Yes | +| name | Catalog.name | Yes | +| comment | Catalog.description | Yes | +| owner | Ownership.owner | No | +| name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_type | PropertyFacet.properties.ucCatalogType | No | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| isolation_mode | PropertyFacet.properties.ucIsolationMode | No | +| accessible_in_current_workspace | PropertyFacet.properties.ucAccessibleInCurrentWorkspace | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|-----------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Catalog: Server URL : full_name | Catalog.qualifiedName | Yes | +| Unity Catalog Catalog | Catalog.deployedImplementationType | Yes | +| Databricks Unity Catalog Catalog: Server URL | Catalog.qualifiedName | No | +| Databricks Unity Catalog Catalog | Catalog.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/function-mapping.md b/site/snippets/unity-catalog/function-mapping.md new file mode 100644 index 0000000000..6570942d05 --- /dev/null +++ b/site/snippets/unity-catalog/function-mapping.md @@ -0,0 +1,35 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|----------------------------------------------------|--------------------------| +| function_id | ExternalIdentifier.identifier | Yes | +| name | DeployedAPI.name | Yes | +| full_name | DeployedAPI.resourceName | Yes | +| comment | DeployedAPI.description | Yes | +| owner | Ownership.owner | No | +| data_source_format | DataAssetEncoding.encoding | Yes | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| schema_name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| storage_location | PropertyFacet.properties.ucStorageLocation | Yes | +| table_type | PropertyFacet.properties.ucTableType | Yes | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| resource_name | PropertyFacet.properties.ucResourceName | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|-----------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Function: Server URL : full_name | DeployedAPI.qualifiedName | Yes | +| Unity Catalog Function | DeployedAPI.deployedImplementationType | Yes | +| Databricks Unity Catalog Function: Server URL : full_name | DeployedAPI.qualifiedName | No | +| Databricks Unity Catalog Function | DeployedAPI.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/schema-mapping.md b/site/snippets/unity-catalog/schema-mapping.md new file mode 100644 index 0000000000..9bf59f1832 --- /dev/null +++ b/site/snippets/unity-catalog/schema-mapping.md @@ -0,0 +1,33 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|----------------------------------------------------|--------------------------| +| schema_id | ExternalIdentifier.identifier | Yes | +| name | DeployedDatabaseSchema.name | Yes | +| full_name | DeployedDatabaseSchema.resourceName | Yes | +| comment | DeployedDatabaseSchema.description | Yes | +| owner | Ownership.owner | No | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| catalog_type | PropertyFacet.properties.ucCatalogType | No | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| properties | PropertyFacet.properties | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|---------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Schema: Server URL : full_name | DeployedDatabaseSchema.qualifiedName | Yes | +| Unity Catalog Schema | DeployedDatabaseSchema.deployedImplementationType | Yes | +| Databricks Unity Catalog Schema: Server URL : full_name | DeployedDatabaseSchema.qualifiedName | No | +| Databricks Unity Catalog Schema | DeployedDatabaseSchema.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/table-mapping.md b/site/snippets/unity-catalog/table-mapping.md new file mode 100644 index 0000000000..3e746858df --- /dev/null +++ b/site/snippets/unity-catalog/table-mapping.md @@ -0,0 +1,35 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|----------------------------------------------------|--------------------------| +| table_id | ExternalIdentifier.identifier | Yes | +| name | VirtualRelationalTable.name | Yes | +| full_name | VirtualRelationalTable.resourceName | Yes | +| comment | VirtualRelationalTable.description | Yes | +| owner | Ownership.owner | No | +| data_source_format | DataAssetEncoding.encoding | Yes | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| schema_name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| storage_location | PropertyFacet.properties.ucStorageLocation | Yes | +| table_type | PropertyFacet.properties.ucTableType | Yes | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| resource_name | PropertyFacet.properties.ucResourceName | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|--------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Table: Server URL : full_name | VirtualRelationalTable.qualifiedName | Yes | +| Unity Catalog Table | VirtualRelationalTable.deployedImplementationType | Yes | +| Databricks Unity Catalog Table: Server URL : full_name | VirtualRelationalTable.qualifiedName | No | +| Databricks Unity Catalog Table | VirtualRelationalTable.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes | diff --git a/site/snippets/unity-catalog/volume-mapping.md b/site/snippets/unity-catalog/volume-mapping.md new file mode 100644 index 0000000000..4a3619c8c4 --- /dev/null +++ b/site/snippets/unity-catalog/volume-mapping.md @@ -0,0 +1,34 @@ + + + + +| Unity Catalog Property | Egeria Type and Attribute | Supported in OSS Version | +|------------------------|--------------------------------------------------------------------------------------------|--------------------------| +| volume_id | ExternalIdentifier.identifier | Yes | +| name | FileFolder.name | Yes | +| full_name | FileFolder.resourceName | Yes | +| comment | FileFolder.description | Yes | +| storage_location | FileFolder.pathName + Endpoint.networkAddress + PropertyFacet.properties.ucStorageLocation | Yes | +| owner | Ownership.owner | No | +| created_at | ExternalIdentifier.externalInstanceCreationTime | Yes | +| created_by | ExternalIdentifier.externalInstanceCreatedBy | No | +| updated_at | ExternalIdentifier.externalInstanceLastUpdateTime | Yes | +| updated_by | ExternalIdentifier.externalInstanceLastUpdatedBy | No | +| catalog_name | ExternalIdentifier.mappingProperties.ucCatalogName | Yes | +| schema_name | ExternalIdentifier.mappingProperties.ucSchemaName | Yes | +| volume_type | PropertyFacet.properties.ucVolumeType | Yes | +| metastore_id | PropertyFacet.properties.ucMetastoreId | No | +| browse_only | PropertyFacet.properties.ucBrowseOnly | No | +| securable_type | PropertyFacet.properties.ucSecurableType | No | +| securable_kind | PropertyFacet.properties.ucSecurableKind | No | +| resource_name | PropertyFacet.properties.ucResourceName | No | + + + +| Additional Values | Egeria Type and Attribute | Supported in OSS Version | +|---------------------------------------------------------|-----------------------------------------------------------|--------------------------| +| Unity Catalog Volume: Server URL : full_name | FileFolder.qualifiedName | Yes | +| Unity Catalog Volume | FileFolder.deployedImplementationType | Yes | +| Databricks Unity Catalog Volume: Server URL : full_name | FileFolder.qualifiedName | No | +| Databricks Unity Catalog Volume | FileFolder.deployedImplementationType | No | +| Server URL | ExternalIdentifier.mappingProperties.serverNetworkAddress | Yes |