Skip to content

Commit

Permalink
Merge pull request #642 from davidradl/git640
Browse files Browse the repository at this point in the history 
git640 update readme
  • Loading branch information
@davidradl
davidradl committed Dec 21, 2022
2 parents dc5d374 + 897f9b4 commit aac6a43
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 25 deletions.
43 changes: 22 additions & 21 deletions site/docs/connectors/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ The connectors that support the exchange and maintenance of metadata help to acc

| Type of Connector | Description |
|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Integration connectors](#integration-connectors) | manage the metadata exchange to a third party technology through an [integration service](/services/omis). |
| [Repository and Event Mapper connectors](#repository-and-event-mapper-connectors) | integrate metadata repositories into the open metadata ecosystem so that they can interact with one or more [open metadata repository cohorts](/services/omrs/cohort). |
| [Open Discovery Services](#open-discovery-services) | analyze the content of [resources](/concepts/resource) in the digital landscape and create annotations that are attached to the resource's [asset](/concepts/asset) metadata element in the open metadata repositories in the form of an [discovery analysis report](/concepts/discovery-analysis-report). |
| [Governance Action Services](#governance-action-services) | perform monitoring of metadata changes, validation of metadata, triage of issues, assessment and/or remediation activities as required. |
| [Governance Daemon Connectors](#governance-daemon-connectors) | contain specialist connectors for the governance servers that make active use of open metadata. |
| [Integration connectors](#integration-connectors) | Manage the metadata exchange to a third party technology through an [integration service](/services/omis). |
| [Repository and Event Mapper connectors](#repository-and-event-mapper-connectors) | Integrate metadata repositories into the open metadata ecosystem so that they can interact with one or more [open metadata repository cohorts](/services/omrs/cohort). |
| [Open Discovery Services](#open-discovery-services) | Analyze the content of [resources](/concepts/resource) in the digital landscape and create annotations that are attached to the resource's [asset](/concepts/asset) metadata element in the open metadata repositories in the form of an [discovery analysis report](/concepts/discovery-analysis-report). |
| [Governance Action Services](#governance-action-services) | Perform monitoring of metadata changes, validation of metadata, triage of issues, assessment and/or remediation activities as required. |
| [Governance Daemon Connectors](#governance-daemon-connectors) | Contain specialist connectors for the governance servers that make active use of open metadata. |

### Integration Connectors

Expand All @@ -37,34 +37,35 @@ The *files* integration connectors run in the [Files Integrator Open Metadata In

| Files Integration Connectors | Description |
|---|---|
| [Data files monitor](/connectors/integration/data-files-monitor-integration-connector) | maintains a `DataFile` asset for each file in the directory (or any subdirectory). When a new file is created, a new DataFile asset is created. If a file is modified, the lastModified property of the corresponding DataFile asset is updated. When a file is deleted, its corresponding DataFile asset is also deleted (or archived if it is still needed for lineage). |
| [Data folder monitor](/connectors/integration/data-folder-monitor-integration-connector) | maintains a `DataFolder` asset for the directory. The files and directories underneath it are assumed to be elements/records in the DataFolder asset and so each time there is a change to the files and directories under the monitored directory, it results in an update to the lastModified property of the corresponding DataFolder asset. |
| [Data files monitor](/connectors/integration/data-files-monitor-integration-connector) | Maintains a `DataFile` asset for each file in the directory (or any subdirectory). When a new file is created, a new DataFile asset is created. If a file is modified, the lastModified property of the corresponding DataFile asset is updated. When a file is deleted, its corresponding DataFile asset is also deleted (or archived if it is still needed for lineage). |
| [Data folder monitor](/connectors/integration/data-folder-monitor-integration-connector) | Maintains a `DataFolder` asset for the directory. The files and directories underneath it are assumed to be elements/records in the DataFolder asset and so each time there is a change to the files and directories under the monitored directory, it results in an update to the lastModified property of the corresponding DataFolder asset. |

#### Cataloguing Databases and their Schemas

The *database* integration connectors run in the [Database Integrator Open Metadata Integration Service (OMIS)](/services/omis/database-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon).

| Database Integration Connectors | Description |
|--------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [PostgreSQL database connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/egeria-connector-postgres){ target=gh } | automatically maintains the open metadata instances for the databases hosted on a [PostgreSQL server :material-dock-window:](https://www.postgresql.org){ target=psql } This includes the database schemas, tables, columns, primary keys and foreign keys. |
| [JDBC integration connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/jdbc-integration-connector){ target=gh } | automatically maintains the open metadata instances on a database server via JDBC. This includes the database schemas, tables, columns, primary keys and foreign keys. |
| [PostgreSQL database connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/egeria-connector-postgres){ target=gh } | Automatically maintains the open metadata instances for the databases hosted on a [PostgreSQL server :material-dock-window:](https://www.postgresql.org){ target=psql } This includes the database schemas, tables, columns, primary keys and foreign keys. |
| [JDBC integration connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/jdbc-integration-connector){ target=gh } | Automatically maintains the open metadata instances on a database server via JDBC. This includes the database schemas, tables, columns, primary keys and foreign keys. |

#### Cataloguing event topics and the structure of their events

The *topic* integration connectors run in the [Topic Integrator Open Metadata Integration Service (OMIS)](/services/omis/topic-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon).

| Topic Integration Connector | Description |
|---|---|
| [Kafka Monitor topic integration connector](/connectors/integration/kafka-monitor-integration-connector) | automatically maintains the open metadata instances for the topics hosted on an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka }.|
| [Kafka Audit topic integration connector](/connectors/integration/kafka-topics-audit-integration-connector) | Validates that topics that are active in an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka } are also catalogued in open metadata. Creates an audit log record for each topic that is not catalogued. |
| Topic Integration Connector | Description |
|----------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Kafka Monitor topic integration connector](/connectors/integration/kafka-monitor-integration-connector) | Automatically maintains the open metadata instances for the topics hosted on an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka }. |
| [Kafka Audit topic integration connector](/connectors/integration/kafka-topics-audit-integration-connector) | Validates that topics that are active in an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka } are also catalogued in open metadata. Creates an audit log record for each topic that is not catalogued. |
| [Strimzi Monitor topic integration connector](/connectors/integration/strimzi-monitor-integration-connector) | Automatically maintains the open metadata instances for the topics hosted in [Strimzi :material-dock-window:](https://strimzi.io){ target=strimzi }. |

#### Cataloguing APIs

The *API* integration connectors run in the [API Integrator Open Metadata Integration Service (OMIS)](/services/omis/api-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon).

| API Integration Connectors | Description |
|---|---|
| [Open API Monitor integration connector](/connectors/integration/open-api-monitor-integration-connector) | automatically maintains the open metadata instances for the APIs extracted from the Open API Specification extracted from an application.|
| [Open API Monitor integration connector](/connectors/integration/open-api-monitor-integration-connector) | Automatically maintains the open metadata instances for the APIs extracted from the Open API Specification extracted from an application.|

#### Populating security enforcement engines

Expand All @@ -75,14 +76,14 @@ The security integration connectors run in the [Security Integrator Open Metadat
The lineage integration connectors run in the [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon). They support [Lineage Management](/features/lineage-management/overview).


| Lineage Integration Connectors | Description |
|--------------------------------------------------------------------------------------------------------------------------------------------|---|
| [Open Lineage Event Receiver integration connector](/connectors/integration/open-lineage-event-receiver-integration-connector) | Connector to receive open lineage events from an event topic and publish them to lineage integration connectors with listeners registered in the same instance of the Lineage Integrator OMIS.|
| [Governance Action to Open Lineage integration connector](/connectors/integration/governance-action-open-lineage-integration-connector) | Connector to listen for governance actions executing in the open metadata ecosystem, generate open lineage events for them and publish them to the integration connectors running in the same instance of Lineage Integrator OMIS that are listening for OpenLineage events. |
| [API-based Open Lineage Log Store integration connector](/connectors/integration/api-based-open-lineage-log-store-integration-connector) | Connector that calls an OpenLineage compliant API to store the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS.|
| Lineage Integration Connectors | Description |
|--------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Open Lineage Event Receiver integration connector](/connectors/integration/open-lineage-event-receiver-integration-connector) | Connector to receive open lineage events from an event topic and publish them to lineage integration connectors with listeners registered in the same instance of the Lineage Integrator OMIS. |
| [Governance Action to Open Lineage integration connector](/connectors/integration/governance-action-open-lineage-integration-connector) | Connector to listen for governance actions executing in the open metadata ecosystem, generate open lineage events for them and publish them to the integration connectors running in the same instance of Lineage Integrator OMIS that are listening for OpenLineage events. |
| [API-based Open Lineage Log Store integration connector](/connectors/integration/api-based-open-lineage-log-store-integration-connector) | Connector that calls an OpenLineage compliant API to store the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS. |
| [File-based Open Lineage Log Store integration connector](/connectors/integration/file-based-open-lineage-log-store-integration-connector) | Connector that stores the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS. Each OpenLineage event is stored in its own file in JSON format. These files are organized according to the namespace and job name in the event. |
| [Open Lineage Cataloguer integration connector](/connectors/integration/open-lineage-cataloguer-integration-connector) | Connector to register an OpenLineage listener with the Lineage Integrator OMIS and to catalog any processes that are not already known to the open metadata ecosystem. |
| [Sample Lineage integration connector](/connectors/integration/sample-lineage-event-receiver-integration-connector) | Connector to register an OpenLineage listener with the Lineage Integrator OMIS and to catalog any processes that are not already known to the open metadata ecosystem. |
| [Open Lineage Cataloguer integration connector](/connectors/integration/open-lineage-cataloguer-integration-connector) | Connector to register an OpenLineage listener with the Lineage Integrator OMIS and to catalog any processes that are not already known to the open metadata ecosystem. |
| [Sample Lineage integration connector](/connectors/integration/sample-lineage-event-receiver-integration-connector) | Sample Connector to listen for lineage events (not in a open lineage format) and catalog the associated assets, processes, schema and lineage. |


??? education "Further information relating to integration connectors"
Expand Down
Loading

0 comments on commit aac6a43

Please sign in to comment.