diff --git a/site/docs/connectors/connector-catalog.drawio b/site/docs/connectors/connector-catalog.drawio index faf2bc3d18..72493823c3 100644 --- a/site/docs/connectors/connector-catalog.drawio +++ b/site/docs/connectors/connector-catalog.drawio @@ -1,4 +1,4 @@ - + @@ -1212,6 +1212,41 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -3720,28 +3755,40 @@ - + - + - + - + - + - + - + + + + + + + + + + + + + diff --git a/site/docs/connectors/databases/postgres-database-survey-action-service.md b/site/docs/connectors/databases/postgres-database-survey-action-service.md new file mode 100644 index 0000000000..732e3bebfb --- /dev/null +++ b/site/docs/connectors/databases/postgres-database-survey-action-service.md @@ -0,0 +1,8 @@ + + + +# PostgreSQL Database Survey Action Service + +Surveys a PostgreSQL database, its tables and columns. + +Its Jar file includes the PostgreSQL client driver. \ No newline at end of file diff --git a/site/docs/connectors/egeria/omag-server-platform-connector.md b/site/docs/connectors/egeria/omag-server-platform-connector.md new file mode 100644 index 0000000000..8df3b58482 --- /dev/null +++ b/site/docs/connectors/egeria/omag-server-platform-connector.md @@ -0,0 +1,21 @@ + + + +--8<-- "snippets/content-status/tech-preview.md" + +# OMAG Server Platform Connector + +???+ info "Connector summary" + - Connector Category: [Digital Resource Connector](/concepts/digital-resource-connector) + - Source Module: [egeria-system-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/system-connectors/egeria-system-connectors){ target=gh } + - Jar File Name: `egeria-system-connectors.jar` + +Th OMAG Server Platform Connector is a [digital resource connector](/concepts/digital-resource-connector) that provides a Java interface to an [OMAG Server Platform](/concepts/omag-server-platform). + +![Figure 1](apache-atlas-rest-connector.svg) +> **Figure 1:** Operation of the OMAG Server Platform Connector + + +--8<-- "snippets/abbr.md" + +- \ No newline at end of file diff --git a/site/docs/connectors/index.md b/site/docs/connectors/index.md index 7b476d06ed..ae898cacca 100644 --- a/site/docs/connectors/index.md +++ b/site/docs/connectors/index.md @@ -30,7 +30,10 @@ The connectors that support the exchange and maintenance of metadata help to acc ### Secrets Stores -* The [Environment Variables Secret Store connector](/connectors/secrets/environment-variable-secrets-store-connector) retrieves secret values from environment variables. +[Secrets stores](/concepts/secrets-store-connector) externalize secrets such as passwords, tokens and certificates so they do not need to be stored in either the [configuration document](/concepts/configuration-document) or [open metadata repositories](/concepts/open-metadata-repository). + +* The [Environment Variables Secret Store connector](/connectors/secrets/environment-variable-secrets-store-connector) retrieves secret values from environment variables. +* The [YAML File Secret Store connector](/connectors/secrets/yaml-secrets-store-connector) retrieves secret values from environment variables. ### Files diff --git a/site/docs/connectors/secrets/yaml-file-secrets-store-connector.md b/site/docs/connectors/secrets/yaml-file-secrets-store-connector.md new file mode 100644 index 0000000000..03fe58a457 --- /dev/null +++ b/site/docs/connectors/secrets/yaml-file-secrets-store-connector.md @@ -0,0 +1,41 @@ + + + +--8<-- "snippets/content-status/tech-preview.md" + +# YAML File Secrets Store Connector + +!!! info "Connector details" + - Connector Category: [Secret Store Connector](/concepts/secrets-store-connector) + - Source Module: [yaml-secrets-store-connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/secrets-store-connectors/yaml-secrets-store-connector){ target=gh } + - Jar File Name: `yaml-secrets-store-connector.jar` + +## Overview + +The *YAML File Secrets Store Connector* retrieves secrets from environment variables. The name of the secret requested is the name of the environment variable it tries to retrieve. It returns null if the environment variable is not defined. + +![Figure 1](yaml-file-secrets-store-connector.svg) +> **Figure 1:** Operation of the YAML File Secrets Store Connector + + +## Configuration + +This is its connection definition to embed into a connector's connection object. + +!!! example "Connection configuration for the environment variable secrets store connector" + ```json linenums="1" hl_lines="14" + { + "connection" : + { + "class" : "Connection", + "qualifiedName" : "Egeria:SecretsStoreConnector:YAML File Connection", + "connectorType" : + { + "class" : "ConnectorType", + "connectorProviderClassName" : "org.odpi.openmetadata.adapters.connectors.secretsstore.yaml.YAMLSecretsStoreProvider" + } + } + } + ``` + +---8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/secrets/yaml-file-secrets-store-connector.svg b/site/docs/connectors/secrets/yaml-file-secrets-store-connector.svg new file mode 100644 index 0000000000..d674c238ab --- /dev/null +++ b/site/docs/connectors/secrets/yaml-file-secrets-store-connector.svg @@ -0,0 +1,4 @@ + + + +
Consuming Connector
YAML File
Secrets Store Connector
mySecrets.
omsecrets
 \ No newline at end of file diff --git a/site/docs/content-packs/index.md b/site/docs/content-packs/index.md index 2bffc4696e..231ebc3cfb 100644 --- a/site/docs/content-packs/index.md +++ b/site/docs/content-packs/index.md @@ -3,4 +3,41 @@ # Content Packs +The Egeria content packs provide a means to distribute standard metadata to different organizations. They are formatted as [Open Metadata Archives](/concepts/open-metadata-archive) and can be loaded using at server start up (see [Configuring the Start-up Archives](/guides/admin/servers/by-server-type/configuring-a-metadata-access-store)) or while the server is running (see [Runtime Manager OMVS](/services/omvs/runtime-manager/overview)). + +The content packs are as follows: + +* [CoreContentPack](/content-packs/core-content-pack/overview) - contains the connector definitions for the open connectors supplied in this distribution along with the valid metadata values for the technologies they support. This content pack is designed to provide a good starting point for a new Egeria deployment. It is loaded automatically in the active-metadata-store sample server. + +* [CloudInformationModel](/content-packs/cim-content-pack/overview) - glossary from the Cloud Information Model's JSONLD formatted model. The content covers basic commerce concepts such as Party, Product, Invoice and Shipping. The cloud information project has been archived. However, this is a useful starter set glossary, + +* [OpenMetadataTypes](/content-packs/types-content-pack/overview) - Archive of the open metadata types defined by Egeria. This archive contains all the open metadata type definitions provided by Egeria. It is supplied for external utilities since each OMAG server capable of being a cohort member will load these types on start up. + +* [CocoComboArchive](/content-packs/coco-content-pack/overview) - supplies metadata to support the Coco Pharmaceuticals scenarios. It is a useful content pack to load when experimenting with Egeria's capabilities since it provides examples of many types of open metadata. In addition, this metadata is also available in the following archives that are used in the Open Metadata Labs where different subsets of this metadata are loading into each of the servers. + + * CocoBusinessSystemsArchive provides a catalog of the business systems and the lineage between them and the load of their data into the data lake. This archive simulates the type of metadata expected from an ETL tool suite. It is intended for cocoMDS5 in the open metadata labs but can be used in any server. + + * CocoOrganizationArchive - provides the profiles, user identifies and team of the featured personas of Coco Pharmaceuticals. + + * CocoClinicalTrialsTemplatesArchive - provides the template assets used for onboarding weekly patient measurements during a clinical trial. + + * CocoGovernanceProgramArchive - provides the metadata to describe Coco Pharmaceuticals governance program. + + * CocoGovernanceEngineDefinitionsArchive - provides the metadata to describe Coco Pharmaceuticals three governance engines: AssetGovernance, AssetDiscovery and AssetQuality. + + * CocoSustainabilityArchive - provides the base definitions for Coco Pharmaceutical's sustainability initiative. + + * CocoTypesArchive - provides additional types for Coco Pharmaceuticals. These are BiopsyScope Enum, BiopsyReport Entity, BiopsySupportingEvidence Relationship and ReviewedByClinicalTrials Classification. + +* [SimpleCatalog](/content-packs/simple-content-pack/overview) - provides an example of a database, an API and an event structure linked to a glossary term. It is loaded automatically in the simple-metadata-store sample server. SimpleCatalog is also supplied as four archives for use in a demo showing 4 metadata access servers connected together in a single cohort. The archives are each loaded into a different server. It is then possible to show how the cohort integrates metadata from different catalogs. These archives are used in the Development labs which are part of the Open Metadata Labs. + + * SimpleAPICatalog - API metadata typically found in an API catalog. + + * SimpleDataCatalog - Database metadata typically found in an Data catalog. + + * SimpleEventCatalog - Event metadata typically found in an API catalog. + + * SimpleGovernanceCatalog - A glossary term linked to metadata elements in the API, Event, Data catalogs. + + --8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/guides/developer/dev-projects/egeria-platform-report-no-config-files.md b/site/docs/guides/developer/dev-projects/egeria-platform-report-no-config-files.md index c4fe834965..07cb60cea5 100644 --- a/site/docs/guides/developer/dev-projects/egeria-platform-report-no-config-files.md +++ b/site/docs/guides/developer/dev-projects/egeria-platform-report-no-config-files.md @@ -33,7 +33,7 @@ Platform report for: https://localhost:9443 DevOps OMAS: Manage a DevOps pipeline Software Developer OMAS: Interact with software development tools Project Management OMAS: Manage governance related projects - Governance Engine OMAS: Set up an operational governance engine + Governance Server OMAS: Set up an operational governance engine Data Privacy OMAS: Manage governance of privacy Security Manager OMAS: Manages exchange of metadata with a security service Repository Governance OMES: Maintains open metadata archives based on the activity in the connected cohorts. diff --git a/site/docs/guides/developer/integration-connectors/integration-connector-design.drawio b/site/docs/guides/developer/integration-connectors/integration-connector-design.drawio new file mode 100644 index 0000000000..31ee48ba04 --- /dev/null +++ b/site/docs/guides/developer/integration-connectors/integration-connector-design.drawio @@ -0,0 +1,82 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/practices/coco-pharmaceuticals/personas/lemmie-stage.md b/site/docs/practices/coco-pharmaceuticals/personas/lemmie-stage.md index 59906f9223..da5bb9ba21 100644 --- a/site/docs/practices/coco-pharmaceuticals/personas/lemmie-stage.md +++ b/site/docs/practices/coco-pharmaceuticals/personas/lemmie-stage.md @@ -1,43 +1,48 @@ -# Lemmie Stage - DataStage Specialist +# Lemmie Stage - DevOps Specialist ![Icon](lemmie-stage.png) Age: 62 -Skills: Information integration using DataStage. +Skills: Automation, Containerized deployment of services. -Scope: IT – links existing systems. +Scope: IT – deploys new services and systems. Job: -* Lemmie is a deep expert in DataStage. He has implemented all of the DataStage jobs that link the company’s IT systems together. +* Lemmie is responsible for building and deploying containers with new systems and services. He is engaged in rolling out the Egeria technology at Coco Pharmaceuticals. * Lemmie works for [Polly Tasker](/practices/coco-pharmaceuticals/personas/polly-tasker). Thinks/Feels: -* DataStage is the most important tool in IT. +* Two years ago, Lemmie started retraining from an IBM DataStage specialist to a DevOps engineer. The transition was initially difficult but now he is Coco Pharmaceuticals principle DevOps Engine. He likes to be a deep expert and is now confident in his ability to work with the DevOps technology. + +* He is keen to follow the cloud-first principles. The rollout of Egeria fits well with many of these principles - although he is a little uneasy that Data Scientists like [Callie Quartile](callie-quartile.md) and Data Engineers like [Peter Profile](peter-profile.md) are able to configure the systems that Egeria talks to rather than encoding this in the container definitions. Hears: -* New requirements for additional DataStage jobs – everything else is filtered out. +* New requirements for additional systems and container changes – everything else is filtered out. Says/Does: -* Lemmie is very focused on what he does. If it is not DataStage then he is not interested. +* Lemmie is very focused on what he does. If it is not DevOps then he is not interested. He is also very professional and is willing to argue with his colleagues if he feels that something is not being done according to best practices. Sees: -* Unnecessary focus on metadata and business user access to information. +* Deep focus on metadata and business user access to information. This is reflected in the way that Egeria is being rolled out at Coco Pharmaceuticals where data professionals are able to configure Egeria after it has been deployed. Top challenges: -* Some of the integrations he does, especially with packaged applications are pretty tricky. He would like some better wizards in the designer tool to help. +* Security of the systems he deploys is a constant worry - particularly with new security exposures being discovered all the time. +* Keeping up-to-date with the latest DevOps technology. +* When Lemmie worked on DataStage, he was left alone to do the job the way he wanted. His new role is more mainstream and he sometimes has to justify what he is doing and make adjustments away from a perfect solution to satisfy a "business requirement". This can be difficult and frustrating for Lemmie, who often finds it difficult to explain why things should be done in a certain way. Desired Outcome: +* He would like to get all the IT systems and services onto the latest container technology with full automation and consistent software stacks. * He would like [Polly](/practices/coco-pharmaceuticals/personas/polly-tasker) and [Erin Overview](/practices/coco-pharmaceuticals/personas/erin-overview) (Information Architect) to appreciate how vital he is to the business and realize how skilled he is. diff --git a/site/docs/services/omvs/runtime-manager/egeria-system-connectors.drawio b/site/docs/services/omvs/runtime-manager/egeria-system-connectors.drawio index 2db271b7d4..834cd0193c 100644 --- a/site/docs/services/omvs/runtime-manager/egeria-system-connectors.drawio +++ b/site/docs/services/omvs/runtime-manager/egeria-system-connectors.drawio @@ -1,45 +1,426 @@ - - - + + + - - + + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/services/omvs/runtime-manager/egeria-system-connectors.svg b/site/docs/services/omvs/runtime-manager/egeria-system-connectors.svg new file mode 100644 index 0000000000..06b119308a --- /dev/null +++ b/site/docs/services/omvs/runtime-manager/egeria-system-connectors.svg @@ -0,0 +1,4 @@ + + + +
View Server
Runtime Manager OMVS
OMAG Server Platform
OMAG Server Platform Connector
Engine Host Connector
Integration Daemon Connector
Metadata Access Server Connector
View Server Connector
Metadata Access Server
Integration
Daemon
Engine Host
View Server
 \ No newline at end of file diff --git a/site/docs/services/omvs/runtime-manager/overview.md b/site/docs/services/omvs/runtime-manager/overview.md index 96db3f6da2..35461ecfbd 100644 --- a/site/docs/services/omvs/runtime-manager/overview.md +++ b/site/docs/services/omvs/runtime-manager/overview.md @@ -6,11 +6,47 @@ hide: ---8<-- "snippets/content-status/in-development.md" +--8<-- "snippets/content-status/tech-preview.md" # Runtime Manager OMVS -The Runtime Manager Open Metadata View Services (OMVS) is a REST API designed to support user interfaces (UIs) that display status information about the Egeria infrastructure. +The Runtime Manager Open Metadata View Services (OMVS) is a REST API designed to support user interfaces (UIs) and python environments that work with the Egeria infrastructure to: + +* Display the status of an [OMAG Server Platform](/concepts/omag-server-platform), and which [OMAG Servers](/concepts/omag-server) are known/active to it. +* The status of an OMAG Server, its active history and the health of the components running within it. +* The status of the connectors an OMAG Server is hosting. + +It is also able to: + +* Activate/deactivate servers on a platform. +* For [metadata access stores](/concepts/metadata-access-store), it can: + + * Load [open metadata archives](/concepts/open-metadata-archive). + * Connect and disconnect from connected [cohorts](/concepts/cohort-member). + +* For [integration daemons](/concept/integration-daemon), it can: + + * Temporarily change integration connector configuration. + * Restart and refresh integration connectors. + * Send [open lineage events](/features/lineage-management/overview) for augmentation, analysis, and distribution through the open metadata ecosystem. + * Refresh the server's integration group configuration stored in the connected metadata access server. + +* For [engine hosts](/concepts/engine-host) it can: + + * Refresh the server's governance engine configuration stored in the connected metadata access server. + + +The Runtime Manager OMVS is able to work with any platform (and its hosted servers) that are reachable via REST APIs. The calling user (provided they have sufficient security privileges) is able to work with the [platform services](/services/platform-services/overview), [server operation services](/services/server-operations), [integration daemon services](/services/integration-daemon-services) and [engine-host-services](/services/engine-host-services) as if they were calling them directly. This allows remote management without needing to expose the back-end platforms' endpoints to the administrative users. + +![Production Deployment](production-deployment.svg) + +## Egeria's resource connectors + +The secret to this service is that it works with Egeria's resource connectors that are responsible for making the calls. + +![Egeria resource connectors](egeria-system-connectors.svg) + +These resource connectors are accessed via the catalog entries for the platforms and servers. The catalog entries for the OMAG Server Platforms need to be created explicitly. ---8<-- "snippets/abbr.md" diff --git a/site/docs/services/omvs/runtime-manager/production-deployment.svg b/site/docs/services/omvs/runtime-manager/production-deployment.svg new file mode 100644 index 0000000000..d746540b78 --- /dev/null +++ b/site/docs/services/omvs/runtime-manager/production-deployment.svg @@ -0,0 +1,4 @@ + + + +
Frontline OMAG Server Platform
View Server
OMVS
OMVS
OMVS
OMVS
Runtime Manager
OMVS
Frontline OMAG Server Platform
View Server
OMVS
OMVS
OMVS
OMVS
Runtime Manager
OMVS
View Server
OMVS
OMVS
OMVS
OMVS
Runtime Manager
OMVS
Firewall
Backend OMAG Server Platform
Metadata
Access
Store
Integration
Daemon
Engine Host
metadata
Protected metadata, processing and data
Backend OMAG Server Platform
Metadata
Access
Store
Integration
Daemon
Engine Host
metadata
Data Pipelines
 \ No newline at end of file diff --git a/site/snippets/admin/configuring-the-engine-host-services.md b/site/snippets/admin/configuring-the-engine-host-services.md index 1deaa0e247..c00e2bad40 100644 --- a/site/snippets/admin/configuring-the-engine-host-services.md +++ b/site/snippets/admin/configuring-the-engine-host-services.md @@ -9,7 +9,7 @@ There are two parts to configuring the engine host services: ### Specify location of governance engine definitions -The location of the [metadata access server](/concepts/metadata-access-server) running the [Governance Engine OMAS](/services/omas/governance-engine) that will supply the definitions of the governance engines, is configured using two properties: +The location of the [metadata access server](/concepts/metadata-access-server) running the [Governance Server OMAS](/services/omas/governance-server/overview) that will supply the definitions of the governance engines, is configured using two properties: - the [server url root](/concepts/platform-url-root) of the metadata access server's OMAG Server Platform, and - the [name of the metadata access server](/concepts/server-name). diff --git a/site/snippets/admin/configuring-the-integration-groups.md b/site/snippets/admin/configuring-the-integration-groups.md index cb042881d5..5d4c28674b 100644 --- a/site/snippets/admin/configuring-the-integration-groups.md +++ b/site/snippets/admin/configuring-the-integration-groups.md @@ -4,7 +4,7 @@ ### Configure integration groups -An [integration group](/concepts/integration-group) identifies a list of [integration connectors](/concepts/integration-connector) that an integration daemon should run. These integration connectors are defined and maintained as open metadata elements stored in the open metadata repositories. Changes can be made to these integration connectors at any time using the [Governance Engine OMAS](/services/omas/governance-engine/overview). The integration daemon is monitoring for changes and updates the integration connectors it is running in line with the changes it discovers. This approach allows changes to be made to the integration daemon's configuration without needing access to the administration commands, nor a restart of the integration daemon. +An [integration group](/concepts/integration-group) identifies a list of [integration connectors](/concepts/integration-connector) that an integration daemon should run. These integration connectors are defined and maintained as open metadata elements stored in the open metadata repositories. Changes can be made to these integration connectors at any time using the [Governance Server OMAS](/services/omas/governance-server/overview). The integration daemon is monitoring for changes and updates the integration connectors it is running in line with the changes it discovers. This approach allows changes to be made to the integration daemon's configuration without needing access to the administration commands, nor a restart of the integration daemon. #### Configure an integration group diff --git a/site/snippets/concepts/governance-action-process-intro.md b/site/snippets/concepts/governance-action-process-intro.md index 8d465e94ff..0f6a042faa 100644 --- a/site/snippets/concepts/governance-action-process-intro.md +++ b/site/snippets/concepts/governance-action-process-intro.md @@ -2,7 +2,7 @@ -A *governance action process* is a predefined sequence of [engine actions](/concepts/engine-action) that are coordinated by the [Governance Engine OMAS](/services/omas/governance-engine/overview). +A *governance action process* is a predefined sequence of [engine actions](/concepts/engine-action) that are coordinated by the [Governance Server OMAS](/services/omas/governance-server/overview). The steps in a governance action process are defined by linked [governance action process steps](/concepts/governance-action-process-step) stored in the open metadata ecosystem. Each governance action process step provides the specification of the governance action to run. The links between them show which [guards](/concepts/guard) cause the next step to be chosen and hence, the governance action to run. @@ -35,16 +35,16 @@ The [engine actions](/concepts/engine-action) generated when a governance action ## Governance Action Process Lifecycle -The diagram below shows a governance action process assembly tool taking in information from a [governance engine pack](/concepts/governance-engine-pack) to build a governance action process flow. This is shared with the open metadata ecosystem either through direct called to the [Governance Engine OMAS](/services/omas/governance-engone/overview) or via a [open metadata archive](/concepts/open-metadata-archive) (possibly the archive that holds the governance engine definition. +The diagram below shows a governance action process assembly tool taking in information from a [governance engine pack](/concepts/governance-engine-pack) to build a governance action process flow. This is shared with the open metadata ecosystem either through direct called to the [Governance Server OMAS](/services/omas/governance-server/overview) or via a [open metadata archive](/concepts/open-metadata-archive) (possibly the archive that holds the governance engine definition. -Once the definition of the governance action process is available, an instance of the process can be started, either by a [watchdog governance action service](/concepts/governance-action-service) or through a direct call to the Governance Engine OMAS. Whichever mechanism is used, it results in the Governance Engine OMAS using the definition to choreograph the creation of [engine action](/concepts/engine-action) entities that drive the execution of the governance services in the [Engine Host](/concepts/engine-host). +Once the definition of the governance action process is available, an instance of the process can be started, either by a [watchdog governance action service](/concepts/governance-action-service) or through a direct call to the Governance Server OMAS. Whichever mechanism is used, it results in the Governance Server OMAS using the definition to choreograph the creation of [engine action](/concepts/engine-action) entities that drive the execution of the governance services in the [Engine Host](/concepts/engine-host). ![Lifecycle](/concepts/governance-action-process-lifecycle.svg) !!! education "Further information" - The [0462 Governance Action Processes](/types/4/0462-Governance-Action-Processes) model shows how the governance action process flow is built out of [governance action process steps](/concepts/governance-action-process-step). - - Governance action processes may be created using the [Governance Engine OMAS](/services/omas/governance-engine/overview) API. + - Governance action processes may be created using the [Governance Server OMAS](/services/omas/governance-server/overview) API. - The [Open Metadata Engine Services (OMES)](/services/omes) provide the mechanisms that support the different types of [governance engines](/concepts/governance-engine). These engines run the [governance services](/concepts/governance-service) that execute the [engine actions](/concepts/engine-action) defined by the governance action process. diff --git a/site/snippets/connectors/secrets-store-connector-intro.md b/site/snippets/connectors/secrets-store-connector-intro.md index b604e22dfd..c1db64b1ab 100644 --- a/site/snippets/connectors/secrets-store-connector-intro.md +++ b/site/snippets/connectors/secrets-store-connector-intro.md @@ -24,5 +24,5 @@ The picture below shows a practical example of using the secrets store connector ![Secured data centre](/connectors/secrets/secrets-store-connector-example.svg) -This pattern can also be used in the [Engine Host](/concepts/engine-host) with [Open Discovery Services](/concepts/open-discovery-service) and [governance action services](/concepts/governance-action-service). +This pattern can also be used in the [Engine Host](/concepts/engine-host) with [governance services](/concepts/governance-service). diff --git a/site/snippets/release-notes/5-1.md b/site/snippets/release-notes/5-1.md index aa88e92643..735e7cae08 100644 --- a/site/snippets/release-notes/5-1.md +++ b/site/snippets/release-notes/5-1.md @@ -1,7 +1,7 @@ -## Release 5.1 (Expected August 2024) +## Release 5.1 (Expected September 2024) _**These are DRAFT release notes. The notes will be updated until & when 5.1 is released**_ @@ -13,10 +13,29 @@ _**These are DRAFT release notes. The notes will be updated until & when 5.1 is * There are two new relationships relating to [governance actions](/concepts/governance-action). [TargetForActionType](/types/4/0462-Governance-Action-Processes) and [TargetForActionProcess](/types/4/0462-Governance-Action-Processes) allow targets for action to be pre-defined for a governance action type or governance action process respectively. The [GovernanceActionProcessFlow](/types/4/0462-Governance-Action-Processes) relationship receives a new property called `requestParameters`. Finally, there is a new entity type called [GovernanceActionProcessInstance](/types/4/0462-Governance-Action-Processes) that represents a running instance of a Governance Action Process. It captures the start and end time of the process's execution. ??? functional "Unity Catalog Support" - * There are new [survey services and integration connectors](/connectors/unity-catalog) to support the new open-source Unity Catalog (UC). + There are new [survey services and integration connectors](/connectors/unity-catalog) to support the new open-source Unity Catalog (UC). ??? functional "Smarter integration daemon" - * In addition to the regular refreshes, the integration daemon will refresh an integration connector whenever a new catalog target is attached. + In addition to the regular refreshes, the integration daemon will refresh an integration connector whenever a new catalog target is attached. + +??? functional "Extensions to the Open Integration Framework" + The OIF includes a RequestedCatalogTargetsManager class plus new base class methods for integration connectors that support dynamic configuration through catalog targets. This means the specialized implementation for processing a single catalog target can be implemented in a simple class, and the IntegrationConnectorBase will handle the changing list of catalog targets. + +??? functional "Open Lineage Connectors now work with catalog targets" + The [Open Lineage Connectors](/connectors/#open-lineage-events) now support dynamic configuration through [catalog targets](/concepts/catalog-target). This means that new Kafka Topics supplying open lineage events can be dynamically configured and new publishing destination APIs and folders can be added to the running connectors. + +??? danger "Changes to the default Configuration Document Store" + The Google Tink methods used in the [encrypted config document store](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/configuration-store-connectors/configuration-encrypted-file-store-connector) have been deprecated for some time. This connector was written (and used as the default) at a time when certificates and passwords were a part of the configuration document. With the introduction of the secrets connector, these are now stored in protected areas outside the configuration document so that an encrypted store is not necessary. This PR switches the default config store to the clear-text file implementation. In a later release, the encrypted file version of the connector will probably be removed unless updates to it to replace the deprecated methods are contributed to the project by an interested party. + + If you are currently using the encrypted file configuration document store, you can continue to use it by setting the `platform.configstore.provider` value in the platform's `application.properties` file. + + ``` + platform.configstore.provider=org.odpi.openmetadata.adapters.adminservices.configurationstore.encryptedfile.EncryptedFileBasedServerConfigStoreProvider + ``` + + Alternatively, if you with to switch to the clear-text version, use the administration services `getStoredConfiguration()` method to extract the JSON for your each of your server's configuration document while the encrypted configuration document file store is in place. Then delete the encrypted files from `platform/data/servers` and restart the platform with the clear-text configuration document store in place. Use the administration services `setOMAGServerConfig()` method to add your saved JSON back into the configuration document store. + + diff --git a/site/snippets/tasks/task-starting-omag-server.md b/site/snippets/tasks/task-starting-omag-server.md index 071ab8133c..858b7b6d05 100644 --- a/site/snippets/tasks/task-starting-omag-server.md +++ b/site/snippets/tasks/task-starting-omag-server.md @@ -15,7 +15,7 @@ The response from the command lists the subsystems that have been activated in t { "class": "SuccessMessageResponse", "relatedHTTPCode": 200, - "successMessage": "Sun Jun 11 21:19:16 BST 2023 myMetadataServer is running the following services: [Open Metadata Repository Services (OMRS), Connected Asset Services, Open Metadata Store Services, Open Integration Service, Data Manager OMAS, Design Model OMAS, Security Officer OMAS, Asset Manager OMAS, IT Infrastructure OMAS, Data Science OMAS, Community Profile OMAS, Digital Architecture OMAS, Asset Owner OMAS, Stewardship Action OMAS, Governance Program OMAS, Digital Service OMAS, Asset Lineage OMAS, Asset Consumer OMAS, DevOps OMAS, Software Developer OMAS, Project Management OMAS, Governance Engine OMAS, Data Privacy OMAS, Security Manager OMAS]" + "successMessage": "Sun Jun 11 21:19:16 BST 2023 myMetadataServer is running the following services: [Open Metadata Repository Services (OMRS), Connected Asset Services, Open Metadata Store Services, Open Integration Service, Data Manager OMAS, Design Model OMAS, Security Officer OMAS, Asset Manager OMAS, IT Infrastructure OMAS, Data Science OMAS, Community Profile OMAS, Digital Architecture OMAS, Asset Owner OMAS, Stewardship Action OMAS, Governance Program OMAS, Digital Service OMAS, Asset Lineage OMAS, Asset Consumer OMAS, DevOps OMAS, Software Developer OMAS, Project Management OMAS, Governance Server OMAS, Data Privacy OMAS, Security Manager OMAS]" } ``` @@ -145,10 +145,10 @@ Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0040 An enterpr Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Project Management OMAS has started Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-PROJECT-MANAGEMENT-0001 The Project Management Open Metadata Access Service (OMAS) is initializing a new server instance Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-PROJECT-MANAGEMENT-0003 The Project Management Open Metadata Access Service (OMAS) has initialized a new instance for server myMetadataServer -Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Governance Engine OMAS -Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Governance Engine OMAS has started -Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-GOVERNANCE-ENGINE-0001 The Governance Engine Open Metadata Access Service (OMAS) is initializing a new server instance -Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-GOVERNANCE-ENGINE-0005 The Governance Engine Open Metadata Access Service (OMAS) has initialized a new instance for server myMetadataServer +Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Governance Server OMAS +Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Governance Server OMAS has started +Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-GOVERNANCE-ENGINE-0001 The Governance Server Open Metadata Access Service (OMAS) is initializing a new server instance +Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-GOVERNANCE-ENGINE-0005 The Governance Server Open Metadata Access Service (OMAS) has initialized a new instance for server myMetadataServer Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Data Privacy OMAS Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Data Privacy OMAS has started Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAS-DATA-PRIVACY-0001 The Data Privacy Open Metadata Access Service (OMAS) is initializing a new server instance @@ -161,7 +161,7 @@ Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAG-ADMIN-0012 25 out of Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0019 An OMRS Topic Connector has registered with an event bus connector for topic myMetadataServer.openmetadata.repositoryservices.enterprise.myMetadataServer.OMRSTopic Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0020 An OMRS Topic Connector is ready to send and receive events on topic myMetadataServer.openmetadata.repositoryservices.enterprise.myMetadataServer.OMRSTopic Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMRS-AUDIT-0015 The listener thread for an OMRS Topic Connector for topic myMetadataServer.openmetadata.repositoryservices.enterprise.myMetadataServer.OMRSTopic has started -Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAG-ADMIN-0004 The myMetadataServer server has successfully completed start up. The following services are running: [Open Metadata Repository Services (OMRS), Connected Asset Services, Open Metadata Store Services, Open Integration Service, Data Manager OMAS, Design Model OMAS, Asset Manager OMAS, IT Infrastructure OMAS, Data Science OMAS, Community Profile OMAS, Digital Architecture OMAS, Asset Owner OMAS, Stewardship Action OMAS, Governance Program OMAS, Digital Service OMAS, Asset Lineage OMAS, Asset Consumer OMAS, Asset Catalog OMAS, DevOps OMAS, Software Developer OMAS, Project Management OMAS, Governance Engine OMAS, Data Privacy OMAS, Security Manager OMAS] +Sun Jun 11 21:19:16 BST 2023 myMetadataServer Startup OMAG-ADMIN-0004 The myMetadataServer server has successfully completed start up. The following services are running: [Open Metadata Repository Services (OMRS), Connected Asset Services, Open Metadata Store Services, Open Integration Service, Data Manager OMAS, Design Model OMAS, Asset Manager OMAS, IT Infrastructure OMAS, Data Science OMAS, Community Profile OMAS, Digital Architecture OMAS, Asset Owner OMAS, Stewardship Action OMAS, Governance Program OMAS, Digital Service OMAS, Asset Lineage OMAS, Asset Consumer OMAS, Asset Catalog OMAS, DevOps OMAS, Software Developer OMAS, Project Management OMAS, Governance Server OMAS, Data Privacy OMAS, Security Manager OMAS] ``` The start-up messages show the [open metadata types](/types) being loaded, followed by the open metadata archive `content-packs/SimpleCatalog.json`. Then the local repository is initialized and finally the OMASs are started.