Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: Proposal for FederatedCatalog Distribution and TargetNodeDirectory #1555

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

docs: Proposal for FederatedCatalog Distribution and TargetNodeDirectory #1555

Changes from 4 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
8b7775f
Doc: Proposal for FederatedCatalog distribution.
bmg13 Sep 16, 2024
40f9a94
Restructure based on PR.
bmg13 Sep 24, 2024
a452459
Changes from PR.
bmg13 Sep 26, 2024
960a5f9
Small doc improvements
bmg13 Sep 27, 2024
48b1900
Changes from PR.
bmg13 Sep 30, 2024
4247446
Merge both Distribution and TargetNodeDirectory decision proposals.
bmg13 Sep 30, 2024
c50c2c9
Merge branch 'main' of https://github.com/eclipse-tractusx/tractusx-e…
bmg13 Oct 7, 2024
b851b92
Changes from PR.
bmg13 Oct 7, 2024
b4b00b0
Changes from PR.
bmg13 Oct 7, 2024
473e800
Update retrieve path.
bmg13 Oct 8, 2024
b3c8ce2
Changes from PR.
bmg13 Oct 14, 2024
686dcee
Merge branch 'main' of https://github.com/eclipse-tractusx/tractusx-e…
bmg13 Oct 14, 2024
800dbc1
Changes from PR.
bmg13 Oct 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions ...t/decision-records/2024-09-16-proposal-federated-catalog-distribution/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Proposal for FederatedCatalog with Tractus-X distribution

## Decision

The proposed solution is to have a Federated Catalog as own runnable service capable of crawling all the chosen catalogs and expose that data.
Copy link
Contributor

@paullatzelsperger paullatzelsperger Sep 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be an unequivocal decision, not a proposal. It is a "decision record", not a "proposal record". words matter.
Also, what is a "runnable service"? We should be very explicit here, for example, in saying: "The Federated Catalog will be deployed as standalone component. For this, the Helm charts will be updated to feature a new deployment template called 'federated-catalog'".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, updated here.


## Rationale

The Federated Catalog aims at providing the aggregation of catalogs from multiple participants in a centralized point to reduce latency.
Copy link
Contributor

@paullatzelsperger paullatzelsperger Sep 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not needed. No need to motivate the existence of the FC, only the decision to deploy it as standalone component.
It also makes sense here to highlight some of the disadvantages (increased configuration complexity, more remote calls,...) and why they are still justified (scalability, etc.).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in here.


Choosing a solution that decouples it from the Control Plane (like the one used for the Data Plane) and able to be scalable will future-proof the Federated Catalog as a feature and embraces wider usage.

## Approach

The Federated Catalog running as own instance can be performed either by running respective Docker image in a container or directly running a generated `jar`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no theoretical concepts needed. Please only highlight how it is going to be deployed in the Tractus-X EDC helm charts, i.e. as separate deployment template. Doing this will bring some increased complexity, for example we need an additional Kubernetes ingress definition, as opposed to: just an additional path in an existing ingress.

Here it would also be advisable to mention, that there is going to be an additional runtime (i.e. docker image), analogous to edc-controlplane-XYZ and edc-dataplane-XYZ and to list the required variants (in-mem, postgres persistence).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, changed in here.


> In case of aiming at testing locally, the [Readme.md](https://github.com/eclipse-edc/FederatedCatalog/blob/main/README.md) includes a very helpful guide.

Regarding deployment, the Tractus-Connector Helm charts will be updated to include this service.

Relevant to highlight a potential challenge of the proposed approach.
- Having its own instance (that itself contains a cache) may use considerable computing resources (related with storage cost).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use considerable computing resources (related with storage cost).

how so? what cost do you anticipate, that wouldn't also be there if the FC ran embedded in the ControlPlane?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. The idea was comparing a specific runtime vs embedded in control plane could incur on having more resources used, but should be a minimal difference and I do not have comparing numbers.
Regarding cost, is a non issue, since same data would be store regardless of solution.
Removed here.


To enable the Federated Catalog flow, please [see this table](https://github.com/eclipse-tractusx/tractusx-edc/blob/75bdacbad43e2cad352204ea28a359c6aac7adea/docs/development/management-domains/README.md#enable-and-configure-the-crawler-subsystem).
Loading