Separate all packages into their own repos

[ocelotl]

This is somewhat related to #1986.

Instead of having single contrib and core repos, I suggest we have a single Github Organization (probably named opentelemetry-python) which holds all the repos.

This is advantageous because:

1. Does not require running CI for every single package when only one is being changed
2. Keeps the git history of each package strictly related to the package hosted in the repo

Ok, adding emojis for voting 😆

If you agree with separating the contrib packages into their own repos: 👍
If you disagree with separating the contrib packages into their own repos: 👎

[owais]

No emojies for voting? :)

Both have upsides and downsides. Another approach would be to just improve our tooling to make CI faster. I don't think git repo history is a big deal and in fact might be better since people can look at the entire history at once when trying to identify issues. This makes tools like git bisect possible to debug issues. I personally would prefer to continue with a single repo. I think it's nicer when working on features/fixes that touch more than one packages and it's more discoverable for users.

[codeboten]

FWIW, this was discussed at some length across the OpenTelemetry project. In my opinion, the management of N repos makes this difficult for maintainers as well as users.

There are some clear upsides to splitting the packages, but I think some of the concerns around git history for example, can be alleviated by not releasing every package every time which is already being discussed. Additionally CI could be smarter and skip tests for packages that have not changed.

[ocelotl]

In fact, I suggest we split in multiple repos all the components of this project, including opentelemetry-api, opentelemetry-sdk, etc.

[dyladan]

JS split the API into its own repository and we do not regret it

[Oberon00]

Does not require running CI for every single package when only one is being changed

It is possible to add GitHub workflows that trigger depending on the paths touched in a PR.

Also consider what happens when you change the API package: Then you somehow need to manually run/trigger a build for all the separate repositories to check if they still work with the changed API. That setup sounds like it heavily discourages changing of depended-upon repositories. As a consequence it becomes easier to copy-paste some utility function around and change the copy than to change the central function, because even if I believe the change to be backwards-compatible, I have no easy way to verify.

EDIT: Basically, splitting repos up would be the opposite of "Continous Integration", because now the code is not automatically integrated across repositories.

[aabmass]

It is possible to add GitHub workflows that trigger depending on the paths touched in a PR.

I looked into this a bit before. Unfortunately the path matching is at the workflow file level, so we'd have to split each package into its own yaml file https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#onpushpull_requestpaths

[Oberon00]

Keeps the git history of each package strictly related to the package hosted in the repo

Is that a tangible advantage? You can already do git log <directory name>. And if commits touch multiple paths, it's sometimes actually good if it is split up into multiple unrelated commits (PRs) but often you add some artificial split.

[ocelotl]

Well, seems like this idea does not have much support from the rest of the community 🤷 😅

We'll leave the repos as they are now ✌️

[ocelotl]

Sorry to bring this issue back, but I just realized this:

The people you choose as code owners must have write permissions for the repository.

I can imagine it being problematic to have a single contrib repo where lots of people, many of whom could be outside of the current group of regular contributors, have write permissions.

This is related to this issue, @NathanielRN.

[Oberon00]

For contrib, I agree with that. I think external contributions which are not maintained by regular OTel-contributors should be in external repositories, in the organizations it belongs to (not in an repo in the open-telemetry organization). AFAIK there is some policy document in the spec which allows/recommends that.

[ocelotl]

For contrib, I agree with that. I think external contributions which are not maintained by regular OTel-contributors should be in external repositories, in the organizations it belongs to (not in an repo in the open-telemetry organization). AFAIK there is some policy document in the spec which allows/recommends that.

You mean this, @Oberon00 ?