OpenTelemetry Protocol with Apache Arrow in Production

[jmacd]

Fixes #5193.

[tiffany76]

First pass edits - had to stop around line 92. Will do more next week! Thanks.

[svrnm]

Thanks for writing this, great read. My main question is around "OTAP" vs "OTel-Arrow", are those 2 things different? So is it "OTel-Arrow Receiver" or "OTAP Receiver" like "OTLP receiver".

Also, a question on where this content should live: there is a lot of practical material in the content, that looks more like docs dann a blog. Is OTel Arrow ready for production such that we can take some of that into the documentation (e.g. within the collector docs)?

[jmacd]

@svrnm About the length --
I have cut the section "Batching and back-pressure in OTel-Arrow streams", moved it to the exporter documentation: open-telemetry/opentelemetry-collector-contrib#35225

[lquerel]

LGTM. Thanks Joshua

[svrnm]

lgtm overall, most comments are suggestions. There is only one around the "collaboration with F5" wording, see my inline comment

[jmacd]

I don't remember changing this file. Is it OK?

[tiffany76]

Hi @jmacd, thanks for your patience. Most of our maintainers are out of the office at the moment. Once they're back, we'll work to get this blog post merged ASAP.

As for the submodule file, it should not be included with the PR. If you're able to remove the changes locally, please go ahead. If that's not possible, the docs maintainers will know how to handle the unintended changes.

[svrnm]

As for the submodule file, it should not be included with the PR. If you're able to remove the changes locally, please go ahead. If that's not possible, the docs maintainers will know how to handle the unintended changes.

You should be able to fix this by npm run sync locally, but we can also fix it here before merging.

[jmacd]

npm run sync appeared to work.
npm run check locally has some errors for me, like

tmp/bin/htmltest --log-level 1
htmltest started at 09:18:56 on public
========================================================================
running in concurrent mode, this is experimental
ja/docs/concepts/sampling/index.html
  target does not exist --- ja/docs/concepts/sampling/index.html --> /docs/specs/otel/trace/tracestate-probability-sampling-experimental/#consistent-probability-sampling

[svrnm]

npm run sync appeared to work.

Perfect.

npm run check locally has some errors for me, like

That's unrleated to your PR unfortunately. We have turned on merge queues recently and apparently didn't set it up properly, so CI is running but not failing and still merging PRs with errors. Ignore it for now.

[svrnm]

/fix:all

[opentelemetrybot]

You triggered fix:all action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11020834246

[opentelemetrybot]

fix:all failed or was cancelled. For details, see https://github.com/open-telemetry/opentelemetry.io/actions/runs/11020834246.

[svrnm]

/fix:submodules

[opentelemetrybot]

You triggered fix:submodules action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11020902472

[opentelemetrybot]

fix:submodules failed or was cancelled. For details, see https://github.com/open-telemetry/opentelemetry.io/actions/runs/11020902472.

[svrnm]

@jmacd I fixed the issue with the git submodules. It took me a few tries, so apologies for the commit & message pollution.

Eventually the solution was:

• merge with main
• run npm run sync locally to have the git submodules updated
• commit that change (!), otherwise in the next step this will be reverted once again
• run npm fix:all to apply any other missing changes / verify that all checks are green