shower_distribution not propagated by HDF5EventSource

[kosack]

Describe the bug

The /simulation/service/shower_distribution histograms are written if you read a simtel file with the SimTelEventSource, but further processing of the HDF5 file by the HDF5EventSource drops this information. This is not critical, as one can use the configuration/simulation/run to compute the spectrum of simulated events, however that may be insufficient in the future if more complex spectra are used.

To Reproduce
Steps to reproduce the behavior:

1. process an existing ctapipe output file with ctapipe-process to a higher level

Expected behavior

Simulated shower distributions end up in output files.

Supporting information

This is not a problem for the HDF5Merger, which does propagate this information, so merging and applying models works fine. it is only an issue if DL1 files are further processed into DL2 by ctapipe-process.

We need a unit test that does:

• simulated-R0 → DL1 → file
• file → DL2
• checks the data model against the output (note this last step probably is more general: we need a ctapipe-verify command )

[maxnoe]

Confirmed. We have no general API on the EventSource and the DataWriter explicitly checks for SimTelEventSource. So no other event source can even try to provide the histograms.

Reading the comments in the DataWriter method, this is a known issue... but we have no corresponding issue here.

Will fix, this is definitely a bug and directly impacts our processing of prod5/prod6 on the GRID, since we recently switched to a setup where we first run ctapipe-process to extract only images for all telescopes and then again ctapipe-process to perform parametrization and stereo-reco for a selected subarray. So we loose the histograms here in the second step.