What is the correct way to extract pedestal dispersion from a set of events?

[cbadams2]

I may be missing something, but I've been struggling to find clear directions for how best to populate the PedestalContainer for an event - specifically the charge_std field. Of course, this is a population-wise number, so it needs to be generated from a number of samples. My naive attempt was to do:

ped_calculator = PedestalIntegrator(
        subarray=source.subarray,
        charge_product="FixedWindowSum",
        sample_size=10000,
        tel_id=2,
    )

source = SimTelEventSource(filename, focal_length_choice='EQUIVALENT', max_events=10001)
for event in source:
    calib = CameraCalibrator(subarray=source.subarray)
    calib(event)
    ped_calculator.calculate_pedestals(event)

but I immediately encounter an error with the following (truncated) traceback.

---------------------------------------------------------------------------
AxisError                                 Traceback (most recent call last)
Cell In[154], line 5
      3 calib = CameraCalibrator(subarray=source.subarray)
      4 calib(event)
----> 5 ped_calculator.calculate_pedestals(event)
      6 print(event)

File ~/anaconda3/envs/ctapipe/lib/python3.12/site-packages/ctapipe/calib/camera/pedestals.py:264, in PedestalIntegrator.calculate_pedestals(self, event)
    261 if not dl1.is_valid:
    262     return False
--> 264 self.collect_sample(dl1.image, pixel_mask)
    266 sample_age = (trigger_time - self.time_start).to_value(u.s)
    268 # check if to create a calibration event

File ~/anaconda3/envs/ctapipe/lib/python3.12/site-packages/ctapipe/calib/camera/pedestals.py:303, in PedestalIntegrator.collect_sample(self, charge, pixel_mask)
    300 """Collect the sample data"""
    302 good_charge = np.ma.array(charge, mask=pixel_mask)
--> 303 charge_median = np.ma.median(good_charge, axis=1)
    305 self.charges[self.n_events_seen] = charge
    306 self.sample_masked_pixels[self.n_events_seen] = pixel_mask

This is unsurprising to me, because the shape of the dl1 image I'm working with will be (11328,), so won't have the dimension corresponding to axis=1.

Am I doing this at all right? Is this a bug? Any help would be appreciated.

[maxnoe]

Hi @cbadams2 ,

Thanks for your interest in using the calibration code in ctapipe.

Before I answer what you need to do to fix the exception you mentioned, let me say that this part of the code is relatively old, mostly untested and has significantly changed downstream in the only place we know it has been used, the calibration code for LST-1.

It is currently being rewritten, and we are making progress in doing so, see recent PRs:

#2554, #2604, #2609 and the discussions there-in.

The final step of actually computing the calibration coefficients will probably not be in ctapipe itself, at least not for some time, but in the codebase of the CTAO calibpipe, which is under development here:
https://gitlab.cta-observatory.org/cta-computing/dpps/calibrationpipeline/calibpipe

That being said, you encountered a bug that I think stems from recent changes of the expected waveform shape we made in #2529

So I'll transfer this from discussions to issues.