Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variance calibration #2636

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/changes/2636.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Update ``CameraCalibrator`` in ``ctapipe.calib.camera.calibrator`` allowing it to correctly calibrate variance images generated with the ``VarianceExtractor``.
- If the VarianceExtractor is used for the ``CameraCalibrator`` the element-wise square of the relative and absolute gain calibration factors are applied to the image;
- For other image extractors the plain factors are still applied.
18 changes: 15 additions & 3 deletions src/ctapipe/calib/camera/calibrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,11 @@ def _calibrate_dl1(self, event, tel_id):
)

# correct non-integer remainder of the shift if given
if self.apply_peak_time_shift.tel[tel_id] and remaining_shift is not None:
if (
self.apply_peak_time_shift.tel[tel_id]
and remaining_shift is not None
and dl1.peak_time is not None
):
dl1.peak_time -= remaining_shift

# Calibrate extracted charge
Expand All @@ -292,13 +296,21 @@ def _calibrate_dl1(self, event, tel_id):
and dl1_calib.absolute_factor is not None
):
if selected_gain_channel is None:
Copy link
Member

@maxnoe maxnoe Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section could use some refactoring to make it easier to follow.

I think the logic is the same if we simplify to:

if selected_gain_channel is None:
    calibration = dl1_calib.relative_factor / dl1_calib.absolute_factor
else:
    calibration = (
        dl1_calib.relative_factor[selected_gain_channel, pixel_index]
         / dl1_calib.absolute_factor[selected_gain_channel, pixel_index]
    )

if isinstance(extractor, VarianceExtractor):
    calibration = calibration**2

dl1.image *= calibration

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I'll get to it when I arrive home

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i put it in

dl1.image *= dl1_calib.relative_factor / dl1_calib.absolute_factor
if extractor.__class__.__name__ == "VarianceExtractor":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if extractor.__class__.__name__ == "VarianceExtractor":
if isinstance(extractor, VarianceExtractor):

dl1.image *= np.square(
dl1_calib.relative_factor / dl1_calib.absolute_factor
)
else:
dl1.image *= dl1_calib.relative_factor / dl1_calib.absolute_factor
else:
corr = (
dl1_calib.relative_factor[selected_gain_channel, pixel_index]
/ dl1_calib.absolute_factor[selected_gain_channel, pixel_index]
)
dl1.image *= corr
if extractor.__class__.__name__ == "VarianceExtractor":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use this to check type, not a string comparison (which is slower and makes refactoring more difficult later)

Suggested change
if extractor.__class__.__name__ == "VarianceExtractor":
if isinstance(extractor, VarianceExtractor):

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, im changing it now

dl1.image *= np.square(corr)
else:
dl1.image *= corr

# handle invalid pixels
if self.invalid_pixel_handler is not None:
Expand Down
61 changes: 61 additions & 0 deletions src/ctapipe/calib/camera/tests/test_calibrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
GlobalPeakWindowSum,
LocalPeakWindowSum,
NeighborPeakWindowSum,
VarianceExtractor,
)
from ctapipe.image.reducer import NullDataVolumeReducer, TailCutsDataVolumeReducer

Expand Down Expand Up @@ -130,6 +131,66 @@ def test_check_dl0_empty(example_event, example_subarray):
assert (event.dl1.tel[tel_id].image == 2).all()


def test_dl1_variance_calib(example_event, example_subarray):
# test the calibration of variance images
tel_id = list(example_event.r0.tel)[0]
calibrator = CameraCalibrator(
subarray=example_subarray,
image_extractor=VarianceExtractor(subarray=example_subarray),
apply_waveform_time_shift=False,
)
calibrator(example_event)
image = example_event.dl1.tel[tel_id].image
assert image is not None
assert image.shape == (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test with the LST (two gains)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to add a test for other extractor with the LST?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use a parametrized test here to test all telescope types. (@pytest.mark.parametrized). See for example the code in ctapipe/instrument/tests/test_telescope.py

1,
1764,
)


def test_calib_LST_camera(example_subarray):
n_channels = 2
n_pixels = 1855 # number of pixels in LSTcam
n_samples = 100

random = np.random.default_rng(1)

tel_id = 1
y = random.normal(0, 6, (n_channels, n_pixels, n_samples))

absolute = random.uniform(100, 1000, (n_channels, n_pixels)).astype("float32")
y *= absolute[..., np.newaxis]

relative = random.normal(1, 0.01, (n_channels, n_pixels))
y /= relative[..., np.newaxis]

pedestal = random.uniform(-4, 4, (n_channels, n_pixels))
y += pedestal[..., np.newaxis]

event = ArrayEventContainer()
event.dl0.tel[tel_id].waveform = y
event.calibration.tel[tel_id].dl1.pedestal_offset = pedestal
event.calibration.tel[tel_id].dl1.absolute_factor = absolute
event.calibration.tel[tel_id].dl1.relative_factor = relative
event.dl0.tel[tel_id].selected_gain_channel = None
event.r1.tel[tel_id].selected_gain_channel = None

calibrator = CameraCalibrator(
subarray=example_subarray,
image_extractor=VarianceExtractor(subarray=example_subarray),
apply_waveform_time_shift=False,
)
calibrator(event)

image = event.dl1.tel[tel_id].image

assert image is not None
assert image.shape == (
2,
1855,
)


def test_dl1_charge_calib(example_subarray):
# copy because we mutate the camera, should not affect other tests
subarray = deepcopy(example_subarray)
Expand Down