Self-inconsistent results with fermipy gta.extension()

[alex-mcdaniel]

EDIT: Update 2/6/2024

It appears that this issue can be avoided by directly setting the width_min and width_max parameters to the desired values, in addition to the width parameters. However according to the documentation, setting only the width argument should supersede the others (e.g. width_min, width_max, width_nstep), so there may be some bug in the extension fitting code.

---

Hello, I have encountered an issue when using the gta.extension() method. The issue is that in several instances we are finding the reported best-fit width and TS values are not consistent with the likelihood profile.

The documentation states that extension fitting procedure as “The best-fit extension is found by performing a likelihood profile scan over the source width (68% containment) and fitting for the extension that maximizes the model likelihood.” and in the source code ”This method will substitute an extended spatial model for the given source and perform a one-dimensional scan of the spatial extension parameter over the range specified with the width parameters. The 1-D profile likelihood is then used to compute the best-fit value, upper limit, and TS for extension.” In our analysis we are often finding not only that the actual best fit values and the profile maximum values dont match, but also the reported best-fit values are often not even included in the widths used in the scan (i.e. the values supplied with the “width” keyword).

As shown in the examples below there seems to be some problem with the gta.extencsion() results. The plots show the likelihood profiles from fermipy’s gta.extension() method. The maximum of this profile is shown with the blue vertical line. The red line shows the reported extension from gta.extension() (accessed with the ‘ext’ keyword), and the red shaded region shows the corresponding error. In the examples shown below we have backgrounds and source normalizations freed, and position fixed. However, we tested with various configurations of freeing and fixing the various parameters and still consistently seem to be finding this error. This analysis is using fermipy 1.2, fermitools v2.2.0 and fermitools-data 0.18, although using earlier fermipy versions still produces the error.

It also appears that when there is disagreement between the profile and the reported value, the reported width value tends to be near ~1 degree.

For comparison we have also written a script to calculate the extension and likelihood profile separate from gta.extension(). This just simply scans through width values, updates the source morphology, performs the fit, and records the likelihood. Doing this yielded similar profile shapes to the likelihood profile generated by gta.extension().

If there is any insight available as to what is the cause of this it would be greatly appreciated.

[jballet]

Hi Alex,
Does this happen with both disk and Gaussian models?

[alex-mcdaniel]

Hey Jean,

Yes it occurs with either model. The issue did seem to go away when setting the width_min and width_max parameters directly (in addition to the width parameter).

[omodei]

I think this is related to the fact that in the default configuration width_max is 1 degree (https://fermipy.readthedocs.io/en/latest/config.html). width is an array of points, if not provided it will be built from width_min and width_max: https://github.com/fermiPy/fermipy/blob/master/fermipy/extension.py#L168
It's true that width_min and width_max will not be overwritten by the extremes of width, so the scan will still be up to width_max. I think we should fix this, but I want to make some test to make sure I don't brake anything. Until it is committed, yes, specifying width_min and width_max is the way to go! Thanks for reporting this.