Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Official EventBuilding ToolChain #253

Merged
merged 16 commits into from
Feb 6, 2024
Merged

Conversation

S81D
Copy link
Contributor

@S81D S81D commented Oct 19, 2023

see commit info, but here is a brief description of the changes (mostly changes to DataDecoder and other toolchains' config files)

  • Created a new toolchain that can serve as our official event builder moving forward
  • Added Johann's saveinfo tool to the eventbuilding toolchain to save git commit hash and config files
  • Included Gian's new channel gains info (October 18th, 2023) in the LoadGeometry toolchain
  • Updated some of the file paths to include newer data sets
  • Added the SQL run information that the BeamFetcher tool uses for fetching beam info from the IF database directly into the toolchain, so no need to call the one that's in Michael's toolanalysis directory on the gpvms
  • Excluded Stage1Data (and LAPPD data) until bug fix with that tool (including it leads to seg fault errors - in talks with Rory to fix it)

To do:

  • Incorporate Andrew's beamfetcherV2 tool within the event building toolchain, which will require some tweaks to ANNIEEventBuilder in order to properly store additional beam device information. Also, need to create an additional beamdb-like output file when running BeamFetcherv2 in order to run jobs on the grid (grid node cannot access IF beam database). Currently there is a .root file that can be outputted - need to either change this to a similar datatype as the BeamFetcher tool output or modify the BeamDecoder tool to load in .root files
  • Figure out what to do with the LAPPD information (either add Rory's tool once that's fixed or make modifications to the event building software to allign LAPPD timestamps now that the time drift problem has been largely resolved).

S81D added 13 commits October 19, 2023 13:14
EventBuilder ToolChain - very similar to DataDecoder toolchain, with some minor configuration differences:
- Updated trigger mask to include all triggers (1-64)
- Updated LoadGeometry file path for Gian's new gain calibration data
- Added Johann's SaveConfigInfo tool
- Excluded Stage1DataBuilder until bug fix
- Minor fix in ANNIEEventBuilderConfig (changing true/false to 1/0)
- Updated file paths for more recent data
Part of updated the BeamFetcher toolchain (needed prior to event building)
Included a copy of the run info from the SQL webpage in the local toolanalysis directory
added local SQL run information for fetching beam info
updated path for trigger mask to include all triggers
added new paths for event building
Updated gains file path within the LoadGeometryConfig file in the LoadGeometry toolchain - many tools call this config file (configfiles/LoadGeometry), instead of calling their own LoadGeometryConfig file that is present within that specific toolchain. Instead of replacing every path within all config files, replace this one.
After testing the new EventBuilder toolchain, including all triggers in the triggermask file seems to slow the toolchain down by 2x (or so). Going back to default file.
Added necessary trigwords for AmBe external trigger (15) and laser (46 or 47, dependent on settings. Through the current laser run campaign, various configurations will either lead to the trigword being 46 or 47... not sure why).
For now, omit BeamDecoder until we have BeamFetcherv2 up and running
Include the undelayed beam trigger for retroactive LAPPD time matching with other subsystems. If there is an LAPPD event, this triggerword can be used to check if the LAPPD event is correctly aligned, in case any hardware was broken (according to Yue)
I swear this is the final change. Yue initially requested I put this trigword in, however he recently encountered errors when trying to pair the LAPPDs to the MRD. For now, omitting this trigword until he can find a suitable trigword for LAPPD timing alignment
@marc1uk
Copy link
Collaborator

marc1uk commented Dec 5, 2023

Sweet! These all sound like great changes. And CI seems to have run successfully for this one, which is also good news. 🎉

I see here the dreaded words DaylightSavings and DaylightSavingsSpring .... I thought in one of your previous PRs we removed dependence on DST timestamps? What is this related to? Is there a non-DST alternative?
As a worst case scenario it might be nice to look at implementing DST correction within the code; if we know the timestamp has a DST offset, and assuming we have a corresponding date too, we should be able to determine what the offset required is.

very minor point, but technically comments should be marked with # rather than //. This probably works because the Store parser will read the first non-whitespace word as a key and the second as a value and then stop there, but lines "commented out" with a preceding // will not be ignored, and it's probably misleading to have inconsistent comment styling for leading and trailing comments.

Another minor comment is that it's generally best not to commit high verbosity levels, as excessive printouts can slow down processing considerably.

As for the outstanding issues, I won't let them hold this up, but we have an update on those?
@rory42edwards have you fixed your segfault? I think considering that the Tool was written, it would be good to have that merged, even if it immediately gets replaced. It's nice to have the progression and it there as a fallback in case of future issues.

Lowering the verbosity
@S81D
Copy link
Contributor Author

S81D commented Dec 5, 2023

Thanks for the comments - I will talk to Yue and Rory about the LAPPD related action items (for Yue, adding tools to do the time matching, for Rory fixing the segfault). As for DST, I can add a separate PR modifying the BeamFetcherConfig file to remove it (#233 , since the tool was modified to remove dependence), and look into altering the MRDDataDecoder tool to no longer rely on DST.

Only question I have is that I was under the impression that the MRD records timestamps in CDT or some other daylight savings dependent time zone, so if we are trying to pair events with the CTC or Tank, how do we modify this correctly? For the old beamfetcher changes to fix DST dependencies, the SQL times used to be in local chicago time, and the tool then adjusted that time into UTC. SQL was then changed to UTC, and the shift was removed from the tool. If the MRD records timestamps in local Chicago time, it appears there is no reference for the MRD tool to use to convert to UTC, correct? Unless I'm missing something simple

S81D and others added 2 commits December 6, 2023 11:08
Since we are omitting the beamfetcher tool for now, do not store the beam information in the final annieevent
change config file comments to use `#` instead of `//`
@marc1uk marc1uk merged commit 062a331 into ANNIEsoft:Application Feb 6, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants