Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dual-readout tubes barrel calorimeter in IDEA_o2 #413

Merged
merged 27 commits into from
Jan 23, 2025

Conversation

lopezzot
Copy link
Member

@lopezzot lopezzot commented Dec 9, 2024

BEGINRELEASENOTES

  • A new subdetector (DRBarrelTubes_o1_v01) is included. It is the dual-readout barrel calorimeter built exploiting the Hidra-like capillary-tubes-based technology. Together with the endcap just merged, it completes the hadronic calorimeter of the IDEA_o2 concept.
  • The IDEA_o2 muon system dimensions are adjusted to avoid overlaps with the barrel dual-readout calorimeter: (BarrelFirstLayerRadius=4630mm, BarrelLength=9260mm, EndcapFirstLayerZOffset=4630mm, EndcapLayersOuterRadius=5450mm)
  • The custom DRTubes sensitive detector action was extended to be used in both the endcap (DREndcapTubes) and the barrel (DRBarrelTubes) subdetectors.
    ENDRELEASENOTES

Hello,

this PR includes a new subdetector, the IDEA dual-readout capillary-tube barrel calorimeter. Together with the endcap calorimeter, it completes the hadronic calorimeter of IDEA_o2. The muon system dimensions have been adjusted to avoid overlaps with this subdetector. The only missing part for IDEA_o2 full-sim is now the crystal em calorimeter (#406 ). DRTubesSDAction is also expanded to produce hits from the barrel calorimeter.

Testing:

  • The memory footprint on lxplus9 for the entire IDEA_o2 is close to 3.8 Gb.
  • 10 GeV electron events showering either in the barrel or the endcap calorimeters are processed at an event rate smaller than 1 s/evt.
  • No overlaps are found when scanned with a tolerance of 50 um. However scanning over millions or tubes takes forever, so the overlap check was performed over calo towers only (without tubes), and each tower (with tubes) was tested separately for overlaps.

Authors:
Geometry by Andreas Loeschcke Centeno, custom sensitive detector action, integration in IDEA_o2 and testing by @lopezzot

idea_o2_hcal

FYI @mahmoudali2

@atolosadelgado
Copy link
Collaborator

Hi @lopezzot

Thank you so much for your contribution.

While running some events, I got the following error, could you please crosscheck?

Geant4Kernel     FATAL +++ Exception while simulating: BitFieldElement 'tower': out of range : 130 for width 8
DDSim            ERROR Simulation failed!

@lopezzot
Copy link
Member Author

Geant4Kernel     FATAL +++ Exception while simulating: BitFieldElement 'tower': out of range : 130 for width 8
DDSim            ERROR Simulation failed!

Hi @atolosadelgado, thanks for spotting this. I did not test it before with an isotropic source so I could not find this. It was an issue with hits at the border between the barrel and the endcap calo. The last commit fixes it. I also rebased.

Let me know if you have other tests in mind we can run.

(Tagging @s6anloes for his information.)

@atolosadelgado
Copy link
Collaborator

Hi @lopezzot

Thanks for the quick fix. I have run several simulations and Geant4 does not seem to complain. I have not inspected the output, I trust you.

I would advise to remove the test named t_test_IDEA_o2_v01: a single test should require less than 10 minutes (if I am not mistaken), but IDEA_o2_v01 needs 18 minutes just to start a simulation.

I noticed that the memory consumption increases over time during the simulation, but this may be related to other issues (hopefully not a memory leak). For reference, when running few events (<100) is 3.8GB (RSS) or 4GB (VSZ), but it goes up to 5.5GB (RSS) when running more events (~2000)

The average time per event, for when using particle gun of 1 electron at 10 GeV, is 1 second (seems to grow linearly with the number of events). The output file looks reasonable (13MB for 1000 events).

Otherwise, I do not see major problems for merging.

lopezzot and others added 26 commits January 23, 2025 11:48
Co-authored-by: Andreas Loeschcke Centeno <andreas.loeschcke.centeno@cern.ch>
Add dual-readout capillary-tubes barrel calorimeter inside the IDEA_o2
xml file. Change IDEA_o2 muon system dimensions to avoid overlaps with
the barrel calorimeter (new params checked with Mahmoud Ali).
Move the detector dimensions parameters inside
DectDimensions_IDEA_o2_v01.xml.
Move the vis attributes for DRBarrelTubes subdetector to
DectDimensions_IDEA_o2_v01.xml.
Increase DRTubesGranularity to 72 rotations aroung the beam pipe (as the
endcap).
Make explicit printout from DRTubesconstructor.
Rename DDDRCaloTubes subdetector for consistency with k4geo standards.
Displace the barrel inner radius by 0.5 cm to avoid overlaps between the
calo barrel staves and the solenoid.
Materials names defined in DDDRCaloTubes_o1_v01.xml where already
defined in the endcap xml file, therefore the barrel geometry was using
the endcap materials. This is a problem in case one wants to
activate/inactivate optical properties for such materials in the endcap
or barrel separately. Fixed by changing names of DDDRCaloTubes
materials. Also, set particle direction to (0,1,0) in the IDEA_o2_vo1.py
file to shoot into the barrel instead of the endcap.
Set the subdetector ID in DectDimensions_IDEA_o2_v01.xml file and
assign.
Add description of the barrel dual-readout-tubes caloriemter in
detectors/calorimeter README.md file.
By adding inclusion of the dual-readout-tubes barrel calorimeter in
Decemeber 2024.
First attempt to produce signals from the barrel dual-readout-tubes
calorimeter.
The theta coordinate was taken from the track momentum direction,
instead is should be taken from the G4Step position to discriminate if a
G4Step is inside the barrel or the endcap geometry.
Change the name of hit collections to avoid duplication of hits while
using ROName in the hit collection name. The barrel was creating also
hit collection for cher_right/left and scin_right/left which are only
for the endcap.
Reinclude optical properties for fiber materials. Create PMMA_Scin
material to differentiate between material used in Scin and Cher fibers
in order to not have optical photons in Scin fibers but only in Cher
fibers.
Remove printout and apply clang-format. I checked that this version of
the custom SD action is consistent with the speedup obtained originally
with the endcap geometry only (about 1 s/evt for 10 GeV e- events).
Fix printout to check that volIDs of the DRBarrelTubes are correctly
reproduced.
Commet out volID printout check for barrel calo in DRTubesSDAction. This
was used to check that the 64-bits volIDs recreated from the barrel
copynumbers are identical to the DD4hep ones. However marking barrel
fibers as sensitive makes the memory footprint explode (34 Gb before it
is killed on lxplus). So the test was done on a single stave.
Change logic to differentiate between barrel and endcap calo in
DRTubesSDAction. Instead of using theta from global coordinates use copy
numbers.
@lopezzot
Copy link
Member Author

Hi again,

I would advise to remove the test named t_test_IDEA_o2_v01: a single test should require less than 10 minutes (if I am not mistaken), but IDEA_o2_v01 needs 18 minutes just to start a simulation.

Done and rebased.

I noticed that the memory consumption increases over time during the simulation, but this may be related to other issues (hopefully not a memory leak). For reference, when running few events (<100) is 3.8GB (RSS) or 4GB (VSZ), but it goes up to 5.5GB (RSS) when running more events (~2000)

I just did a run with 5k events and observed that the memory (RES) taken was 5.5 Gb already at the very first events, and it was stable for the whole run. However, I also observed 3.8 Gb when I opened this PR in December. I do not know what is causing this.
image

The average time per event, for when using particle gun of 1 electron at 10 GeV, is 1 second (seems to grow linearly with the number of events). The output file looks reasonable (13MB for 1000 events).

I got slightly better numbers averaged over a run with 5k events: 0.69 s/evt. Would be nice to understand what is causing this shift. Anyhow, both numbers are good.
image

Otherwise, I do not see major problems for merging.

Thanks! should we proceed or you want more tests?

@atolosadelgado
Copy link
Collaborator

I have run some more events (10k, proton at 10GeV), and I agree with @lopezzot results. The memory slowly increases, and there are jumps, but only up to 5.6GB.

image

I do not see any error/warning or exception after running with different particles/energies, so I merge, thank you!

@s6anloes
Copy link
Contributor

I'm also seeing the same, testing with 5k electron events.
image

@lopezzot
Copy link
Member Author

Thank you for your tests and for merging. :)

@atolosadelgado atolosadelgado merged commit 915923d into key4hep:main Jan 23, 2025
6 checks passed
@lopezzot lopezzot deleted the lp-ideao2drbarrel branch January 23, 2025 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants