GLOQC: Reduce memory footprint #12871

f3sch · 2024-03-15T11:40:12Z

stop creation of MC histos in Data

github-actions · 2024-03-15T11:40:24Z

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass
async-2023-pp-apass1
async-2022-pp-apass6
async-2022-pp-apass4
async-mc
async-data

f3sch · 2024-03-15T12:10:15Z

Changing to ready to test CI.

chiarazampolli · 2024-03-15T12:39:33Z

Detectors/GlobalTracking/include/GlobalTracking/MatchITSTPCQC.h

-  TH1D* getHisto1OverPtPhysPrimNum(matchType m) const { return m1OverPtPhysPrimNum[m]; }
-  TH1D* getHisto1OverPtPhysPrimDen(matchType m) const { return m1OverPtPhysPrimDen[m]; }
-  TEfficiency* getFractionITSTPCmatchPhysPrim1OverPt(matchType m) const { return mFractionITSTPCmatchPhysPrim1OverPt[m]; }
+  /* do we need these?


what is the problem in keeping them?

No problem at all just wondering if this is deadcode or if there is use case? Anyways I just commented them out to test if something will fail.

alibuild · 2024-03-15T18:01:30Z

Error while checking build/O2/fullCI for e6a4c90 at 2024-03-21 03:34:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-full-system-test-latest/log
task timeout reached .. killing all processes


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/dedbf1a8a490ba64749d6c7e1c8e54e6a5b3fcc6/slc8_x86-64/o2checkcode/1.0-local254/etc/modulefiles
++ cat
--

Full log here.

chiarazampolli · 2024-03-20T10:10:35Z

Before merging, we need to check if TH*F is enough for e.g. an 8h long run in pp and PbPb, @mfaggin , @noferini

chiarazampolli · 2024-03-20T10:53:38Z

Hello,

According to https://docs.google.com/spreadsheets/d/1sXokWFLjyceNhTlTcKL_-CczFZ6Vy2IMeCUJ2ujsTWs/edit#gid=0, the gain will anyway be minimal. @f3sch , why don't we keep all TH*D, and change the rest in this PR?

Chiara

alibuild · 2024-03-22T16:28:09Z

Error while checking build/O2/fullCI for 40e7d6b at 2024-03-22 17:28:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:224:17: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:228:17: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:233:24: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:237:24: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:284:22: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:286:22: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:293:32: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:295:32: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:300:33: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:302:33: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:307:33: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:309:33: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:358:32: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
/sw/SOURCES/O2/12871-slc8_x86-64/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:360:32: error: cannot convert 'TH1F*' to 'TH1D*' in assignment
ninja: build stopped: subcommand failed.

Full log here.

- stop creation of MC histos in Data Signed-off-by: Felix Schlepper <felix.schlepper@cern.ch>

chiarazampolli · 2024-03-25T14:44:52Z

Hello @f3sch ,
Now you kept the TH*D, right?
@shahor02 , can you please check and approve if ok?
Chiara

f3sch · 2024-03-25T15:13:02Z

Hi @chiarazampolli, yes I did revert this part.

chiarazampolli · 2024-03-25T15:28:09Z

Thanks @f3sch !
@shahor02 , the error does not seem related to this PR, what do you think?
Chiara

shahor02

Hi, looks fine, the failure is unrelated.
Eventually, I would substitute all pT axes by 1/pT and use less bins, we still have too many 2D histos with 100x200 bins.
@chiarazampolli I did not understand your reference on https://docs.google.com/spreadsheets/d/1sXokWFLjyceNhTlTcKL_-CczFZ6Vy2IMeCUJ2ujsTWs/edit#gid=0 concerning THD vs THF effect: there should be 50% difference for TH1*, and factor 2 for TH2*, so in latter case the double version should be avoided (as it is in this code).

chiarazampolli · 2024-03-25T18:07:44Z

Hello @shahor02 ,

We also have QC vs 1/pt already, but not 2D.

For the THD vs THF, Felix had changed from THD to THF in this PR. We then were not sure since we might hit the limit of precision of THF... So we preferred to keep THD, considering also that the gain in memory did not look so big according to Felix's expectation (written in the spreadsheet).

Does this clarify?

cheers,
Chiara

shahor02 · 2024-03-25T18:16:55Z

this was actually my question, in the spreadsheet I did not see anything about THxD vs THxF...

f3sch marked this pull request as ready for review March 15, 2024 12:10

f3sch requested a review from shahor02 as a code owner March 15, 2024 12:10

chiarazampolli reviewed Mar 15, 2024

View reviewed changes

f3sch force-pushed the qc/memred branch 3 times, most recently from 220a4bb to 40e7d6b Compare March 22, 2024 14:58

GLOQC: Reduce memory footprint

5ac9e1f

- stop creation of MC histos in Data Signed-off-by: Felix Schlepper <felix.schlepper@cern.ch>

f3sch force-pushed the qc/memred branch from 40e7d6b to 5ac9e1f Compare March 23, 2024 08:02

shahor02 approved these changes Mar 25, 2024

View reviewed changes

shahor02 merged commit 9ad1144 into AliceO2Group:dev Mar 25, 2024
12 of 13 checks passed

f3sch deleted the qc/memred branch November 25, 2024 10:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GLOQC: Reduce memory footprint #12871

GLOQC: Reduce memory footprint #12871

f3sch commented Mar 15, 2024 •

edited

Loading

github-actions bot commented Mar 15, 2024

f3sch commented Mar 15, 2024

chiarazampolli Mar 15, 2024

f3sch Mar 15, 2024 •

edited

Loading

alibuild commented Mar 15, 2024 •

edited

Loading

chiarazampolli commented Mar 20, 2024

chiarazampolli commented Mar 20, 2024

alibuild commented Mar 22, 2024

chiarazampolli commented Mar 25, 2024

f3sch commented Mar 25, 2024

chiarazampolli commented Mar 25, 2024

shahor02 left a comment

chiarazampolli commented Mar 25, 2024

shahor02 commented Mar 25, 2024

GLOQC: Reduce memory footprint #12871

GLOQC: Reduce memory footprint #12871

Conversation

f3sch commented Mar 15, 2024 • edited Loading

github-actions bot commented Mar 15, 2024

f3sch commented Mar 15, 2024

chiarazampolli Mar 15, 2024

Choose a reason for hiding this comment

f3sch Mar 15, 2024 • edited Loading

Choose a reason for hiding this comment

alibuild commented Mar 15, 2024 • edited Loading

chiarazampolli commented Mar 20, 2024

chiarazampolli commented Mar 20, 2024

alibuild commented Mar 22, 2024

chiarazampolli commented Mar 25, 2024

f3sch commented Mar 25, 2024

chiarazampolli commented Mar 25, 2024

shahor02 left a comment

Choose a reason for hiding this comment

chiarazampolli commented Mar 25, 2024

shahor02 commented Mar 25, 2024

f3sch commented Mar 15, 2024 •

edited

Loading

f3sch Mar 15, 2024 •

edited

Loading

alibuild commented Mar 15, 2024 •

edited

Loading