This repository contains utilities to prepare the ECAL TPG weights configuration of the online and offline CMS world. The structure of the weights configuration is the same for Even and Odd weights.
For a general introduction to the ECAL trigger primitive generation have a look at Davide's thesis.
Once a weight configuration is the ConfDB the script query_weight_config.py can be used to query the TPmode, the WeightGroups and
WeightIdMap from a specific version. The script creates the txt files in the standard configuration to be used for the
ECALTPGParamBuilder or to build custom CondDB tags.
python query_weight_config.py -h
usage: query_weight_config.py [-h] -w WEIGHT_CONFIG -wt WEIGHT_TYPE
[-l LOGICID] -o OUTPUTNAME -p PASSWORD
optional arguments:
-h, --help show this help message and exit
-w WEIGHT_CONFIG, --weight-config WEIGHT_CONFIG
Weight config id
-wt WEIGHT_TYPE, --weight-type WEIGHT_TYPE
Weight type [even/odd]
-l LOGICID, --logicid LOGICID
Logicid mapping
-o OUTPUTNAME, --outputname OUTPUTNAME
Name for the output files
-p PASSWORD, --password PASSWORD
ConfDB password
For example to query a specific odd weights configuration:
$ python query_weight_config.py --weight-config 328 --weight-type odd -o splashApril22Odd -p **
Info for Weight config: 328
Tag: oddweights_killtag_singleSM
Created on: 2022-04-27 22:58:25.652198
Number of groups 2
###################################
TP MODE flags
Enable EB Odd filter 1
Enable EE Odd filter 0
Enable EB Odd Peak finder 1
Enable EE Odd Peak finder 0
Disable EB Even peak finder 0
Disable EE Even peak finder 0
Fenix EB Strip output 2 # 0=even, 1=odd, 2=larger, 3=sum
Fenix EE Strip output 0 # 0=even, 1=odd, 2=larger, 3=sum
Fedix EB Strip infobit2 1 #odd>event strip flag
Fenix EE Strip infobit2 0 #odd>event strip flag
Fenix EB TCP Output 0 # 0=even sum, 1=larger, 2=even+odd sum
Fenix EB TCP infobit1 1 #0=FGVB, 1=odd>even TCP sum
Fenix EE TCP Output 0 # 0=even sum, 1=larger, 2=even+odd sum
Fenix EE TCP infobit1 0 #0=FGVB, 1=odd>even TCP sum
#############################
Weights infos
Groups info
Group ID: 0, Weights: 0,0,0,128,0
Group ID: 1, Weights: 68,0,0,191,125
Querying the mapping and saving to file
DONE
The files ChannelNumberingEB.csv and ChannelNumberingEE.csv contain a full mapping of all the strips in EB and EE of
ECAL. For weights gymnastic we would need only the following information:
stripidused to identify the strip in the Weights studies and in the input files for ECALTPGParamBuildercmsswidgeneral identifier used in CMSSW and CondDB (offline DB)logididused to identify the strip in the ConfDB, the online P5 DB for configuration
The stripid, cmsswid and the mapping needed to idenitify the logicid are extracted from the full maps in
ChannelNumbering*.csv and saved for convenience in params_EB.csv and params_EE.csv.
To extract these files execute the script:
python get_params_EEEB.pyThen, the confDB is directly queried to the fetch the logicids in the DB. To perform this task the script
logicid_retriver.py are used. The output is params_E*_logicid.txt files.
EE and EB are split because the mapping in the confDB is different between them. Please checkout the logicid_retriver
scripts to have a look at which column of the channelview table of the ConfDB is used to retrive the logicid.
- EB: logicid of the strip by mapping: FED, TT, VFE
- EE: logicid of the strip by mapping: FED, CCU, VFE
This is the same mapping used by the ECALTPGParamBuilder.
python logicid_retriver.py -h
usage: logicid_retriver.py [-h] -i INPUTFILE -o OUTPUTFILE -s SUBDET -p
PASSWORD
optional arguments:
-h, --help show this help message and exit
-i INPUTFILE, --inputfile INPUTFILE
Input parameters file
-o OUTPUTFILE, --outputfile OUTPUTFILE
Output file
-s SUBDET, --subdet SUBDET
Subdetector EB/EE
-p PASSWORD, --password PASSWORD
ConfDB password
python logicid_retriver.py -i params_EB.csv -o params_EB_logicid.csv -s EB -p ****
python logicid_retriver.py -i params_EE.csv -o params_EE_logicid.csv -s EE -p ****
Finally all the logicids have been concatenated in a single file params_EBEE_logicids.csv.
The ECAL TPG weight configuration is based on WeightGroups and WeightIDMap:
- WeightGroups: set of 5 encoded numbers representing the weights.
- WeightIDMap: map between each stripid/logicid to a weight group.
This structure is kept in the confDB and condDB. Two files representing this structure are needed for the ECALTPGParamBuilder to build the DB tags.
In the configuration created by this repo is used with the ECALTPGParamBuilder: the weights must be encoded, and specified in the natural ordering (w0,w1,w2,w3,w4). Moreover, the 8th bit of weight, which signal to the electronics the position of the peak must NOT be included, since it is automatically added by the configuration code.
If the weights are manually loaded in the ConfDB instead, the 8th bit of the peak weight should be activate
manually. The order of the weights will be automatically inverted by the load_weights.py script.
Practically, the weights must always be ordered in the natural way (w0,w1,w2,w3,w4) when written in the txt configuration.
The weights are encoded following the formula:
W_encoded = (1 - sign(W))*64 + W*64
Then a nearest integer approximation is performed. Since the sum of the 5 weights must be always 0, there is a special code to perform the approximation in the best way: script
A script, create_idmaps.py is provided to help populating the mapping between strips and WeightGroups.
The script requires a simple txt configuration with a special syntax:
ALL 0
SUBDET EB 1
SUBDET EE 2
SUBDET EE+ 3
SUBDET EE- 4
FED 610 5
TT 1 6
FEDTT 610 1 7
STRIP 303176 6
The mapping is created by applying the rules specified one line after the other. The index of the WeightGroup is linked to the specified groups of strips.
- ALL: link the WeightGroup to all the strips, useful to be applied as the first rule.
- SUBDET: [EB/EE/EE+/EE-] assign WeightGroup to all the strips in a subdetector
- FED: specify a single FED
- TT: specify a weight group for the strips in the same TT(EB) or CCU(EE) in all the FEDs
- FEDTT: specify a weight group for the strips in a specific TT of a single FED
- STRIP: specify a specific Strip
The script is called like:
python create_idmaps.py -h
usage: create_idmaps.py [-h] [-l LOGICID] -i INPUTFILE -o OUTPUTFILE
options:
-h, --help show this help message and exit
-l LOGICID, --logicid LOGICID
Logicid mapping
-i INPUTFILE, --inputfile INPUTFILE
Input file
-o OUTPUTFILE, --outputfile OUTPUTFILE
Output file
python create_idmaps.py -l params_EBEE_logicid.csv -i idmap_example.txt -o WeighIdMap_example.txt
The WeightGroup file must be created manually specifying in each line the corresponding group of weights.
$ cat WeightGroup_custom.tx
86 99 17 32 22 --> weight group 0
86 96 17 32 25 --> weight group 1
86 97 16 32 25
87 96 17 31 25
85 96 17 33 25
...The WeightGroup and WeightIDMap files can then be uploaded manually in the DB (see next section) or used as inputs for the ECALTPGParamsBuilder code (recommended!!).
The WeightGroup and WeightIDMap files can also be converted directly to condDB tags in sqlite format by using CMSSW scripts. For more documentation about this topic see ETT Analyzer page.
A script has been prepared to manually load a set of weights in the ConfDB for manual tests on the system.
N.B.: This is not the standard way to interact with the Weights configuration!! Very dangerous!! Every weights configuration should pass through the ECALTPGParamBuilder.
python load_weights.py -h
usage: load_weights.py [-h] [-d] -t TAG --wg WG --wi WI [-l LOGICIDS] -p
PASSWORD
optional arguments:
-h, --help show this help message and exit
-d, --dry Dry run
-t TAG, --tag TAG Tag
--wg WG Weight group file
--wi WI Weight ID map
-l LOGICID, --logicid LOGICID
Logicid mapping
-p PASSWORD, --password PASSWORD
ConfDB password
The script needs WeightGroup and WeightIDMap files and the confDb password. It creates an entry in the necessary tables
with the specified tag.
The code should be run into the x2go machine in the CMS network, with python2 where the Oracle DB python libraries
are installed in the system.
For example:
python load_weights.py -t TEST --dry --wg output_files/EcalTPGWeightGroup_2018_PU50_S2.txt
--wi output_files/EcalTPGWeightIdMap_2018_PU50_S2.txt --password ******
N weights 627
New weight conf_id: None
QUERY: INSERT into FE_WEIGHT_PER_GROUP_DAT (wei_conf_id, group_id, W4,W3,W2,W1,W0) VALUES (None, 0, 92,93,16,159,24)
QUERY: INSERT into FE_WEIGHT_PER_GROUP_DAT (wei_conf_id, group_id, W4,W3,W2,W1,W0) VALUES (None, 7, 85,94,18,162,25)
QUERY: INSERT into FE_WEIGHT_PER_GROUP_DAT (wei_conf_id, group_id, W4,W3,W2,W1,W0) VALUES (None, 8, 83,96,19,161,25)
QUERY: INSERT into FE_WEIGHT_PER_GROUP_DAT (wei_conf_id, group_id, W4,W3,W2,W1,W0) VALUES (None, 12, 84,95,18,162,25)
QUERY: INSERT into FE_WEIGHT_PER_GROUP_DAT (wei_conf_id, group_id, W4,W3,W2,W1,W0) VALUES (None, 13, 84,95,19,161,25)
QUERY: INSERT into FE_WEIGHT_PER_GROUP_DAT (wei_conf_id, group_id, W4,W3,W2,W1,W0) VALUES (None, 14, 85,95,17,161,26)
[.................]
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1216021205,287)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1216021201,74)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1216021203,83)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1216021202,75)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1151052401,574)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1151052403,580)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1151052402,583)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1151052405,584)
QUERY: INSERT into FE_CONFIG_WEIGHT_DAT (wei_conf_id, logic_id, group_id) VALUES (None,1151052404,575)
[.............]There is a special macro to upload the PU optimized weights from the files produced by the PU optimization.
python prepare_config_PUweights.py --wEE txt_files/strip_weights_Sep18_PU50_S2-30_fullEE_encoded.tx --wEB
txt_files/strip_weights_Sep18_PU50_S2-30_fullEB_encoded.txt --PU 50 --S 2 -o PU50_S2The script random_weights_uploader.py creates a special configuration in which each script gets a random set of
weights.
Since the weights are not normalized to 0 this configuration creates large energy TPs in the detectors.
python random_weights_uploader.py