How to change the output parquet file with jagged array to root file?

[ZhenxuanZhang-Jensen]

Hi experts,
Recently I used a preselection framework to output a parquet file for further analysis, and in the framework, I save a jagged array in the output parquet file to save the different jets in each event:

events['SelectedJet_mass']
origin print: 
 [[32.3, 18.5, 7.44, 6.9, 8.02, 9.36, 8.24], ... 6.08], [24.7, 8.09, 9.2, 12.8, 4.04]]
type: <class 'awkward.highlevel.Array'>
the list of this info: 
 [[32.34375, 18.484375, 7.4375, 6.8984375, 8.0234375, 9.359375, 8.2421875], [18.625, 18.984375, 9.40625, 10.0, 6.58203125, 6.27734375], [12.921875, 12.28125, 6.58203125, 6.27734375], [40.375, 14.5703125, 6.48046875, 7.921875, 6.8359375, 6.58203125], [12.84375, 12.765625, 8.390625, 8.78125, 8.0859375], [16.09375, 8.109375, 11.2578125, 4.9140625], [16.625, 3.734375, 4.296875, 5.20703125], [11.03125, 15.8359375, 12.1875, 7.015625], [8.359375, 6.12890625, 4.375, 3.984375, 5.30859375], [10.1953125, 7.24609375, 10.2734375, 8.1953125], [27.578125, 9.9609375, 7.01171875, 6.81640625, 6.23828125], [10.296875, 10.0703125, 7.7265625, 5.24609375], [16.578125, 10.1875, 8.9453125, 5.19140625],...]

And for now, I need to change it to the root file to compute the limit. I use this package to do that:

from parquet_to_root import parquet_to_root
parquet_to_root("merged_nominal_addVars.parquet", "data.root", treename='Data_13TeV_cat0', verbose=False)

And it shows me a bug like this:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_526/1388603186.py in <module>
      2 from parquet_to_root import parquet_to_root
      3 # parquet_to_root("/eos/user/z/zhenxuan/hhwwgg_data/merged_nominal_addVars.parquet", "/eos/user/z/zhenxuan/hhwwgg_root/hhwwgg_data.root", treename='Data_13TeV_cat0', verbose=False)
----> 4 parquet_to_root("/eos/user/z/zhenxuan/hhwwgg_M300_test/merged_nominal.parquet", "/eos/user/z/zhenxuan/hhwwgg_root/hhwwgg_check_jaggedArray.root", treename='Data_13TeV_cat0', verbose=False)

~/.local/lib/python3.9/site-packages/parquet_to_root/parquet_to_root_pyroot.py in parquet_to_root_pyroot(infiles, outfile, treename, verbose)
    150             _setup_branch_list(field, tree, vectorlens, stringarrs)
    151         else:
--> 152             raise ValueError(f'Cannot translate field "{branch}" of input Parquet schema. Field is described as {field.type}')
    153 
    154     # Fill loop

ValueError: Cannot translate field "SelectedJet_mass" of input Parquet schema. Field is described as large_list<item: float not null>

As far as I know, if we change the awkward array to root file, we actually change the array field to the root's tree branches. If so, it seems no way we can save different numbers of variable in different branches so we can't convert the jagged array to root file right?
If so, the only method is to pad it all to the same length and then convert it right?

[agoose77]

I think, from a cursory glance of the source code, parquet_to_root only supports writing single-field lists to a TTree. I suspect that your list has more than one field in order for you to see that error.

I'd suggest using uproot to write to a TTree.

[ZhenxuanZhang-Jensen]

Hi Angus,
I do some check on the uproot to write the TTree. I find that if we use uproot to write the jagged array in the tree and use uproot to open it, then it's perfectly fine. But if we want to open it with ROOT,then the result looks strange to me and I can't understand. As I said above, I try to convert my jagged array file to root file so that I can do some further moves on a C++ framework. So do you know how to use ROOT(you know, like TFile TTree GetEntry) to open the tree and branch and access the right numbers?
By the way, Do you know how to write the code and the output with like ">>>" or "..." in GitHub so that I can show my code and result more beautifully and clearly? thanks!

import uproot
file = uproot.recreate("example.root")
import awkward as ak
file["tree3"] = {"branch": ak.Array([[1.1, 2.2, 3.3], [], [4.4, 5.5]])}
file["tree3"].show()

name                 | typename                 | interpretation                
---------------------+--------------------------+-------------------------------
nbranch              | int32_t                  | AsDtype('>i4')
branch               | double[]                 | AsJagged(AsDtype('>f8'))

import ROOT

tfile = ROOT.TFile("example.root")
tree = tfile.Get('tree3')
nentries = tree.GetEntries()
for i in range(0,nentries):
    print(tree.GetEntry(i))

28
4
20

[agoose77]

TTree::GetEntry loads the given entry and returns the number of bytes read. In PyROOT, to read the branch, you can write

>>> import ROOT
>>> import numpy as np
>>> tfile = ROOT.TFile("example.root")
>>> tree = tfile.Get('tree3')
>>> for entry in tree:
>>>     branch = np.asarray(entry.branch)
>>>     print(branch)    
[1.1 2.2 3.3]
[]
[4.4 5.5]

the loop over tree is syntactic sugar for your for loop. The entry object is actually just tree.

[ZhenxuanZhang-Jensen]

Okay, thanks a lot