LHE to ROOT in pylhe

[ganowak]

Hello pylhe experts, I was wondering if the pylhe package is able to take a .lhe file and transform the event data into a .root file? It seems like many MC generators output a .lhe file and I didn’t know if there was a tool to take the .lhe event data and analyze it in ROOT all within python. Within my group, analysis code is often developed in pyROOT, and so it would be nice to only make slight modifications to the analysis code when switching between the different output file types of MC generators.

[eduardo-rodrigues]

Hi there. I'm very low on time right now but let me cc @jpivarski already now.

You could for example get your data into an awkward array, see https://github.com/scikit-hep/pylhe/blob/main/examples/awkward_example.ipynb, and then convert to an RDataFrame, see https://awkward-array.org/doc/main/user-guide/how-to-convert-rdataframe.html, at which point you're in the ROOT world?

I may be missing something but this is certainly viable. Dunno if others have better ideas.

[jpivarski]

If you go this route (LHE → Awkward Array → RDataFrame → ROOT file), then you'll need to convert the Awkward C++ iterators into generic C++ types, since Awkward C++ iterators ("ArrayView") can't be serialized into an on-disk file with RDataFrame::Snapshot. (The issue here is that iterators over an Awkward Array in RDataFrame don't "own" their data; they are pointers to a data structure in Python, and when ROOT tries to serialize that, it doesn't know that it's only saved a symlink...)

Perhaps I'm being overcautious, since your data types might be simple: if the columns in the RDataFrame are numeric types or jagged arrays of numbers (which can come from an Awkward Array), those types are automatically serializable. It's only if you have lists-of-lists-of-numbers or records/class instances that you'd need to copy the data out of the transient Awkward C++ iterators and into C++ types that own their own data.

Another option is to go LHE → Awkward Array → ROOT file with Uproot (docs), but this will also involve manual type-translations if you have any types more complex than arrays of numbers or arrays of lists of numbers (i.e. jagged arrays).

For all of these methods, the limiting factor is the type-translation: LHE, Awkward Arrays, and ROOT-serializable C++ types are different type systems, different vocabularies for describing data structures. There's a tight connection between C++ objects that own their own data and ROOT files (ROOT's mission was, from the beginning, invested in making C++ serializable), and there's a tight connection between Awkward Arrays, Arrow, Python's pickle files, and Parquet files. If you don't have any constraint on file format, you'll have an easier time putting the Awkward Arrays on disk with Parquet (ak.to_parquet/ak.from_parquet), but you might have some reason to make them .root.

[ganowak]

Thank you @jpivarski and @eduardo-rodrigues, we were able to figure out the conversion from LHE to ROOT using awkward arrays.

[eduardo-rodrigues]

Hello @ganowak, glad to hear that. Your question is one that is fair and bound to come up on a regular basis. Would you like to share some example or even a little notebook example (for example based on a scikit-hep-testdate .lhe file or one of your own) for the community? Contributions are really welcome. It would be great! Let us know, thanks a lot.