Access symbolic linked .root files with uproot

[roy-brener-cern]

Dear uproot community,

Is there a way to open a .root file with uproot in cases where the file is symbolically-linked to its actual location ("target") on another sever/location?
In standard ROOT for example, there is an option of using TDavixFile, e.g.
(in shell)
ls -l root_file.root
to obtain the symbolic link target, and then in (py)ROOT

import ROOT
file = ROOT.TDavixFile(symbolic_link_target)

(TDavixFile is used in such cases instead of TFile)

This works fine with ROOT. I've not found a parallel option in uproot, but perhaps the option exists somewhere/how...?
Any help or suggestion would be warmly welcomed.

Many thanks in advance,

Roy

[jpivarski]

If it's a filesystem symbolic link, then you can just use the link file itself:

$ ls -l root_file.root
lrwxrwxrwx  1 i_made this_up   1066 Oct 14 21:46 root_file.root -> /elsewhere/in/the/filesystem/here.root

>>> import uproot
>>> with uproot.open("root_file.root") as file:
...

That is to say, let the filesystem do its thing with symbolic links. (Uproot isn't handling that; when Python asks for a file at root_file.root, it gets data from the linked file. ROOT would do that, too, with its ordinary TFile.)

But if you need to bring TDavixFile into it, then you're not talking about ordinary symbolic links, the kind that can be identified with ls -l. Uproot doesn't have a Davix handler. If you're just using Davix to follow a root:// or https:// URL, then use the URL because Uproot does have XRootD and HTTP handlers, but not Davix, specifically.

[roy-brener-cern]

Hi Jim,

Many thanks for your quick reply.

uproot.open("root_file.root") doesn't work for me as it's indeed the other kind of symbolic link i.e. to a different file system.

Actually, I can identify the target of it using ls -l root_file.root but to be very specific my full command is
ls -l root_file.root | awk '{print $NF}' | xargs -i echo "davs://lcg-se01.icepp.jp:18443{}"
where davs://lcg-se01.icepp.jp:18443 is the path to the target file system in which root_file.root is actually stored which I need to add as to point to the file location.
This works with TDavixFile (I don't know exactly what that protocol is except that it works --- its documentation seems a bit lacking https://root.cern/doc/master/namespaceDavix.html). If you have any suggestion how to make this work with uproot I would be very grateful.

Kind regards,

Roy

[jpivarski]

I don't know much about the DaviX protocol (docs, GitHub, Wikipedia). I just learned that you're supposed to capitalize the "X"!

If the actual file-reading resolves to an HTTP or XRootD URL, then you can use that directly in the uproot.open function. I'm going to leave this open for anybody who has more expertise and might be able to say how DaviX relates to HTTP or XRootD.

If we really need a new "DavixSource" in Uproot, that's a medium-sized undertaking. Uproot currently has two remote protocols:

uproot5/src/uproot/source/http.py

Line 532 in 02c4cd8

class HTTPSource(uproot.source.chunk.Source):

and

uproot5/src/uproot/source/xrootd.py

Line 250 in 02c4cd8

class XRootDSource(uproot.source.chunk.Source):

A DavixSource would have to link Uproot's Source protocol to DaviX requests. Something like that would take at least a week and at most a month for someone unfamiliar with the code.

Looking at the DaviX documentation (and PyPI), I'm not finding any Python bindings. Having to also create Python bindings for DaviX would make this a somewhat-larger-than-medium-sized project. It could potentially be done through the command-line tools, but that would be glitchy and probably doesn't support multi-range queries, which is an important feature.

You might need to wait until someone writes a quick pybind11 binding for it (or convince someone to do that) if they really don't have a Python interface. It would be a well-scoped student project.

[roy-brener-cern]

Hi Jim,

I understand.

Alas, uproot.open doesn't seem to work with this davs://lcg-se01.icepp.jp:18443 path/protocol. If you're on CERN lxplus I would be happy to provide some example files for you to try, but given your comments and my trials I'm doubtful anything would change without updates to uproot like the ones you've suggested be applied.

In the meantime we'll try to move these files to somewhere accessible with uproot.open.

Thanks,

Roy