Thread safety in libdwarf

[jeremy-rifkin]

Hello,
I'm a little confused by the wording of section 1.3 of the documentation "Libdwarf can safely open multiple Dwarf_Debug pointers simultaneously but all such Dwarf_Debug pointers must be opened within the same thread. And all libdwarf calls must be made from within that single (same) thread."

but all such Dwarf_Debug pointers must be opened within the same thread

Does this mean it's unsafe to call dwarf_init_path from more than one thread, even if not simultaneous?

And all libdwarf calls must be made from within that single (same) thread

Does this mean it's unsafe to use a Dwarf_Debug object produced in thread A in thread B, even if a mutex is used?

I would like to be able to call dwarf_init_path from multiple threads simultaneously and work on the Dwarf_Debug objects independently, as well as also being able to use a Dwarf_Debug object produced in a thread other than the one it was produced on but with mutex locks over the resource.

[davea42]

My email response has not appeared here. Hmm.

On reviewing libdwarf source code, I find only one static array, staticerrlist[] a very special
thing for cases with a NULL dbg (and not fussy, it is just there to give dwarf_finish() a
way to free things that could not be in a Dwarf_Debug error allocation.

It is possible that if the caller ensures that , on a given Dwarf_Debug (either directly or
indirectly) only one thread at a time refers to that Dwarf_Debug, that multiple threads
could use libdwarf. But libdwarf, as currently written, is NOT locking anything
or doing anything for thread safety other than ensuring all data used by a Dwarf_Debug
is recorded in that dbg, not recorded in static memory.

Many calls in the library not directly referencing a Dwarf_Debug do so indirectly
through other arguments passed in, and many calls make updates to data
hidden in the Dwarf_Debug. Hence two or more functions active at the same time
on one Dwarf_Debug is a guarantee of unpredictable and bad behavior.

Even a simple global lock per Dwarf_Debug is difficult to implement correctly,
not to be undertaken lightly.

[davea42]

I would be happy to consult on the library aspect in case you run into
difficulty using it as you are thinking of doing. I'll be considering whether
the library should have (at least) a global lock so multiple threads cannot
collide.

[davea42]

Just asking: you using C11 locks or ???

[davea42]

Might libdwarf somehow get a thread-id and (if built for such) verify the current
thread versus something recorded in the Dwarf_Debug.
Just thinking out loud here.

[jeremy-rifkin]

Thank you so much for the replies and taking the time to look into the issue. For my use case I think worst case scenario I can lock around all calls to libdwarf just to be safe. I'm writing most of the code working with the library in C++11 and was planning on using std::mutex's.

For context I'm working on a stack tracing library. For most users traces should be infrequent but there are cases where I could imagine multiple threads simultaneously requesting traces. I'm doing a lot of caching of the contents of dwarf symbols, mainly creating a lookup table for subprograms. The main thing threads would be querying at the same time is dwarf_srclines_from_linecontext / dwarf_lineaddr / dwarf_lineendsequence. Possibly some walking of the children nodes of a subprogram might be needed depending on if I add support for resolving inlined calls.

But all that said, I think I am happy locking around libdwarf calls for now. I would be interested in knowing if line table access and die access can be thread safe.

[davea42]

dwarf_lineaddr() dwarf_lineendsequence() and similar where one passes in
a Dwarf_Line, such as dwarf_line_srcfileno() dwarf_line_is_addr_set()
and others like that simply return data from the Dwarf_Line passed in.
They never change anything UNLESS the call i fails due to a NULL Dwarf_Line
or a null pointer for the return value. In case of such an error
the applicable Dwarf_Debug is updated with error data (several changes
that must be completed without interference).

dwarf_srclines_from_linecontext() is similar in that the line context
is just read UNLESS an input argument is bogus in which case (a hard error)
the applicable Dwarf_Debug is updated with error data

Hence locking is required for all of these. The updates involved
in case of error include malloc and recording data in the Dwarf_Debug.

There are options here I am not mentioning. A Choice one can make with libdwarf
to simplify what it does on error.

Versions of these functions that just returned an integer error code through a pointer would
not have complexities at all. Just assignment of hidden struct data into the return-value
pointers.

[jeremy-rifkin]

Thank you so much!