Cache symbols locally before reloading symbol file

[unknown321]

Ghidra decompiled ~95k symbols from my library which are reloaded on any gdb command. It takes about 5 seconds to invoke objcopy ~60 times:

(rr) decompiler connect ghidra --base-addr-start 0xb5c06000 --base-addr-end 0xb67df2b7
[+] 94252 symbols will be added
[+] Connected to decompiler!
(rr) b FUN_000357ac
Breakpoint 1 at 0xb5c2b7ac
(rr) c
Continuing.

Thread 1 "Thread" hit Breakpoint 1, 0xb5c2b7ac in ?? () from /lib.so
[+] 94252 symbols will be added
[+] _add_symbol_file /tmp/tmp96c__ccz.c.debug0
...  # (waiting for objcopy...)
[+] _add_symbol_file /tmp/tmp96c__ccz.c.debug62

Error in re-setting breakpoint 1: Function "FUN_000357ac" not defined.
────────── decompiler:ghidra:FUN_000257ac:0 ────
     1	 
     2	 void FUN_000257ac(int param_1,int **param_2,undefined4 *param_3)
     3	 
     4	 {
     5	   int iVar1;
──────────

(rr) ni
0xb5c2b7b0 in ?? () from /lib.so
[+] 94252 symbols will be added 
[+] _add_symbol_file /tmp/tmp96c__ccz.c.debug0
...  # (waiting for objcopy again....)
[+] _add_symbol_file /tmp/tmp96c__ccz.c.debug62

────────── decompiler:ghidra:FUN_000257ac:0 ────
     1	 
     2	 void FUN_000257ac(int param_1,int **param_2,undefined4 *param_3)
     3	 
     4	 {
     5	   int iVar1;
──────────
(rr)

Symbols in Ghidra are not modified by me, why load them again? Is this a correct behaviour?

[mahaloz]

It is, unfortunately, the intended behavior right now due to lazy programming. I never fixed it because on IDA Pro, it was fast enough (1 second or less) that I never fixed it. The problem is coming from here:

decomp2dbg/decomp2dbg/clients/gdb/gdb_client.py

Line 58 in 08f3b11

global_vars, func_headers = self.update_global_vars(), self.update_function_headers()

Nothing is cached on the client side. Things are cached server-side, but that is useless since, as you know, objcopy will be invoked many times. A way to fix would be to make a simple dict that stores things after they are sent. If the dict ever matches what was sent, don't invoke the native_symbol_add code below the line above. That should at least get rid of the many run objcopy. If you PR this, I will approve it.

[mahaloz]

Closed by #88, but a full fix should be fully implemented when #84 is implemented.

[unknown321]

#88 removes all symbols when new symbol is introduced:

--- a/decomp2dbg/clients/gdb/gdb_client.py
+++ b/decomp2dbg/clients/gdb/gdb_client.py
@@ -91,6 +91,10 @@ class GDBDecompilerClient(DecompilerClient):
             if new_entry:
                 syms_to_add.append(symbol)
 
+        if len(syms_to_add) == 0:
+            syms_to_add.append(("test123", int("0xdeadf00d",0), "function", 8))
+            print("add test123")
+
         try:

(gdb) attach 341338
Attaching to process 341338
Reading symbols from /usr/bin/less...
0x00007fd1e18a809d in __GI___libc_read (fd=3, buf=0x7ffe00f9db67, nbytes=1) at ../sysdeps/unix/sysv/linux/read.c:26
(gdb) pipe info functions | wc -l
8013
(gdb) decompiler connect ghidra 
[+] Connected to decompiler!
(gdb) pipe info functions  | grep FUN_00105020
0xbaa8e020  FUN_00105020
(gdb) ni
0x00007fd1e18a809d	26	in ../sysdeps/unix/sysv/linux/read.c
add test123
[*] Decompiler failed to get a response from decompiler on 0x2a5426e1f09d with: int exceeds XML-RPC limits, are you in a library function?
───────────────────────────────────────────────
[!] Unable to decompile function
───────────────────────────────────────────────
(gdb) pipe info functions | wc -l
8014
(gdb) pipe info functions | grep test123
0x9956800c  test123
(gdb) pipe info functions  | grep FUN_00105020
(gdb) 

It happens because all temporary symbol files are removed; there is no file to add symbol to, so a new empty file is created.

https://github.com/mahaloz/decomp2dbg/blob/main/decomp2dbg/clients/gdb/symbol_mapper.py#L259

Potential fix: don't remove symbol files, create only one file and increment on it. You'll also need to remove syms from obj which were removed from decompiler.

I suggest rolling back #88.

[mahaloz]

Rip np, I'll roll back