How to add support for a new architecture

Introduction

RetDec stands for Retargetable Decompiler. As such, it aims to support many different architectures, and tries to make adding a new architecture as easy as possible. This article lists steps one needs to do in order to implement support for a new architecture. Not all of the actions are strictly obligatory, but it is a good idea to do them all anyway.

Basic information

RetDec uses Capstone disassembler in the process of translation of binary data to intermediate representation (LLVM IR). Therefore, RetDec can conceivably support all the architectures Capstone supports. Using a different disassembler is not unimaginable, but would be extremely difficult in the current design.

The bulk of work on adding a new architecture is isolated to capstone2llvmir library and can be done without any knowledge of the rest of the decompilation process. However, in order to produce a good quality results, it might be necessary to implement architecture-specific analyses.

List of actions

1. Browse RetDec issues and try to find an issue asking for adding a support for the architecture you are interested in (e.g. #9).
   • If such an issue exists, carefully study the comments. Maybe there are some problems you should be aware of. Maybe someone is already working on it. If you decide to work on it yourself, please let us and others know - write a comment, provide a link to your fork, etc.
   • If such an issue doesn't yet exist, create it (e.g. #494), and let us/others know you are working on it.
2. Take a look at the capstone-dumper tool. If the architecture you plan to implement is not yet supported, implement a module for it. Take inspiration from other modules - it is quite straightforward. This is beneficial because:
   • You get familiar with Capstone in general and Capstone's module for the selected architecture.
   • Dumping tool will come in handy later when you will work on instruction translation (it takes a lot of experimentation).
   • Others will be able to inspect instructions for this architecture. (Please send us a Pull Request once you are finished.)
3. Add a new module to capstone2llvmir library.
   • Study library's general design.
   • Enable the new architecture in deps/capstone/CMakeLists.txt (e.g. -DCAPSTONE_ARM64_SUPPORT=ON).
   • Take heavy inspiration in modules for other architectures. They are all very similar in design. Keep the same design and just adapt it to the specifics of the selected architecture.
   • Write unit tests as you go. Again, just adapt what is already used in the existing modules.
   • Don't try to implement all/many instructions right away. Implement only the core set and move to other steps in this list. You can come back later and add more instructions if/when necessary. Keep in mind that some instructions should not be implemented at all.
   • When in doubt contact us (RetDec authors). If you are serious about implementing a new architecture, we will be glad to help you.
4. Add support for the new architecture to capstone2llvmirtool.
   • This is very similar to capstone-dumper, but instead of dumping info about instruction, it translate it to a sequence of LLVM IR using the capstone2llvmir library.
5. TODO