improve configuration steps (#90)

* improve configuration steps

* add lints to CI workflow

* update building.md contributing.md

* fix unsatisfied linking error

.github/workflows/tests.yml

@@ -8,9 +8,38 @@ on:
 
 jobs:
 
+  code_style:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout Ghidrathon
+      uses: actions/checkout@v4
+    - name: Configure Java
+      uses: actions/setup-java@v4
+      with:
+        distribution: "temurin"
+        java-version: "17"
+    - name: Configure Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: "3.12"
+    - name: Lint with isort
+      run: |
+        pip install isort
+        isort --profile black --length-sort --line-width 120 -c .
+    - name: Lint with black
+      run: |
+        pip install black
+        black -l 120 --check .
+    - name: Lint with google-java-format
+      run: |
+        mkdir ../tmp
+        wget https://github.com/google/google-java-format/releases/download/v1.19.2/google-java-format-1.19.2-all-deps.jar -O ../tmp/google-java-format.jar
+        find . -name "*.java" -type f -print | xargs java -jar ../tmp/google-java-format.jar --dry-run --set-exit-if-changed
+
   tests:
     name: Tests in ${{ matrix.python-version }} on ${{ matrix.os }}
     runs-on: ${{ matrix.os }}
+    needs: code_style
     strategy:
       fail-fast: false
       matrix:
@@ -36,10 +65,10 @@ jobs:
         python-version: ${{ matrix.python-version }}
     - name: Configure temp folder
       run: mkdir ../tmp
-    - name: Install Python Jep
+    - name: Install Python requirements
       run: |
-        pip install numpy
-        pip install jep==4.2.0
+        pip install -r util/requirements.txt
+        pip install -r data/python/tests/requirements.txt
         python -c "import importlib.util;import pathlib;print(pathlib.Path(importlib.util.find_spec('jep').origin).parent)"
     - name: Download dependencies Linux/macOS
       if : ${{ matrix.os != 'windows-latest' }}
@@ -73,7 +102,6 @@ jobs:
       run: python util/ghidrathon_configure.py ../tmp/ghidra/ghidra_PUBLIC -d
     - name: Run tests
       run: |
-        pip install packaging
         ../tmp/ghidra/ghidra_PUBLIC/support/analyzeHeadless ${{ github.workspace }}/../tmp/ghidra test -Import ${{ github.workspace }}/../tmp/ghidra/ghidra_PUBLIC/GPL/DemanglerGnu/os/linux_x86_64/demangler_gnu_v2_24 -PostScript ${{ github.workspace }}/data/python/tests/hello.py -PostScript ${{ github.workspace }}/data/python/tests/runall.py > ../tmp/log.txt
     - name: Check tests
       run: |

README.md

@@ -3,27 +3,67 @@
 [![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE.txt)
 [![CI](https://github.com/mandiant/ghidrathon/actions/workflows/tests.yml/badge.svg)](https://github.com/mandiant/ghidrathon/actions/workflows/tests.yml)
 
-Ghidrathon is a Ghidra extension that adds Python 3 scripting capabilities to Ghidra. Why? Ghidra natively supports scripting in Java and Jython. Unfortunately, many open-source analysis tools, like [capa](https://github.com/mandiant/capa), [Unicorn Engine](https://github.com/unicorn-engine/unicorn), [angr](https://github.com/angr/angr), etc., are written in Python 3 making it difficult, and in some cases, impossible to use these tools in Ghidra. More so the security community has released several great plugins for other SRE frameworks like IDA Pro and Binary Ninja, but again, because many of these plugins use Python 3 it is difficult to port them to Ghidra. Ghidrathon helps you use existing and develop new Python 3 tooling in Ghidra and script Ghidra using modern Python in a way that tightly integrates with Ghidra's UI.
+Ghidrathon is a Ghidra extension that adds Python 3 scripting capabilities to Ghidra. Why? Ghidra natively supports scripting in Java and Jython. Unfortunately, many open-source analysis tools, like [capa](https://github.com/mandiant/capa), [Unicorn Engine](https://github.com/unicorn-engine/unicorn), [angr](https://github.com/angr/angr), etc., are written in Python 3 making it difficult, and in some cases, impossible to use these tools in Ghidra. More so the security community has released several great plugins for other SRE frameworks like IDA Pro and Binary Ninja, but again, because many of these plugins use Python 3 it is difficult to port them to Ghidra. Ghidrathon helps you use existing and develop new Python 3 tooling in Ghidra and script Ghidra using modern Python in a way that tightly integrates with Ghidra's UI. It replaces the existing Python 2.7 extension implemented via Jython. This includes the interactive interpreter window, integration with the Ghidra Script Manager, and script execution in Ghidra headless mode.
+
+![example](./data/ghidrathon_interp.png)
+
+Please see our Ghidra Python 3 script example [here](./ghidra_scripts/ghidrathon_example.py) for a closer look at writing Python 3 scripts for Ghidra.
 
 Check out:
 
 - The overview in our first [Ghidrathon blog post](https://www.mandiant.com/resources/blog/ghidrathon-snaking-ghidra-python-3-scripting)
 
-Ghidrathon replaces the existing Python 2.7 extension implemented via Jython. This includes the interactive interpreter window, integration with the Ghidra Script Manager, and script execution in Ghidra headless mode.
+## Installing Ghidrathon
+
+### Requirements
+Tool | Version |Source |
+|---|---|---|
+| Ghidrathon | `>= 4.0.0` | https://github.com/mandiant/Ghidrathon/releases |
+| Python | `>= 3.8.0` | https://www.python.org/downloads |
+| Jep | `== 4.2.0` | https://pypi.org/project/jep |
+| Ghidra | `>= 10.3.2` | https://github.com/NationalSecurityAgency/ghidra/releases |
+| Java | `>= 17.0.0` | https://adoptium.net/temurin/releases |
+
+Use the following steps to install Ghidrathon to your Ghidra environment:
+
+1. Download and unzip the latest Ghidrathon [release](https://github.com/mandiant/Ghidrathon/releases)
+2. Execute the following commands using the Python interpreter that you'd like to use with Ghidrathon:
+```
+$ python -m pip install -r requirements.txt
+$ python ghidrathon_configure.py <absolute_path_to_ghidra_install_dir>
+```
+3. Install the Ghidrathon extension (`.zip`) into Ghidra:
+   * Using Ghidra's UI:
+        * Navigate to `File > Install Extensions...`
+        * Click the green `+` button
+        * Navigate to the Ghidrathon extension (`.zip`)
+        * Click `Ok`
+    * Using a limited environment:
+        * Extract the Ghidrathon extension (`.zip`)  to `<absolute_path_to_ghidra_install_dir>\Ghidra\Extensions`
+
+### Switching Python Interpreters
+
+You can switch Ghidrathon to use a different Python interpreter by running `ghidrathon_configure.py` using the new Python interpreter.
+
+### Python Virtual Environments
+
+Ghidrathon supports Python virtual environments. **To use a Python virtual environment, complete step `1` from within your virtual environment.** Do the same when running `ghidrathon_configure.py` to switch the Ghidrathon to use a different interpreter.
 
-## Python 3 Interpreter Window
+## Using Ghidrathon
+
+### Python 3 Interpreter Window
 
 The interpreter window provides interactive access to your Python 3 interpreter. Click "Window" and select "Ghidrathon" to open the interpreter window.
 
 ![example](./data/ghidrathon_interp.png)
 
-## Ghidra Script Manager Integration
+### Ghidra Script Manager Integration
 
 Ghidrathon integrates directly with the Ghidra Script Manager enabling you to create, edit, and execute Python 3 scripts within Ghidra. Click "Create New Script" and select "Python 3" to create a new Python 3 script. Click "Run Script" or "Run Editors's Script" to execute your Python 3 script and check the Ghidra Console window for script output.
 
 ![example](./data/ghidrathon_script.png)
 
-## Ghidra Headless Mode
+### Ghidra Headless Mode
 
 Ghidrathon helps you execute Python 3 scripts in Ghidra headless mode. Execute the `analyzeHeadless` script located in your Ghidra installation folder, specify your Python 3 script, and check the console window for script output.
 
@@ -39,17 +79,6 @@ Function _start @ 0x101060: 1 blocks, 13 instructions
 Function deregister_tm_clones @ 0x101090: 4 blocks, 9 instructions
 Function register_tm_clones @ 0x1010c0: 4 blocks, 14 instructions
 Function __do_global_dtors_aux @ 0x101100: 5 blocks, 14 instructions
-Function frame_dummy @ 0x101140: 1 blocks, 2 instructions
-Function main @ 0x101149: 1 blocks, 9 instructions
-Function __libc_csu_init @ 0x101170: 4 blocks, 34 instructions
-Function __libc_csu_fini @ 0x1011e0: 1 blocks, 2 instructions
-Function _fini @ 0x1011e8: 1 blocks, 4 instructions
-Function _ITM_deregisterTMCloneTable @ 0x105000: 0 blocks, 0 instructions
-Function printf @ 0x105008: 0 blocks, 0 instructions
-Function __libc_start_main @ 0x105010: 0 blocks, 0 instructions
-Function __gmon_start__ @ 0x105018: 0 blocks, 0 instructions
-Function _ITM_registerTMCloneTable @ 0x105020: 0 blocks, 0 instructions
-Function __cxa_finalize @ 0x105028: 0 blocks, 0 instructions
 [...]
 INFO  REPORT: Post-analysis succeeded for file: /example.o (HeadlessAnalyzer)
 INFO  REPORT: Save succeeded for processed file: /example.o (HeadlessAnalyzer)
@@ -75,42 +104,6 @@ Ghidrathon links your local Python installation to Ghidra using the open-source
 
 For more information on how Jep works to embed Python in Java see their documentation [here](https://github.com/ninia/jep/wiki/How-Jep-Works).
 
-## Installing Ghidrathon
-
-### Requirements
-Tool | Version |Source |
-|---|---|---|
-| Ghidrathon | `>= 4.0.0` | https://github.com/mandiant/Ghidrathon/releases |
-| Python | `>= 3.8.0` | https://www.python.org/downloads |
-| Jep | `== 4.2.0` | https://pypi.org/project/jep |
-| Ghidra | `>= 10.3.2` | https://github.com/NationalSecurityAgency/ghidra/releases |
-| Java | `>= 17.0.0` | https://adoptium.net/temurin/releases |
-
-Use the following steps to install Ghidrathon to your Ghidra environment:
-
-1. Download and unzip the latest Ghidrathon [release](https://github.com/mandiant/Ghidrathon/releases)
-2. Execute the following commands using the Python interpreter that you'd like to use with Ghidrathon:
-```
-$ python -m pip install jep==4.2.0
-$ python ghidrathon_configure.py <absolute_path_to_ghidra_install_dir>
-```
-3. Install the Ghidrathon extension (`.zip`) into Ghidra:
-   * Using Ghidra's UI:
-        * Navigate to `File > Install Extensions...`
-        * Click the green `+` button
-        * Navigate to the Ghidrathon extension (`.zip`)
-        * Click `Ok`
-    * Using a limited environment:
-        * Extract the Ghidrathon extension (`.zip`)  to `<absolute_path_to_ghidra_install_dir>\Ghidra\Extensions`
-
-### Switching Python Interpreters
-
-You can switch Ghidrathon to use a different Python interpreter by running `ghidrathon_configure.py` using the new Python interpreter.
-
-### Python Virtual Environments
-
-Ghidrathon supports Python virtual environments. **To use a Python virtual environment, complete step `1` from within your virtual environment.** Do the same when running `ghidrathon_configure.py` to switch the Ghidrathon to use a different interpreter.
-
 ## Considerations
 
 Ghidrathon uses the open-source library [Jep](https://github.com/ninia/jep) which uses the Java Native Interface (JNI) to embed Python in the JVM. The Ghidra developers advise against JNI in Ghidra for reasons discussed [here](https://github.com/NationalSecurityAgency/ghidra/issues/175).

data/python/jepwelcome.py

@@ -14,7 +14,6 @@
 
 import sys
 
-
 message = r"""
    _____ _     _     _           _   _                 
   / ____| |   (_)   | |         | | | |                

data/python/jepwrappers.py

@@ -5,10 +5,10 @@
 # Unless required by applicable law or agreed to in writing, software distributed under the License
 #  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and limitations under the License.
-import sys
-import abc
 import io
 import os
+import abc
+import sys
 
 import java.lang
 
@@ -107,20 +107,16 @@ def wrapped(*args, **kwargs):
 
 class GhidrathonTextIOWrapperBase(abc.ABC):
     @abc.abstractproperty
-    def __stream__(self):
-        ...
+    def __stream__(self): ...
 
     @abc.abstractproperty
-    def name(self):
-        ...
+    def name(self): ...
 
     @abc.abstractproperty
-    def closed(self):
-        ...
+    def closed(self): ...
 
     @abc.abstractmethod
-    def fileno(self):
-        ...
+    def fileno(self): ...
 
     @property
     def line_buffering(self):

data/python/tests/hello.py

@@ -10,6 +10,7 @@
 # See the License for the specific language governing permissions and limitations under the License.
 
 import pathlib
+
 import jep
 
 path = pathlib.Path("hello.txt")

data/python/tests/requirements.txt

@@ -0,0 +1,2 @@
+numpy
+packaging

data/python/tests/runall.py

@@ -14,9 +14,9 @@
 Note: you must run this harness from the Ghidra script manager or headless mode
 """
 
-import unittest
-import pathlib
 import sys
+import pathlib
+import unittest
 
 
 def main():

data/python/tests/test_jepbridge.py

@@ -29,8 +29,8 @@ def assertIsNotJavaObject(self, o):
 
     def test_type_instance(self):
         # see Jep: https://github.com/ninia/jep/blob/15e36a7ba54eb7d8f7ffd85f16675fa4fd54eb1d/src/test/python/test_import.py#L54-L65
-        from java.lang import Object
         from java.io import Serializable
+        from java.lang import Object
         from java.util import Date
         from ghidra.program.database import ProgramDB
 

doc/building.md

@@ -1,4 +1,4 @@
-# Building Ghidrathon
+# Building
 
 ## Requirements
 

doc/contributing.md

@@ -0,0 +1,18 @@
+# Contributing
+
+## Linting
+
+### Requirements
+
+Tool | Source |
+|---| ---|
+| isort | https://pypi.org/project/isort |
+| black | https://pypi.org/project/black |
+| google-java-format | https://github.com/google/google-java-format/releases/download/v1.19.2/google-java-format-1.19.2-all-deps.jar |
+
+Use the following commands to identify format errors:
+```
+$ isort --profile black --length-sort --line-width 120 -c /local/path/to/src
+$ black -l 120 -c /local/path/to/src
+$ find /local/path/to/src -name "*.java" -type f -print | xargs java -jar google-java-format-1.19.2-all-deps.jar --dry-run --set-exit-if-changed
+```

ghidra_scripts/ghidrathon_example.py

@@ -10,8 +10,7 @@
 #  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and limitations under the License.
 
-from ghidra.program.model.block import SimpleBlockIterator
-from ghidra.program.model.block import BasicBlockModel
+from ghidra.program.model.block import BasicBlockModel, SimpleBlockIterator
 
 for func in currentProgram().getListing().getFunctions(True):
     block_count = 0

src/main/java/ghidrathon/GhidrathonConsoleInputThread.java

@@ -140,7 +140,8 @@ private boolean evalPython(String line) throws RuntimeException {
     GhidrathonScript interactiveScript = plugin.getInteractiveScript();
     Program program = plugin.getCurrentProgram();
 
-    try (Transaction tx = program != null ? program.openTransaction("Ghidrathon console command") : null) {
+    try (Transaction tx =
+        program != null ? program.openTransaction("Ghidrathon console command") : null) {
 
       interactiveTaskMonitor.clearCanceled();
       interactiveScript.setSourceFile(new ResourceFile(new File("Ghidrathon")));