Implement bulk read_as (dargueta#20)

* Implement bulk read_as

* Add documentation

* Fix wrong variable name

* Overhaul, use batch reading

* Move to Github Actions instead of Travis

* Bump installed Lua versions

* More verbose logging, add library search hints for Unicorn

* Fix working directory so ctest sees the tests

* Mess with LD_LIBRARY_PATH on Linux

* Fix early collection bug

* Fix garbage collecting crash and add __close on engines too

* Add CI for Mac OS

* Stop using deprecated CMake stuff

* Fix included comment, upgrade LuaRocks

* Lazily install rocks, not in CMake initialization

* --local no longer required

* Minor code cleanup, fix exit code in `configure`

* Many many bugfixes and C-isms

* Fix broken examples

* Miscellaneous minor build stuff

* Add release/debug flags to settings

* Use build-specific C++ flags instead of if/else

* Add installation configuration

* Use `make install` in CI

* Fix some typecasting, initialize arrays, use range expressions

* Make test depend on test executable

* Use lua_Number instead of uclua_float80, add sNaN/qNaN distinction

* Use platform-independent CMake commands

* Finally fix infinity/NaN problems

* Fix datatypes in disabled tests

* Fix install prefix for Lua 5.1

* Fix signaling NaN on Mac OS

* Move templates for busted stuff into the templates dir

* Make `configure` run CMake

* Fix LuaJIT linker error on OSX

* Consolidate generated and configured files

* Context double-free test

* Fix attempt to match null pointer to C registry when closing engine

* LuaJIT mimics Lua 5.1.4 not 5.1.0

* Remove definition of lua_Unsigned

* [DEBUG ONLY] Disable context tests

* Try to fix segfault on OSX LuaJIT in lua_seti() implementation

* Try verbose mode for debugging

* Make engine and Lua state private variables for UCLuaEngine

* Let Lua manage context allocation

* Start adding tests for LuaResourceTable

* Fix wrong CMake variable name in a template

* Update changelog

* Fix wrong function call that just happened to work

* Clean up the Lua stack before crashing in allocate()

* [BROKEN] Allocate contexts on the heap, use indirect pointers for the userdata

* Overhaul contexts again to make them structs

* Fix segfault in destructor due to modifying set as we are iterating

* Delete dead code, disable lua_seti tests on OSX+LuaJIT

* Remove virtualenv dir when doing `make clean`

* Update changelog

* Update documentation

* Disable segfaulting tests

* Use more thorough asserts, remove extraneous engine handle

* I must be losing my mind

* Check the stack when panicking before pushing an error message

* [BROKEN] Require the stack to be clean when exiting a test

* Unconditionally clear the stack in ul_crash_on_error tests

We need this because Lua can put stuff on the stack aside
from our error message that we don't know about. Since all
we care about is the error message at the top of the stack,
we can clear it.

* Update library version

Co-authored-by: Diego Argueta <620513-dargueta@users.noreply.github.com>

.github/workflows/ci.yml

@@ -0,0 +1,46 @@
+name: CI
+
+on: [ push ]
+
+jobs:
+  testing:
+    runs-on: ${{ matrix.operating-system }}
+    strategy:
+      fail-fast: false
+      matrix:
+        operating-system:
+          - ubuntu-20.04
+          - macos-10.15
+        lua-version:
+          - "5.1"
+          - "5.2"
+          - "5.3"
+          - "5.4"
+          - "luajit2.0"
+    steps:
+      - uses: actions/checkout@v1
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: "3.8"
+      - name: Install Unicorn
+        run: ./tools/ci/install-unicorn.sh 1.0.2
+      - name: Set up Lua
+        # We need the MACOSX_DEPLOYMENT_TARGET environment variable because of
+        # a linker error that happens with LuaJIT:
+        # https://github.com/LuaJIT/LuaJIT/issues/449
+        run: |
+          export MACOSX_DEPLOYMENT_TARGET=10.14
+          python3 ./tools/lua_venv.py -l -o settings.json ${{ matrix.lua-version }} __lua${{ matrix.lua-version }}
+      - name: Configure build
+        run: python3 ./configure --venv-config settings.json --build-type debug
+      - name: Install
+        run: sudo make install
+      - name: Run tests
+        # We get a linker error if we don't mess with the library path. For some
+        # reason, /usr/lib64 isn't on the default search path.
+        run: |
+          export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}
+          make test
+
+  # TODO (dargueta): Test on Windows?

.gitignore

@@ -5,14 +5,24 @@
 *.a
 .git/
 build/
+cmake-build-debug/
+cmake-build-release/
 tests/c/doctest.h
 .downloaded/
 venv/
 .idea/
 .vscode/
 
 # Autogenerated files:
-Makefile.in
+*.in
 configuration.cmake
 .busted
+include/unicornlua/platform.h
+
+# Other garbage
 core*
+.venv-5.1/
+.venv-5.2/
+.venv-5.3/
+.venv-5.4/
+.venv-luajit/

CHANGELOG.rst

@@ -1,6 +1,69 @@
 Changes
 =======
 
+1.2.0 (2021-08-11)
+------------------
+
+New Features
+~~~~~~~~~~~~
+
+* Added a new (non-standard) method to engines, ``reg_read_batch_as()``, which
+  is like ``reg_read_as()`` but allows you to efficiently read multiple registers
+  at the same time. See ``docs/api.rst`` for details.
+* Added ``__close`` metamethod to engines and contexts, so they can now be used
+  with Lua 5.4's ``<close>`` local attribute.
+* Unified installation process for all platforms; ``configure`` now generates all
+  CMake stuff for you.
+* The appropriate Lua installation directory is now automatically determined.
+  Before, it used to install in the normal system directories which is *not* where
+  Lua looks.
+* Added ``--install-prefix`` to the configure script to override where the library
+  is installed.
+
+Bugfixes
+~~~~~~~~
+
+* **Potentially Breaking:** Signaling NaNs in a CPU are now passed back to Lua
+  as signaling NaNs. Before, all NaNs were converted to quiet NaNs. This brings
+  it in line with other bindings. Unless you do significant amounts of
+  floating-point operations, this won't affect you.
+* Added ``REG_TYPE_INT16_ARRAY_32``, a 32-element array of 16-bit integers.
+  I'd left it out by mistake.
+* Fixed a crash when if a context or engine object was explicitly freed, if it got
+  garbage-collected the object may think it's a double free and throw an exception.
+  This eliminates a long-standing bug in LuaJIT on Mac OS and an edge case on other
+  platforms.
+* Fixed crash resulting from a race condition, where if Lua schedules an engine
+  to be freed before a dependent context, the context would try to release its
+  resources using an invalid engine. Now the engine cleans up all contexts created
+  from it and signals all remaining Lua context objects to do nothing.
+* ``reg_read_as()`` truncated floats in arrays to integers due to a copy-paste error.
+* All the examples were broken by the ``unicorn_const`` change in 1.0b8.
+* Setting floating-point registers now (theoretically) works on a big-endian host
+  machine.
+* Fixed bug where the engine pointer/engine object pair wasn't removed from the C
+  registry upon closing. This is because the Engine pointer gets nulled out upon
+  closing, and then after closing we tried removing the pointer. It never matched
+  because it was null.
+
+Other Changes
+~~~~~~~~~~~~~
+
+* [C++] All register buffers are now zeroed out upon initialization.
+* [C++] read_float80 and write_float80 now operate on ``lua_Number``
+  rather than the platform-dependent 64-, 80-, or 128-bit floats.
+* [C++] Removed definition of ``lua_Unsigned`` for Lua 5.1 since it was both
+  wrong and unused anyway.
+* [C++] The engine handle and Lua state are now private variables for UCLuaEngine.
+* [C++] Overhauled implementation of contexts to avoid a race condition where
+  the engine was garbage-collected before a context derived from it.
+* Switched to Github Actions for CI instead of Travis.
+* The Makefile now generates the build directory if you're on CMake 3.13+.
+* ``make install`` now builds the library if it hasn't been built already.
+* ``make clean`` now removes the virtualenv directory as well.
+* ``configure`` defaults to a release build; debug builds are opt-in.
+* Removed a lot of C-isms from when this library was written in C.
+
 1.1.1 (2021-05-15)
 ------------------
 

CMakeLists.txt

@@ -9,15 +9,6 @@ set(CMAKE_CXX_STANDARD 11)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)
 
-
-# These are the rocks required for running tests and using examples.
-if (IN_CI_ENVIRONMENT)
-    set(REQUIRED_ROCKS busted)
-else()
-    set(REQUIRED_ROCKS lcurses;busted)
-endif()
-
-
 #find_package(Python3 REQUIRED)
 
 if(NOT USE_VIRTUALENV)
@@ -45,38 +36,6 @@ execute_process(
     OUTPUT_VARIABLE LUAROCKS_CPATH
     OUTPUT_STRIP_TRAILING_WHITESPACE
 )
-execute_process(
-    COMMAND ${LUA_EXE} -e "print(string.match(package.cpath, '^([^?]+)'))"
-    OUTPUT_VARIABLE LUA_INSTALL_DIR
-    OUTPUT_STRIP_TRAILING_WHITESPACE
-)
-
-message(STATUS "Installing required rocks...")
-if(USE_VIRTUALENV)
-    set(LUAROCKS_LOCAL_FLAG "")
-else()
-    set(LUAROCKS_LOCAL_FLAG "--local")
-endif()
-
-foreach(_rock IN ITEMS ${REQUIRED_ROCKS})
-    if(USE_VIRTUALENV)
-        execute_process(
-            COMMAND ${LUAROCKS_EXE} install "${_rock}"
-            OUTPUT_QUIET
-            RESULT_VARIABLE rocks_install_result
-        )
-    else()
-        execute_process(
-            COMMAND ${LUAROCKS_EXE} install "--local" "${_rock}"
-            OUTPUT_QUIET
-            RESULT_VARIABLE rocks_install_result
-        )
-    endif()
-
-    if(NOT rocks_install_result EQUAL 0)
-        message(FATAL_ERROR "Installing rock ${_rock} failed.")
-    endif()
-endforeach()
 
 
 get_filename_component(LUAROCKS_BIN_DIR "${LUAROCKS_EXE}" DIRECTORY)
@@ -109,28 +68,31 @@ endif()
 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Define targets ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
 enable_testing()
 add_subdirectory("src" "${CMAKE_CURRENT_BINARY_DIR}/lib")               # Library code
-
-cmake_policy(PUSH)
-cmake_policy(SET CMP0026 OLD)
-get_property(BUILT_LIBRARY_FILE_PATH TARGET unicornlua_library PROPERTY LOCATION)
-cmake_policy(POP)
-
-get_filename_component(BUILT_LIBRARY_DIRECTORY "${BUILT_LIBRARY_FILE_PATH}" DIRECTORY)
-
 add_subdirectory("tests/c" "${CMAKE_CURRENT_BINARY_DIR}/tests_c")       # Tests with C++ code
 add_subdirectory("tests/lua" "${CMAKE_CURRENT_BINARY_DIR}/tests_lua")   # Tests with Lua code
 
 configure_file(
-    "${CMAKE_CURRENT_SOURCE_DIR}/cmake/makefile-variables.in"
-    "${PROJECT_SOURCE_DIR}/Makefile.in"
+    "${CMAKE_CURRENT_SOURCE_DIR}/templates/Makefile.in"
+    "${CMAKE_CURRENT_BINARY_DIR}/Makefile.in"
+    @ONLY
+)
+file(GENERATE
+     OUTPUT "${PROJECT_SOURCE_DIR}/Makefile.in"
+     INPUT "${CMAKE_CURRENT_BINARY_DIR}/Makefile.in"
+)
+
+configure_file(
+
+    "${CMAKE_CURRENT_SOURCE_DIR}/templates/platform.h"
+    "${PROJECT_SOURCE_DIR}/include/unicornlua/platform.h"
     @ONLY
 )
 
 
 find_package(Doxygen)
 if(DOXYGEN_FOUND)
     configure_file(
-        "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Doxyfile.in"
+        "${CMAKE_CURRENT_SOURCE_DIR}/templates/Doxyfile.in"
         "${CMAKE_CURRENT_BINARY_DIR}/Doxyfile"
         @ONLY
     )

Makefile

@@ -7,39 +7,51 @@ MIPS_BINARY_IMAGES=$(MIPS_ASM_SOURCE_FILES:%.s=%.mips32.bin)
 
 .PHONY: all
 all: $(BUILD_DIR)
-	make -C $(BUILD_DIR)
+	$(MAKE) -C $(BUILD_DIR)
 
 
 .PHONY: clean
 clean:
-	rm -rf $(DOXYGEN_OUTPUT_BASE) $(BUILD_DIR) core*
+	cmake -E rm -rf $(DOXYGEN_OUTPUT_BASE) $(BUILD_DIR) $(VIRTUALENV_DIR) core* *.in configuration.cmake
 
 
 $(BUILD_DIR):
-	$(error You must create the build directory with CMake. See the README for details.)
+	cmake -S $(REPO_ROOT) -B $(BUILD_DIR) -DCMAKE_VERBOSE_MAKEFILE=YES
 
 
 $(SHARED_LIB_FILE): $(BUILD_DIR)
-	make -C $(BUILD_DIR)
+	$(MAKE) -C $(BUILD_DIR) unicornlua_library
+
+
+$(TEST_EXE_FILE): $(SHARED_LIB_FILE) $(TEST_SOURCES)
+	$(MAKE) -C $(BUILD_DIR) cpp_test
 
 
 .PHONY: install
 install: $(SHARED_LIB_FILE)
-	make -C $(BUILD_DIR) install
+	$(MAKE) -C $(BUILD_DIR) install
 
 
 .PHONY: docs
 docs:
-	make -C $(BUILD_DIR) docs
+	$(MAKE) -C $(BUILD_DIR) docs
 
 
 .PHONY: examples
 examples: $(X86_BINARY_IMAGES) $(SHARED_LIB_FILE)
 
 
 .PHONY: test
-test: $(BUILD_DIR) $(SHARED_LIB_FILE)
-	make -C $(BUILD_DIR) test "ARGS=--output-on-failure -VV"
+test: $(BUILT_LIBRARY_DIRECTORY)/.test-sentinel
+
+
+$(BUILT_LIBRARY_DIRECTORY)/.test-sentinel: $(TEST_EXE_FILE) $(BUSTED_EXE)
+	cmake -E touch $@
+	$(MAKE) -C $(BUILD_DIR) test "ARGS=--output-on-failure -VV"
+
+
+$(BUSTED_EXE):
+	$(LUAROCKS_EXE) install busted
 
 
 .PHONY: run_example