Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Windows CI #5

Merged
merged 40 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
83970bb
test windows ci
lobis Dec 8, 2023
1c40d9e
rename jobs
lobis Dec 8, 2023
1cd14d1
give name to steps
lobis Dec 8, 2023
68c6aa6
try other geant4 flags to compile on windows
lobis Dec 8, 2023
df299a8
do not use ninja in windows geant4 build
lobis Dec 8, 2023
93e1a20
no Ninja for xerces either
lobis Dec 8, 2023
a8da8ad
playing around with flags
lobis Dec 9, 2023
7215615
run other steps in windows too
lobis Dec 9, 2023
69c1e7d
lower cpp standard to 17
lobis Dec 9, 2023
c24ed62
sourcing geant4 in windows
lobis Dec 9, 2023
528285d
sourcing geant4 in windows
lobis Dec 9, 2023
273e3a7
sourcing geant4 in windows
lobis Dec 9, 2023
4534726
ssh debug ci
lobis Dec 9, 2023
1c53337
source powershell
lobis Dec 9, 2023
3070d0e
remove quotes
lobis Dec 9, 2023
e686b97
explicit call to powershell
lobis Dec 9, 2023
bb3291d
use .bat
lobis Dec 9, 2023
4666a0b
explicit call
lobis Dec 9, 2023
535668f
debug windows ci
lobis Dec 10, 2023
cfb2ab3
sourcing geant4
lobis Dec 10, 2023
bf356f5
do not use getpid in windows
lobis Dec 10, 2023
8c61eb9
include filesystem
lobis Dec 10, 2023
f5a7f21
windows ci
lobis Dec 10, 2023
d2755c8
windows ci
lobis Dec 10, 2023
62e781a
cmake windows config from https://github.com/HaarigerHarald/geant4_py…
lobis Dec 10, 2023
ee0b419
windows ci
lobis Dec 10, 2023
cb72886
windows ci
lobis Dec 10, 2023
3c4fd70
windows ci
lobis Dec 10, 2023
2cc1475
trigger ci
lobis Dec 10, 2023
423584a
missing deliminator
lobis Dec 10, 2023
c910df8
increase build time
lobis Dec 10, 2023
6a05a66
move datasets windows ci
lobis Dec 10, 2023
f4a301c
mkdir -p does not work in windows
lobis Dec 10, 2023
10dc4c8
move datasets windows powershell
lobis Dec 10, 2023
2647de6
powershell
lobis Dec 10, 2023
58560c2
is this env variable needed?
lobis Dec 10, 2023
3a81c66
windows env
lobis Dec 10, 2023
2ed9340
env variables windows
lobis Dec 10, 2023
1c452a4
comment debugging ci
lobis Dec 10, 2023
f08be7e
are these cmake settings needed?
lobis Dec 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 71 additions & 41 deletions .github/workflows/build-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,28 @@ on:
schedule:
- cron: 0 */24 * * *
workflow_dispatch:
inputs:
debug_enabled:
type: boolean
description:
Run the build with tmate debugging enabled
(https://github.com/marketplace/actions/debugging-with-tmate)
required: false
default: false
# inputs:
# debug_enabled:
# type: boolean
# description:
# Run the build with tmate debugging enabled
# (https://github.com/marketplace/actions/debugging-with-tmate)
# required: false
# default: false

pull_request:
push:
branches: [main]
branches:
[main]

## Paste this snippet into the workflow file to enable tmate debugging
# - name: Setup tmate session
# uses: mxschmitt/action-tmate@v3
# if:
# ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled
# }}
# with:
# limit-access-to-actor: false

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
Expand All @@ -23,20 +34,22 @@ concurrency:
env:
GEANT4_VERSION: 11.2.0
XERCES_VERSION: 3.2.4
CPP_STANDARD: 20
CPP_STANDARD: 17
Geant4_DIR: ${{ github.workspace }}/geant4

jobs:
build-geant4:
name: "Geant4 Install - No Datasets"
name: Build Geant4 on ${{ matrix.platform }}
runs-on: ${{ matrix.platform }}
timeout-minutes: 60
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
platform: [ubuntu-latest, macos-latest]
platform: [ubuntu-latest, macos-latest, windows-latest]

steps:
- uses: lukka/get-cmake@v3.27.9
- name: Setup CMake
uses: lukka/get-cmake@v3.27.9

- name: Cache Xerces-C Installation
id: cache-xerces
Expand All @@ -50,14 +63,14 @@ jobs:
run: |
git clone https://github.com/apache/xerces-c.git xerces-source
git -C xerces-source checkout tags/v${{ env.XERCES_VERSION }}
cmake -G Ninja -B xerces-build -S xerces-source -DCMAKE_INSTALL_PREFIX=${{ github.workspace }}/xerces -DCMAKE_CXX_STANDARD=${{ env.CPP_STANDARD }} -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_C_FLAGS=-fPIC ${{ matrix.platform == 'windows-latest' && '-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded' || '-Dnetwork-accessor=socket -Dtranscoder=iconv' }}
cmake --build xerces-build --parallel $(nproc) --target install
cmake -B xerces-build -S xerces-source -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=${{ github.workspace }}/xerces -DCMAKE_CXX_STANDARD=${{ env.CPP_STANDARD }} -DBUILD_SHARED_LIBS=OFF ${{ matrix.platform == 'windows-latest' && '-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DCMAKE_CXX_FLAGS_RELEASE="/MT"' || '-G Ninja -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_C_FLAGS=-fPIC -Dnetwork-accessor=socket -Dtranscoder=iconv' }}
cmake --build xerces-build --parallel $(nproc) --config Release --target install

- name: Cache Geant4 Installation
id: cache-geant4
uses: actions/cache@v3
with:
path: ${{ github.workspace }}/geant4
path: ${{ env.Geant4_DIR }}
key:
geant4-${{ env.GEANT4_VERSION }}-${{ matrix.platform }}-cpp${{
env.CPP_STANDARD }}
Expand All @@ -66,18 +79,18 @@ jobs:
if: steps.cache-geant4.outputs.cache-hit != 'true'
run: |
git clone https://github.com/Geant4/geant4.git ${{ github.workspace }}/geant4-source --depth 1 --branch v${{ env.GEANT4_VERSION }}
cmake -G Ninja -B geant4-build -S geant4-source -DCMAKE_INSTALL_PREFIX=${{ github.workspace }}/geant4 -DXERCESC_ROOT_DIR=${{ github.workspace }}/xerces -DCMAKE_CXX_STANDARD=${{ env.CPP_STANDARD }} -DCMAKE_BUILD_TYPE=Release -DGEANT4_USE_GDML=ON -DGEANT4_INSTALL_EXAMPLES=OFF -DGEANT4_INSTALL_DATA=OFF -DGEANT4_BUILD_TLS_MODEL=global-dynamic -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_C_FLAGS=-fPIC -DGEANT4_USE_SYSTEM_EXPAT=OFF -DBUILD_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF ${{ matrix.platform == 'windows-latest' && '-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded' || '' }}
cmake --build geant4-build --parallel $(nproc) --target install
cmake -B geant4-build -S geant4-source -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=${{ env.Geant4_DIR }} -DXERCESC_ROOT_DIR=${{ github.workspace }}/xerces -DCMAKE_CXX_STANDARD=${{ env.CPP_STANDARD }} -DGEANT4_USE_GDML=ON -DGEANT4_INSTALL_EXAMPLES=OFF -DGEANT4_INSTALL_DATA=OFF -DGEANT4_BUILD_TLS_MODEL=global-dynamic -DBUILD_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF ${{ matrix.platform == 'windows-latest' && '-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DCMAKE_CXX_FLAGS_RELEASE="/MT"' || '-G Ninja -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_C_FLAGS=-fPIC -DGEANT4_USE_SYSTEM_EXPAT=OFF' }}
cmake --build geant4-build --parallel $(nproc) --config Release --target install

install-geant4-datasets:
name: "Geant4 Datasets"
name: Download Geant4 datasets for ${{ matrix.platform }}
needs: [build-geant4]
runs-on: ${{ matrix.platform }}
timeout-minutes: 60
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
platform: [ubuntu-latest, macos-latest]
platform: [ubuntu-latest, macos-latest, windows-latest]

steps:
- name: Cache Geant4 Installation
Expand All @@ -99,40 +112,55 @@ jobs:
enableCrossOsArchive: true
lookup-only: true

- name: Move Geant4 Datasets
if: steps.cache-geant4-data.outputs.cache-hit != 'true'
- name: Move Geant4 Datasets (Ubuntu/MacOS)
if:
steps.cache-geant4-data.outputs.cache-hit != 'true' && matrix.platform
!= 'windows-latest'
run: |
source ${{ github.workspace }}/geant4/bin/geant4.sh
geant4-config --install-datasets
geant4-config --check-datasets
mv ${{ github.workspace }}/geant4/share/Geant4/data ${{ github.workspace }}/geant4-data

- name: Move Geant4 Datasets (Windows)
if:
steps.cache-geant4-data.outputs.cache-hit != 'true' && matrix.platform
== 'windows-latest'
run: |
.\geant4\bin\geant4-config --install-datasets
ls .\geant4\share\Geant4\data
.\geant4\bin\geant4-config --check-datasets
mv .\geant4\share\Geant4\data .\geant4-data

build-test:
name: "Build and Test"
name:
Build and Test for ${{ matrix.platform }} (Python ${{
matrix.python-version }})
needs: [install-geant4-datasets]
timeout-minutes: 30
runs-on: ${{ matrix.platform }}
strategy:
fail-fast: false
matrix:
platform: [ubuntu-latest, macos-latest]
platform: [ubuntu-latest, macos-latest, windows-latest]
python-version:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"

runs-on: ${{ matrix.platform }}

steps:
- uses: actions/checkout@v4
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- uses: lukka/get-cmake@v3.27.9
- name: Setup CMake
uses: lukka/get-cmake@v3.27.9

- name: Cache Xerces-C Installation
id: cache-xerces
Expand All @@ -154,17 +182,9 @@ jobs:

- name: pip install the package
run: |
source ${{ github.workspace }}/geant4/bin/geant4.sh
${{ matrix.platform == 'windows-latest' && '$env:CMAKE_ARGS="-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded"' || ''}}
python -m pip install .[test]

- name: Setup tmate session
uses: mxschmitt/action-tmate@v3
if:
${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled
}}
with:
limit-access-to-actor: false

# test ability to download datasets in one job and use the cache in the others
- name: Cache Geant4 Datasets
if: matrix.python-version != '3.11'
Expand All @@ -176,13 +196,23 @@ jobs:
fail-on-cache-miss: true
enableCrossOsArchive: true

- name: Move datasets
if: matrix.python-version != '3.11'
- name: Move datasets (Ubuntu/MacOS)
if:
matrix.python-version != '3.11' && matrix.platform != 'windows-latest'
run: |
GEANT4_DATA_DIR=$(python -c "import geant4_python_application; print(geant4_python_application.data_dir)")
mkdir -p ${{ github.workspace }}/geant4-data $GEANT4_DATA_DIR
mv ${{ github.workspace }}/geant4-data/* $GEANT4_DATA_DIR

- name: Move datasets (Windows)
if:
matrix.python-version != '3.11' && matrix.platform == 'windows-latest'
run: |
$env:GEANT4_DATA_DIR = python -c "import geant4_python_application; print(geant4_python_application.data_dir)"
New-Item -ItemType Directory -Path "${{ github.workspace }}\geant4-data" -Force
New-Item -ItemType Directory -Path $env:GEANT4_DATA_DIR -Force
Move-Item "${{ github.workspace }}\geant4-data\*" $env:GEANT4_DATA_DIR -Force

- name: Check import
run: |
python -c "import geant4_python_application; geant4_python_application.Application()"
Expand Down
29 changes: 19 additions & 10 deletions .github/workflows/wheels.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ jobs:
name: Source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
Expand All @@ -32,7 +33,8 @@ jobs:
python -m pip install build
python -m build --sdist .

- uses: actions/upload-artifact@v3
- name: Upload sdist
uses: actions/upload-artifact@v3
with:
name: dist
path: dist/*.tar.gz
Expand Down Expand Up @@ -90,14 +92,17 @@ jobs:
platform_id: macosx_arm64

steps:
- uses: actions/checkout@v4
- name: Checkout
uses: actions/checkout@v4

- uses: lukka/get-cmake@v3.27.9
- name: Setup CMake
uses: lukka/get-cmake@v3.27.9

- name: Setup Python
uses: actions/setup-python@v5

- uses: pypa/cibuildwheel@v2.16.2
- name: Build wheel
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_TEST_COMMAND:
python -m pytest {project}/tests -vv --reruns 3 --reruns-delay 30
Expand Down Expand Up @@ -134,7 +139,7 @@ jobs:
cmake -G Ninja -B /tmp/geant4/build -S /tmp/geant4 -DCMAKE_OSX_ARCHITECTURES=${{ matrix.platform_id == 'macosx_x86_64' && 'x86_64' || 'arm64' }} -DCMAKE_CXX_STANDARD=${{ env.CPP_STANDARD }} -DCMAKE_BUILD_TYPE=Release -DGEANT4_USE_GDML=ON -DGEANT4_INSTALL_EXAMPLES=OFF -DGEANT4_INSTALL_DATA=OFF -DGEANT4_BUILD_TLS_MODEL=global-dynamic -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_C_FLAGS=-fPIC -DGEANT4_USE_SYSTEM_EXPAT=OFF -DBUILD_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF
cmake --build /tmp/geant4/build --parallel $(nproc) --target install > /dev/null 2>&1
rm -rf /tmp/geant4
- name: Upload Wheels
- name: Upload wheel
uses: actions/upload-artifact@v3
with:
name: dist
Expand All @@ -149,8 +154,10 @@ jobs:
github.event.inputs.publish_to_test_pypi

steps:
- uses: actions/setup-python@v4
- uses: actions/download-artifact@v3
- name: Setup Python
uses: actions/setup-python@v4
- name: Download wheels
uses: actions/download-artifact@v3
with:
name: dist
path: dist
Expand All @@ -168,8 +175,10 @@ jobs:
if: github.event_name == 'release' && github.event.action == 'published'

steps:
- uses: actions/setup-python@v4
- uses: actions/download-artifact@v3
- name: Setup Python
uses: actions/setup-python@v4
- name: Download wheels
uses: actions/download-artifact@v3
with:
name: dist
path: dist
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
#include "geant4_application/PrimaryGeneratorAction.h"
#include "geant4_application/StackingAction.h"

#include <filesystem>

namespace py = pybind11;

namespace geant4_app {
Expand Down Expand Up @@ -57,6 +59,8 @@ class Application {
const PrimaryGeneratorAction& GetPrimaryGeneratorAction() const;
const StackingAction& GetStackingAction() const;
const DetectorConstruction& GetDetectorConstruction() const;

static std::filesystem::path GetTemporaryApplicationDirectory();
};

}// namespace geant4_app
8 changes: 8 additions & 0 deletions src/geant4_application/src/Application.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -183,3 +183,11 @@ const DetectorConstruction& Application::GetDetectorConstruction() const {
}
return *detectorConstruction;
}

filesystem::path Application::GetTemporaryApplicationDirectory() {
const auto dir = filesystem::temp_directory_path() / "geant4_python_application";
if (!filesystem::exists(dir)) {
filesystem::create_directories(dir);
}
return dir;
}
14 changes: 11 additions & 3 deletions src/geant4_application/src/DetectorConstruction.cpp
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@

#include "geant4_application/DetectorConstruction.h"
#include "geant4_application/Application.h"
#include "geant4_application/SensitiveDetector.h"

#include <G4LogicalVolumeStore.hh>
#include <G4NistManager.hh>
#include <G4PhysicalVolumeStore.hh>
#include <G4VisAttributes.hh>

#include <filesystem>
#include <unistd.h>
#include <random>

using namespace std;
using namespace geant4_app;
Expand Down Expand Up @@ -38,7 +38,15 @@ void DetectorConstruction::SetSensitiveVolumes(const set<string>& volumes) {
G4VPhysicalVolume* DetectorConstruction::Construct() {
G4GDMLParser parser;

auto gdmlTmpPath = filesystem::temp_directory_path() / ("GDML_PID" + to_string(getpid()) + ".gdml");
const auto gdmlTemporaryDir = Application::GetTemporaryApplicationDirectory() / "gdml";
if (!fs::exists(gdmlTemporaryDir)) {
fs::create_directories(gdmlTemporaryDir);
}

// get a unique id for the temporary file. This should be unique across all processes (batch jobs)
auto uniqueString = std::to_string(std::hash<unsigned int>{}(std::random_device()()));
const auto gdmlTmpPath = gdmlTemporaryDir / ("GDML_" + uniqueString + ".gdml");
cout << "Writing temporary GDML file to " << gdmlTmpPath << endl;
// write contents of gdml string into file, raise exception if there is a problem
ofstream gdmlTmpFile(gdmlTmpPath);
if (!gdmlTmpFile.is_open()) {
Expand Down