Skip to content
This repository has been archived by the owner on Jul 23, 2024. It is now read-only.

Create chimp worker #23

Merged
merged 45 commits into from
Jul 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
6c66b77
Create chimp chomp crate
garryod Jul 4, 2023
1798efc
Setup ONNXRuntime inference session
garryod Jul 4, 2023
d05a525
Add image loading utils
garryod Jul 4, 2023
835c3be
Create chimp_protocol for common structs
garryod Jul 5, 2023
01c5ed6
Create inference worker
garryod Jul 5, 2023
ceb5c9c
Add (de)serializers to protocol structs
garryod Jul 5, 2023
346e5d4
Setup RabbitMQ job consumer
garryod Jul 5, 2023
7089c59
Create predictions producer worker
garryod Jul 5, 2023
1d0cd03
Bump clap from 4.3.9 to 4.3.10
garryod Jul 5, 2023
10fb145
Add chimp chomp & protocol to dockerfile
garryod Jul 5, 2023
359173e
Disable parallel container builds in CI
garryod Jul 5, 2023
18e238d
Add basic docs to chimp crates
garryod Jul 5, 2023
923e464
Add rabbitmq server to dev docker-compose
garryod Jul 5, 2023
a9c1f6b
Add v4 feature to uuid dependency
garryod Jul 5, 2023
b624e90
Rename chimp chimp rabbitmq url argument
garryod Jul 5, 2023
4239f2f
Structure model inputs / outputs for batch inference
garryod Jul 6, 2023
d28feff
Add opencv to development container
garryod Jul 12, 2023
b86cb8c
Use OpenCV for image loading & processing
garryod Jul 12, 2023
8681062
Refine chimp protocol types
garryod Jul 12, 2023
f4cc22d
Load images with OpenCV
garryod Jul 12, 2023
faa2f18
Fix inference worker
garryod Jul 12, 2023
1583069
Determine insertion point from masks
garryod Jul 12, 2023
8acc21f
Find well centers with Hough transforms
garryod Jul 12, 2023
6184a70
Update job interface
garryod Jul 12, 2023
282a056
Spawn workers from main thread
garryod Jul 12, 2023
a8dc203
Improve inference logging
garryod Jul 12, 2023
0e7ec62
Fix well centering
garryod Jul 12, 2023
c5f6ca8
Remove tokio thread limit
garryod Jul 12, 2023
688ca46
Add timeout option to shutdown when unoccupied
garryod Jul 13, 2023
7dee89f
Re-bias task scheduler to prefer sending responses
garryod Jul 13, 2023
366df4b
Add thread option to control worker thread count
garryod Jul 13, 2023
94b52d8
Propagate inference backpressure to RabbitMQ
garryod Jul 13, 2023
70cb0a5
Refactor main function into setup & runtime
garryod Jul 13, 2023
25ca8dd
Install chimp_chomp dependencies in CI
garryod Jul 14, 2023
dc139c0
Add chimp chomp onnx model with Git LFS
garryod Jul 14, 2023
ecf86bf
Export chimp model alongside binary
garryod Jul 14, 2023
9696297
Fix docker build for chimp chomp
garryod Jul 14, 2023
6f283f3
Remove tensorrt dependency
garryod Jul 14, 2023
447395d
Fix insertion point location
garryod Jul 17, 2023
7a81952
Silence chimp_chomp build
garryod Jul 17, 2023
dd6e594
Cascade processing errors to response
garryod Jul 17, 2023
26e01fb
Use single frame model with rockmaker image dimensions
garryod Jul 17, 2023
b5c990d
Add well centering tests
garryod Jul 17, 2023
a34bb86
Add insertion point finding tests
garryod Jul 17, 2023
7813cb7
Add chimp_chomp docstrings
garryod Jul 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ RUN rustup component add rustfmt clippy
RUN apt-get update \
&& apt-get install --yes --no-install-recommends \
sqlite3 pre-commit \
libopencv-dev clang libclang-dev \
&& rm -rf /var/lib/apt/lists/*
3 changes: 2 additions & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@
"ghcr.io/devcontainers/features/common-utils:2": {
"username": "none",
"upgradePackages": false
}
},
"ghcr.io/devcontainers/features/git-lfs:1": {}
},
// Make sure the files we are mapping into the container exist on the host
"initializeCommand": "bash -c 'for i in $HOME/.inputrc; do [ -f $i ] || touch $i; done'",
Expand Down
9 changes: 8 additions & 1 deletion .devcontainer/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ services:
environment:
OPA_URL: http://opa:8181
POSTGRES_URL: postgres://postgres:password@postgres
RABBITMQ_URL: amqp://rabbitmq:password@rabbitmq

opa:
image: docker.io/openpolicyagent/opa:0.53.1
Expand All @@ -27,4 +28,10 @@ services:
postgres:
image: docker.io/library/postgres:15.3-bookworm
environment:
POSTGRES_PASSWORD: password
POSTGRES_PASSWORD: password

rabbitmq:
image: docker.io/library/rabbitmq:3.12.1
environment:
RABBITMQ_DEFAULT_USER: rabbitmq
RABBITMQ_DEFAULT_PASS: password
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
chimp_chomp/chimp.onnx filter=lfs diff=lfs merge=lfs -text
10 changes: 10 additions & 0 deletions .github/workflows/code.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ jobs:
- name: Checkout source
uses: actions/checkout@v3.5.2

- name: Install dependencies
uses: awalsh128/cache-apt-pkgs-action@v1.3.0
with:
packages: libopencv-dev clang libclang-dev

- name: Install stable toolchain
uses: actions-rs/toolchain@v1.0.6
with:
Expand Down Expand Up @@ -50,6 +55,11 @@ jobs:
- name: Checkout source
uses: actions/checkout@v3.5.2

- name: Install dependencies
uses: awalsh128/cache-apt-pkgs-action@v1.3.0
with:
packages: libopencv-dev clang libclang-dev

- name: Install stable toolchain
uses: actions-rs/toolchain@v1.0.6
with:
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/container.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@ jobs:
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.repository
strategy:
matrix:
service:
target:
- chimp_chomp
- soakdb_sync
- pin_packing
max-parallel: 1
runs-on: ubuntu-latest
steps:
- name: Generate Image Name
run: echo IMAGE_REPOSITORY=ghcr.io/$(tr '[:upper:]' '[:lower:]' <<< "${{ github.repository }}")-${{ matrix.service }} >> $GITHUB_ENV
run: echo IMAGE_REPOSITORY=ghcr.io/$(tr '[:upper:]' '[:lower:]' <<< "${{ github.repository }}")-${{ matrix.target }} >> $GITHUB_ENV

- name: Log in to GitHub Docker Registry
if: github.event_name != 'pull_request'
Expand All @@ -41,8 +43,7 @@ jobs:
- name: Build Image
uses: docker/build-push-action@v4.0.0
with:
build-args: |
SERVICE=${{ matrix.service }}
target: ${{ matrix.target }}
push: ${{ github.event_name == 'push' && startsWith(github.ref, 'refs/tags') }}
load: ${{ !(github.event_name == 'push' && startsWith(github.ref, 'refs/tags')) }}
tags: ${{ steps.meta.outputs.tags }}
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ jobs:
- name: Checkout source
uses: actions/checkout@v3.5.2

- name: Install dependencies
uses: awalsh128/cache-apt-pkgs-action@v1.3.0
with:
packages: libopencv-dev clang libclang-dev

- name: Install nightly toolchain
uses: actions-rs/toolchain@v1.0.6
with:
Expand Down
Loading
Loading