Handle smaller cr3 pdf page size

[johnclary]

Associated issues

• Update CRIS import to handle new CR3 PDF size atd-data-tech#18798

Testing

URL to test: Local

Steps to test:

The problematic extract is available in the dev inbox in S3 👍

1. Start your local stack. Use this modified version of the vision-zero helper

2. Run the CRIS import ETL:

$ docker compose build
$ docker run -it  --rm --env-file .env --network host -v $PWD:/app cris_import-cris_import ./cris_import.py --csv --pdf --s3-download  --s3-upload

4. You can inspect the results in your local extracts directory: ./extracts/extract_2023_[...]/crash_diagrams

---

Ship list

• Check migrations for any conflicts with latest migrations in master branch
• Confirm Hasura role permissions for necessary access
• Code reviewed
• Product manager approved

[johnclary]

i was having trouble hitting pure black pixels in the smaller format, so i added this narrow tolerance of 0 to 5.

[johnclary]

i know there's been a lot of noise here: this is ready for review again—thank you!

[johnclary]

i added this so that the number of CSVs processed will be saved in the import log even if the PDF import fails. the records_processed will look something like this:

{"pdfs": 0, "units": 82, "charges": 16, "crashes": 38, "persons": 85}

and the completed_at value will be null, indicating that the import is not complete.

[chiaberry]

I started following the test steps, and got an error about not finding the docker image, and I checked and my image was created this time as cris_import-cris_import 😬

[johnclary]

ah yes—same here. i think i had both images laying around this time, but we do need to update that command if you're using docker run instead of docker compose run. i like using the docker run command because it saves my command history, rather than losing it inside the container shell

[chiaberry]

✔️

[chiaberry]

After getting past my user errors, I got this error
requests.exceptions.InvalidSchema: No connection adapters were found for '"http://localhost:8084/v1/graphql"'

after updating my env file to remove the ", I was good to go