A uni project to detect objects using image classification. What you see above is a box with a WiFi camera module and a motorized dial to show the most likely prediction.
Since our budget for this amazing contraption was whole 15 euros, we couldn't afford a proper server and instead hacked together a solution using an existing Home Assistant instance already exposed to the internet.
The ESP-CAM module periodically captures an image and sends it to a Home Assistant instance, which forwards it to an API running MobileNet image classification model. The server returns a list of predictions (detected objects) which is then used by ESP to move the dial to the correct position.
- ESP-CAM module
- 28BYJ-48 5V stepper motor
- ULN2003 stepper motor driver
- Stripped USB-A cable
- An amazing DIY wooden case as seen above
esp32-ai-server- Docker container doing the actual AI TensorFlow magicpyscript- Contains a custom script for Home Assistant that relays communication between ESP and API serverobject_detector- Arduino sketch running on the ESP that captures images, communicates with API and moves the motor- Note:
Base64.cppandBase64.hfiles are copied here from an older version of the adamvr/arduino-base64 library to fix compilation problems (see this issue)
- Note:
- Install PyScript in Home Assistant and copy the
pyscriptfolder from this repository into your HA configuration folder - Setup the API server container, you can use the following docker-compose configuration:
esp32-ai-server: container_name: esp32-ai-server build: "./esp32-ai-server" ports: - "2137:2137" restart: unless-stopped
- Replace the ip address in
pyscript/esp32_ai.pywith the ip of your container - Generate a Long lived access token in Home Assistant
- Replace values inside
object_detector/SecretsExample.hand rename it toSecrets.h - Download necessary libraries using Arduino Library Manager, compile the project and flash the ESP-CAM module