This project implements a basic eye tracking system using open-source Python libraries and my MacBook camera to estimate the direction of the user's gaze. Using OpenCV for camera input and image processing and with Dlib for facial landmark prediction, the program locates and isolates the user's eye, applies thresholding to detect the iris, and calculates the gaze direction based on the estimated iris position.
The main goal of this project was to determine whether or not I could create a functional gaze tracking tool with relatively simple techniques – without advanced geometry, neural methods, or hardware typically found in SOTA eye tracking systems – and to what extent this tool could be useful. It is certainly robust: it adapts to different lighting levels, is not thrown off by lens glare from eyeglasses, and can function with the laptop in any position as long as the user is facing the camera. However, the output is only returned as one of three directions (left, right, or center).
One immediate improvement could be to include magnitude, which is represented by the variable center_x. I would imagine that any further significant improvements (estimating with better precision, estimating up/down in addition to left/right, working for other eyes, etc.) would probably need much more sophisticated algorithms that would either take much more data into account (face position, distance from camera, dimensions of screen, etc.), require calibration, use more advanced geometry, or implement more neural methods. Thus, while I would like to work on improving my personal eye tracking project in the future, there are definitely other areas to work on first!