Using NLP and computer vision to detect/diagnose and label problems identified in chest x-ray images
Medium blog series: Using Computer Vision and NLP to Caption X-Rays
Automation has been a major driving force of increased efficiency, reliability, and speed across multiple industries, ranging from banking to transportation to agriculture. In this project, we investigate the potential of deep learning models to automate the process of medical image reporting, looking specifically at chest X-ray images.
Developing a deep learning model to generate/ support the reporting of findings and impressions from X-ray images would be a highly valuable development since it takes radiologists a significant amount of time to carry out this process for a large number of patients. Depending on the level of correctness achieved by the model, it may also be able to reduce human error, which is especially costly in the medical field.
We utilize the Chest X-Rays Indiana University dataset hosted on Kaggle for this project. Our goal is to measure the similarity between our predicted captions and the actual captions provided by doctor's (as presented in the dataset). To do so, we evaluate our model's performance using the BLEU metric.
Our process has been broken down into multiple steps (see notebooks directory):
- Cleaning the Indiana X-Ray imaging data (Medium article)
- Exploring ways to increase and engineer features for better results.
- Using machine learning, NLP, computer vision, and other methods to label the chest X-Rays.
- Comparing the labels we generate against the actual label provided by the doctors.
├── LICENSE
├── README.md <- The top-level README for developers using this project.
├── data
│ ├── external <- Data from third party sources.
│ ├── image_data <- Data for average frontal and lateral images
│ ├── interim <- Intermediate data that has been transformed.
│ ├── processed <- The final, canonical data sets for modeling.
│ └── raw <- The original, immutable data dump.
│
├── notebooks <- Jupyter notebooks.
│
├── reference.txt <- File containing self-determined descriptions of the variables
│ as well as an indicator for variables dropped
│
├── requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
│ generated with `pip freeze > requirements.txt`
│
├── setup.py <- makes project pip installable (pip install -e .) so src can be imported
│
└── src <- Source code for use in this project.
To setup the project, run the following:
pip install -r requirements.txt
Project based on the cookiecutter data science project template. #cookiecutterdatascience