System overview

Design goals

First of all, Annotation Web was created mainly to make it easy to annotate ultrasound videos.

Web-based

Annotaters are often clinicians who may not have high technical skills nor access to computers at the hospital where they can install software and store data. By making the system web-based, the users don't have to install anything, nor transfer any data. Everything is stored on the server and continously saving the annotations allows busy clinicians leave when they have to, and resume the annotation at a later time. Since many researchers today are using Python, we choose to use the Python web framework Django. To learn Django we highly recommend the official tutorial, however make sure you use the tutorial for the correct version of Django. Annotation web currently use Django version 2.2.

Fast interactive frontend

Ultrasound is essentially a temporal imaging modality, and it is essential for experts to see the ultrasound video playing, and being able to scroll back and forth, to know where the important structures/tissue types are in order to properly annotate the images. One option could be to use the HTML element to have playback of the ultrasound video. However, this has several downsides: 1) visualizing annotations on top of the video would be difficult, 2) videos often compress the data, 3) there exist many video codecs and proper playback requires the user to have the codec installed, 4) ultrasound recordings are not always stored as videos, but instead a series of images. Thus, we instead decided to use HTML canvas to draw the ultrasound images in the browser manually using javascript. This may sound complicated, but is actually very simple. With HTML canvas, we can draw aribtrary 2D geometry using javascript: bounding boxes, splines, lines, dots etc. on top of the ultrasound image. When a user starts to annotate an ultrasound sequence, the browser will load every image in the sequence using HTTP requests, and will store the ultrasound images in memory. Thus, when the ultrasound sequence is finally loaded, playback is very fast and smooth. However, if you have very long ultrasound sequences, memory usage and loading time can become an issue. The images are currently transferred using png and lossless compression.

Modular and Extendable

Annotation of medical images can be done in many different forms; image classification, bounding boxes, segmentation, landmarks ++. Each of these annotation tasks will have some functionality that will be specific to that task only, and there will be some functionality that is common for all tasks. In annotation web, we have tried keep the task specific functionality and database tables separated from the common ones. A Django project consists of a set of apps, which should be as lously coupled to eachother as possible, ideally it should be possible to add and remove apps with almost no code change. Thus in annotation web, every annotation task (classification, bounding box, segmentation, landmark..) are separate django apps. This allows you to easily remove annotation app not needed, and create new or copy and existing annotation app and modify it to our needs. We have used annotation web in many projects often adapting the annotation apps to specific tasks.

Database

Django uses a relational database with and object-relational mapping layer (ORM). Thus every database table has a class located in the models.py files througout the repo.

By default annotation web uses a sqlite3 database for storing all annotations and users. The images themselves are not stored in the database, but stored on disk with the path to the images stored in the database. Sqlite3 is a single file-based database, so the database is stored entirely in a single file called db.sqlite3 located in the root folder of annotation web. To backup your database simply copy the db.sqlite3 file.

Code structure

Each annotaiton task is a separate django app which resides in its own folder with the corresponding views.py, urls.py, models.py, templates/, static/ components. Below is a short explanation of the files and directories of the project:

annotationweb/        # Main project app, with all the common interface (task list, image list, annotation, dataset ++)
    templates/        # Contains HTML templates for most pages
    static/           # Contains common css and javascript code for loading images, sequence playback etc.
    views.py          # All views are defined are which are not directly annotation task specific
    models.py         # Database tables definitions, nothing annotation specific here
    urls.py           # Urls definitions
    settings.py       # Annotation web settings
    admin.py          # Django admin panel defintions exposing models.py to the admin interface.
boundingbox/          # Bounding box annotation task/app
cardiac/              # Cardiac specific annotation task/app such as LV segmentation
classification/       # Image classification annotation task/app
common/               # Contains several functions common to most annotation tasks
exporters/            # Contains annotation exporters (will probably be moved to each task soon)
importers/            # Contains all image sequence importers
landmark/             # Landmark annotation task/app
spline_segmentation/  # Spline segmentation task/app
user/                 # Some special login code for 2FA. Anything extending the default django user stuff should be put here.
manage.py             # Script to run Django commands
requirements.txt      # List of python dependencies                      

Security

Security is very important when working with medical image data. Annotation Web comes with absolutely no security guarentees, ultimately you who are setting up annotation web are responsible for doing this in a secure way as possible and in accordance with your nations privacy laws. To help you with this, here is a check list of things you should consider when setting up annotation web:

• Make sure you keep the SECRET_KEY in the file annotationweb/settings.py a secret.
• Make sure you disable debug mode in annotationweb/settings.py.
• Make sure the image data and the database (db.sqlite3 file) are stored on encrypted harddrives.
• Consider using a firewall or VPN. Do you know the IP range of the hospital or your university, then you can use a software firewall to block anyone else from conencting. Ubuntu has a very simple firewall to do this. Example: Let's say the IP range of your university is X.Y.*.*. Then you can do this:

sudo apt install ufw # Install the uncomplicated firewall (UFW)
sudo ufw allow from X.Y.0.0/16 to any port 80  # HTTP
sudo ufw allow from X.Y.0.0/16 to any port 443 # HTTPS

• Use HTTPS and SSL for end-to-end encryption between the users browser and the server. If you are not using HTTPS nor a VPN, you are essentially transferring everything; login password, ultrasound images, subject IDs on the annotation webpage, totally unecrypted over the internet! To use HTTPS/SSL encryption you need an SSL certificate, you can buy one cheap from services like namecheap.com or free from Let's encrypt. See the server setup wiki page on how to set it up on your server.
• Force your users to use two-factor authentication. This requires users to use an app on their phone, like Google Authenticator, to get a six digit code which they need to login, in addition to their normal password.
• Force users to use complicated passwords.
• Do not store any personal information (name, personal ID, ++) about patients/subjects in the database, use anonymization IDs to separate patients/subjects.
• Run python manage.py check --deploy on your server to check your setup. And also see the Django security checklist: https://docs.djangoproject.com/en/3.1/howto/deployment/checklist/