Merge develop

.github/workflows/docs.yml

@@ -1,18 +1,21 @@
 name: Build & Publish Docs & Storybook
 
 on:
+  workflow_dispatch:
   push:
     branches:
       - main
       - develop
     paths:
       - 'frontend/**'
+      - 'backend/docs/**'
       - '.github/workflows/docs.yml'
       - '.yarn/**'
       - '.storybook/**'
   pull_request:
     paths:
       - 'frontend/**'
+      - 'backend/docs/**'
       - '.github/workflows/docs.yml'
       - '.yarn/**'
       - '.storybook/**'
@@ -43,15 +46,15 @@ jobs:
         working-directory: ./frontend
 
       - name: Setup Github Pages
-        uses: actions/configure-pages@v2
+        uses: actions/configure-pages@v5
 
       - name: Upload artifact
-        uses: actions/upload-pages-artifact@v1
+        uses: actions/upload-pages-artifact@v3
         with:
           path: ./docs-pages
 
       - name: Archive production artifacts
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: build
           path: ./docs-pages
@@ -67,4 +70,4 @@ jobs:
     steps:
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v1
+        uses: actions/deploy-pages@v4

.github/workflows/podman.yml

@@ -129,6 +129,7 @@ jobs:
         # Variables
         AML_ALLOWED_HOSTS: ${{ vars.AML_ALLOWED_HOSTS }}
         AML_CORS_ORIGIN_WHITELIST: ${{ vars.AML_CORS_ORIGIN_WHITELIST }}
+        CSRF_TRUSTED_ORIGINS: ${{ vars.CSRF_TRUSTED_ORIGINS }}
         AML_DEBUG: ${{ vars.AML_DEBUG }}
         AML_LOCATION_PROVIDER: ${{ vars.AML_LOCATION_PROVIDER }}
         AML_SUBPATH: ${{ vars.AML_SUBPATH }}
@@ -229,6 +230,7 @@ jobs:
       # Variables
       AML_ALLOWED_HOSTS: ${{ vars.AML_ALLOWED_HOSTS }}
       AML_CORS_ORIGIN_WHITELIST: ${{ vars.AML_CORS_ORIGIN_WHITELIST }}
+      CSRF_TRUSTED_ORIGINS: ${{ vars.CSRF_TRUSTED_ORIGINS }}
       AML_DEBUG: ${{ vars.AML_DEBUG }}
       AML_LOCATION_PROVIDER: ${{ vars.AML_LOCATION_PROVIDER }}
       AML_SUBPATH: ${{ vars.AML_SUBPATH }}

README.md

@@ -30,22 +30,22 @@ Install [Docker Desktop](https://docs.docker.com/desktop/).
 Make a copy of [the file](https://github.com/Amsterdam-Music-Lab/MUSCLE/blob/develop/.env.dist) `.env.dist` (in the same directory as this README) and rename it to `.env.` This file contains variables used by Docker to start up a container network serving MUSCLE.
 
 Start Docker (the app icon is a whale carrying containers). Then, open a terminal and run
-`docker-compose up` (add `sudo` on Linux).
-This command starts up the containers defined in `docker-compose.yaml`:
+`docker compose up` (add `sudo` on Linux).
+This command starts up the containers defined in `docker compose.yaml`:
 - a PostgreSQL container, for storing experiment/user/playlist data, saved on the host machine in the Docker user data, represented in the volume `db_data`. Data added to the database will persist if the container is shut down.
 - a ip2country container, which provides country codes for ip addresses, used for demographic information of users.
 - a container of the server, defined in DockerfileDevelop in `backend`. The Dockerfile defines the Python version and installs development dependencies. The startup command runs migrations and then starts up a Django development server.
 - a container of the client, defined in DockerfileDevelop in `frontend`. The Dockerfile defines the node version and installs node modules. The startup command kicks off a React development server.
 
 Once you see all containers have started up, open your browser and navigate to [localhost:3000](http://localhost:3000). You should now be able to see the first screen of the Goldsmiths Musical Sophistication Index questionnaire.
 
-Since the `docker-compose.yaml` defines bind mounts for `backend` and `frontend`, any changes to the files on the host are immediately reflected in the containers, which means code watching works and hot reload works in the same way as with a native node or Django server.
+Since the `docker compose.yaml` defines bind mounts for `backend` and `frontend`, any changes to the files on the host are immediately reflected in the containers, which means code watching works and hot reload works in the same way as with a native node or Django server.
 
 To stop the containers, press `ctrl-c` or (in another terminal) run
-`docker-compose down`.
+`docker compose down`.
 
 ## Production build
-A production build should define its own `docker-compose.yaml`, making use of the `Dockerfile` of the `backend` and `frontend` environments. It should also define a custom .env file, with safe passwords for the SQL database and the Python backend. Instead of mounting the entire backend and frontend directory and using the development servers, the backend should serve with gunicorn, and the frontend should use a build script to compile static html, css and JavaScript.
+A production build should define its own `docker compose.yaml`, making use of the `Dockerfile` of the `backend` and `frontend` environments. It should also define a custom .env file, with safe passwords for the SQL database and the Python backend. Instead of mounting the entire backend and frontend directory and using the development servers, the backend should serve with gunicorn, and the frontend should use a build script to compile static html, css and JavaScript.
 
 ## Troubleshooting
 

backend/aml/base_settings.py

@@ -18,6 +18,7 @@
 # Workaround for deprecated ugettext_lazy in django-inline-actions
 import django
 from django.utils.translation import gettext_lazy
+
 django.utils.translation.ugettext_lazy = gettext_lazy
 
 logger = logging.getLogger(__name__)
@@ -30,87 +31,88 @@
 # See https://docs.djangoproject.com/en/3.0/howto/deployment/checklist/
 
 # SECURITY WARNING: keep the secret key used in production secret!
-SECRET_KEY = os.getenv("AML_SECRET_KEY", 'topsecret')
+SECRET_KEY = os.getenv("AML_SECRET_KEY", "topsecret")
 
 # SECURITY WARNING: don't run with debug turned on in production!
-DEBUG = os.getenv('AML_DEBUG', '') != 'False'
+DEBUG = os.getenv("AML_DEBUG", "") != "False"
 
 ALLOWED_HOSTS = os.getenv("AML_ALLOWED_HOSTS", "localhost").split(",")
 
 # Application definition
 
 INSTALLED_APPS = [
-    'admin_interface',
-    'modeltranslation', # Must be before django.contrib.admin
-    'django.contrib.admin',
-    'django.contrib.auth',
-    'django.contrib.contenttypes',
-    'django.contrib.sessions',
-    'django.contrib.humanize',
-    'django.contrib.messages',
-    'django.contrib.staticfiles',
-    'nested_admin',
-    'inline_actions',
-    'django_markup',
-    'corsheaders',
-    'experiment',
-    'image',
-    'participant',
-    'result',
-    'session',
-    'section',
-    'theme',
-    'question'
+    "admin_interface",
+    "modeltranslation",  # Must be before django.contrib.admin
+    "django.contrib.admin",
+    "django.contrib.auth",
+    "django.contrib.contenttypes",
+    "django.contrib.sessions",
+    "django.contrib.humanize",
+    "django.contrib.messages",
+    "django.contrib.staticfiles",
+    "django_select2",
+    "nested_admin",
+    "inline_actions",
+    "django_markup",
+    "corsheaders",
+    "experiment",
+    "image",
+    "participant",
+    "result",
+    "session",
+    "section",
+    "theme",
+    "question",
 ]
 
 MIDDLEWARE = [
-    'corsheaders.middleware.CorsMiddleware',
-    'django.middleware.security.SecurityMiddleware',
-    'django.contrib.sessions.middleware.SessionMiddleware',
-    'django.middleware.locale.LocaleMiddleware',
-    'django.middleware.common.CommonMiddleware',
-    'django.middleware.csrf.CsrfViewMiddleware',
-    'django.contrib.auth.middleware.AuthenticationMiddleware',
-    'django.contrib.messages.middleware.MessageMiddleware',
-    'django.middleware.clickjacking.XFrameOptionsMiddleware',
+    "corsheaders.middleware.CorsMiddleware",
+    "django.middleware.security.SecurityMiddleware",
+    "django.contrib.sessions.middleware.SessionMiddleware",
+    "django.middleware.locale.LocaleMiddleware",
+    "django.middleware.common.CommonMiddleware",
+    "django.middleware.csrf.CsrfViewMiddleware",
+    "django.contrib.auth.middleware.AuthenticationMiddleware",
+    "django.contrib.messages.middleware.MessageMiddleware",
+    "django.middleware.clickjacking.XFrameOptionsMiddleware",
 ]
 
-ROOT_URLCONF = 'aml.urls'
+ROOT_URLCONF = "aml.urls"
 
 TEMPLATES = [
     {
-        'BACKEND': 'django.template.backends.django.DjangoTemplates',
-        'DIRS': [],
-        'APP_DIRS': True,
-        'OPTIONS': {
-            'context_processors': [
-                'django.template.context_processors.debug',
-                'django.template.context_processors.request',
-                'django.contrib.auth.context_processors.auth',
-                'django.contrib.messages.context_processors.messages',
+        "BACKEND": "django.template.backends.django.DjangoTemplates",
+        "DIRS": [],
+        "APP_DIRS": True,
+        "OPTIONS": {
+            "context_processors": [
+                "django.template.context_processors.debug",
+                "django.template.context_processors.request",
+                "django.contrib.auth.context_processors.auth",
+                "django.contrib.messages.context_processors.messages",
             ],
         },
     },
 ]
 
-WSGI_APPLICATION = 'aml.wsgi.application'
+WSGI_APPLICATION = "aml.wsgi.application"
 
 
 # Password validation
 # https://docs.djangoproject.com/en/3.0/ref/settings/#auth-password-validators
 
 AUTH_PASSWORD_VALIDATORS = [
     {
-        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
+        "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
     },
     {
-        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
+        "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
     },
     {
-        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
+        "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
     },
     {
-        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
+        "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
     },
 ]
 
@@ -120,7 +122,7 @@
 
 # LANGUAGE_CODE = 'en-us'
 
-TIME_ZONE = 'Europe/Amsterdam'
+TIME_ZONE = "Europe/Amsterdam"
 
 USE_I18N = True
 
@@ -141,28 +143,27 @@
 # Static files (CSS, JavaScript, Images)
 # https://docs.djangoproject.com/en/3.0/howto/static-files/
 
-STATIC_URL = '/static/'
+STATIC_URL = "/static/"
 
 # Added to run : python manage.py collectstatic
-STATIC_ROOT = os.path.join(BASE_DIR, 'static')
-MEDIA_ROOT = os.path.join(BASE_DIR, 'upload')
-MEDIA_URL = '/upload/'
+STATIC_ROOT = os.path.join(BASE_DIR, "static")
+MEDIA_ROOT = os.path.join(BASE_DIR, "upload")
+MEDIA_URL = "/upload/"
 
 # Geoip service
 LOCATION_PROVIDER = os.getenv("AML_LOCATION_PROVIDER", "")
 
 # From mail address, used for sending mails
-FROM_MAIL = 'name@example.com'
+FROM_MAIL = "name@example.com"
 CONTACT_MAIL = FROM_MAIL
 
 # Target url participants will be redirected to after reloading their user-session
-RELOAD_PARTICIPANT_TARGET = 'https://app.amsterdammusiclab.nl'
-HOMEPAGE = 'https://www.amsterdammusiclab.nl'
+RELOAD_PARTICIPANT_TARGET = "https://app.amsterdammusiclab.nl"
+HOMEPAGE = "https://www.amsterdammusiclab.nl"
 
 # CORS origin white list from .env
 CORS_ORIGIN_WHITELIST = os.getenv(
-    "AML_CORS_ORIGIN_WHITELIST",
-    "http://localhost:3000,http://127.0.0.1:3000,{}".format(HOMEPAGE)
+    "AML_CORS_ORIGIN_WHITELIST", "http://localhost:3000,http://127.0.0.1:3000,{}".format(HOMEPAGE)
 ).split(",")
 
 
@@ -171,19 +172,19 @@
 CORS_ALLOW_CREDENTIALS = True
 
 CORS_ALLOW_HEADERS = list(default_headers) + [
-    'sentry-trace',
-    'baggage',
+    "sentry-trace",
+    "baggage",
 ]
 
-CSRF_TRUSTED_ORIGINS = [os.getenv('CSRF_TRUSTED_ORIGINS')]
+CSRF_TRUSTED_ORIGINS = [os.getenv("CSRF_TRUSTED_ORIGINS")]
 
-SESSION_SAVE_EVERY_REQUEST = False # Do not set to True, because it will break session-based participant_id
+SESSION_SAVE_EVERY_REQUEST = False  # Do not set to True, because it will break session-based participant_id
 
 CSRF_USE_SESSIONS = False
 
-LOCALE_PATHS = [os.path.join(BASE_DIR, 'locale')]
+LOCALE_PATHS = [os.path.join(BASE_DIR, "locale")]
 
-DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
+DEFAULT_AUTO_FIELD = "django.db.models.AutoField"
 
 if os.getenv("SENTRY_DSN"):
     sentry_sdk.init(
@@ -201,10 +202,6 @@
 else:
     logger.info("SENTRY_DSN is not defined. Skipping Sentry initialization.")
 
-MARKUP_SETTINGS = {
-    'markdown': {
-        'safe_mode': False
-    }
-}
+MARKUP_SETTINGS = {"markdown": {"safe_mode": False}}
 
-SUBPATH = os.getenv('AML_SUBPATH', '')
+SUBPATH = os.getenv("AML_SUBPATH", "")

backend/docs/03_The_admin_interface.md

@@ -12,7 +12,7 @@ Log in:
 (This is set through the .env file. Obviously, these passwords are only suitable for local development!)
 
 You can see an overview of different Django apps:
-<img width="656" alt="adminList" src="/assets/images/AdminInterface.png">
+<img width="656" alt="adminList" src="../assets/images/AdminInterface.png">
 
 - **Admin_Interface** to customize how this admin interface appears to you
 - **Authentication and Authorization** to give other researchers access to this admin interface

backend/docs/05_Creating_an_experiment.md

@@ -4,21 +4,38 @@
 Make sure you're logged in to the admin interface (see previous step).
 
 Then, next to "Experiments" click "Add".
-<img width="1002" alt="AddExperiment" src="https://github.com/Amsterdam-Music-Lab/MUSCLE/assets/11174072/f71e77a1-1308-40f3-b8d5-e6f81f2ae0c0">
+<img alt="AddExperiment" src="../assets/images/AddExperiment.png">
 
 ## The Experiment form
 The form which appears lets you select the following fields:
-1. a name for your new experiment (required)
-2. a slug for the experiment (required). The slug is the part of the part of the URL which will be used to find the experiment from the frontend. In this case, the experiment will be available from `localhost:3000/someslug`.
-3. a URL with more information about the experiment (optional)
-4. a hashtag for posting about the experiment on social media (optional)
-5. a preferred language in which the experiment should be shown (optional). If this field is left "Unset", this means that the language will be determined based on the user's browser settings. Otherwise, the set language will be shown, with a fallback to English in case no translation is available.
-6. the rules of the experiment (required). This sets which logic the experiment follows. You can select implemented "rules" that have already been implemented in Python.
-7. the number of rounds the experiment should run (optional). This is set to 10 by default, which is usually fine. Not all rules files use rounds to control when specific "phases" of the experiment start or end. Think of staircasing experiments, which will present more or less difficult stimuli depending on the user's responses. These rules just ignore the "Rounds" field.
-8. bonus points (optional). Few experiments use this field, but you can use this to give bonus points to a participant who completed the experiment.
-9. a playlist (optional). For most experiments (except for those which are questionnaires) you will want to select a playlist here.
-<img width="828" alt="AddExperimentPart1" src="https://github.com/Amsterdam-Music-Lab/MUSCLE/assets/11174072/97d57e23-aa26-40db-83ec-a1312873fdf4">
-
-10. an experiment series (optional). This is used to string several experiments together. Best to leave alone, may be deprecated.
-11. questions (optional). You can select any questions you want to ask your participants before starting an audio experiment here.
-<img width="992" alt="AddExperimentPart2" src="https://github.com/Amsterdam-Music-Lab/MUSCLE/assets/11174072/d8253af5-9be2-4428-a90a-9e612c863fb8">
+
+1. a unique slug for your new experiment (required) If you test locally, the experiment will be available from `localhost:3000/someslug`.
+
+2. a checkbox indicating whether the experiment is active
+
+3. optional: a theme config (through which you can change the background, logos, etc.)
+
+4. `Translated Content` containing information about the experiment, a consent file, and messages to be displayed on social media, in a given language.
+
+5. Phases (required): a phase of your experiment, which may contain one or more blocks. Note that every experiment needs to have at least one phase with one block configured.
+
+    If you select "dashboard", the phase will appear as a dashboard, from which users can select which block to play. If "dashboard" is false, the blocks will be presented in linear order to the participant. If you select "randomize", the blocks will be shuffled, either in the dashboard view, or the linear procedure.
+
+    Within a phase, you have the option to add one or more **blocks**, with the following options:
+
+    - Index: order in which block should appear in the phase (will be ignored when `phase.randomize` is set)
+
+    - Slug: unique slug of the block
+
+    - Rules: the ruleset for the block
+
+    - Rounds: how many rounds should be presented to the participant (*used in some, but not all rulesets!*)
+
+    - Bonus points: bonus points to be awarded to the participant under given conditions (*used by very few rulesets*)
+
+    - Playlists: select one or more playlists to be associated with the block (*note that some rulesets require a very specific format for the playlist, this will be checked when you save the experiment, and may generate warnings*)
+
+    - BlockTranslatedContent: a name and description of the block in a given language, will only be shown in the dashboard view
+
+6. a Social Media Config to customize the tags and url which should be shown when participants share that they played the experiment on social media, as well as the channels in which they can share
+