Merge pull request #158 from wtsi-npg/devel

Release 1.2.0

CHANGELOG.md

@@ -3,7 +3,16 @@
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
-## [Unreleased] - yyyy-mm-dd
+## [1.2.0] - 2023-07-12
+
+### Added
+
+* Display the library_tube_barcode / Pool name in the QC View
+* Calculate and display deplexing percentages when appropriate
+
+### Changed
+
+* Modified the search-by-run interface to allow multiple runs to be shown at once
 
 ## [1.1.0] - 2023-05-30
 

README.md

@@ -3,6 +3,10 @@
 Please see the documents in the [docs](docs) folder for documentation on
 subjects other than development, testing and deployment.
 
+- [Background](docs/background.md)
+- [The QC process](docs/qc_process.md)
+- [The QC database schema](docs/qc_schema_explained.md)
+
 ## Install and run locally
 
 You can install the package with `pip install .` from the repository's root.
@@ -51,31 +55,12 @@ You might want to `chmod 600 /path/to/env/file` as it contains passwords.
 The env file can contain definitions that are user by the frontend. These definitions
 should be prefixed with `VITE_`, for example `VITE_SOME_PORT="4567"`. To be visible
 to the build of the frontend container, they have to be copied to the `frontend/.env`
-file: `(cat /path/to/env/file | grep VITE_) > frontend/.env` 
+file: `(cat /path/to/env/file | grep VITE_) > frontend/.env`
 
 To build containers, from the root of this repository, run :
 Build: `docker-compose --env-file /path/to/env/file build`
 Run: `docker-compose --env-file /path/to/env/file up -d`
 
-Finally, you can setup a systemd service in `/etc/systemd/system/npg_langqc.service` (change the paths accordingly):
-
-```conf
-[Unit]
-Description=%i service with docker compose
-Requires=docker.service
-After=docker.service
-
-[Service]
-Type=oneshot
-RemainAfterExit=true
-WorkingDirectory=/path/to/repository/root
-ExecStart=/usr/local/bin/docker-compose --env-file /path/to/env/file up -d --remove-orphans
-ExecStop=/usr/local/bin/docker-compose --env-file /path/to/env/file  down
-
-[Install]
-WantedBy=multi-user.target
-```
-
 ### Development setup
 
 Follow the same steps as above. Then use `docker-compose.dev.yml` to override `docker-compose.yml`:
@@ -165,44 +150,3 @@ docker run \
   -v ${CONFIG_FOLDER}:/config \
   ghcr.io/wtsi-npg/npg_langqc:devel
 ```
-
----------------------------------------------------------------------------------------------------
-
-The next steps are to set the server up as a SystemD service.
-
-### Step 5
-
-Create a service file `/etc/systemd/system/npg_langqc.service`, with the following
-contents:
-
-```conf
-[Unit]
-Description=npg_langqc Docker container
-Wants=docker.service
-
-[Service]
-Restart=always
-ExecStart=/usr/bin/docker start -a npg_langqc
-ExecStop=/usr/bin/docker stop -t 10 npg_langqc
-
-[Install]
-WantedBy=multi-user.target
-```
-
-### Step 6
-
-Enable and start the service:
-
-```bash
-sudo systemctl enable npg_langqc.service
-sudo systemctl start npg_langqc.service
-```
-
-### Step 7
-
-Verify everything is working:
-
-```bash
-systemctl status npg_langqc.service
-journalctl -xe
-```

docs/background.md

@@ -0,0 +1,27 @@
+# Background
+
+`npg_langqc` comprises an API server, a DB schema and a frontend GUI to enable users to assess long-read sequencing outcomes. In principle we are able to expand to support any post-sequencing QC assessments. The "long-read" distinguishes between our primarily Illumina short-read sequencing pipeline and the later adopted ONT, Pac Bio etc. machines. The Illumina short-read QC system cannot be reasonably extended to support other non-Illumina sequencers due to deeply embedded assumptions about operations and naming conventions.
+
+## Tech choices
+
+### Backend
+
+A Python FastAPI server was developed to protect the interface from needing to understand the highly volatile multi-LIMS warehouse (MLWH) schema. It would also allow programatic querying of the QC schema, but there is no immediate demand for that. FastAPI automatically generates OpenAPI definitions.
+
+The MLWH schema is hosted on a MySQL server, so the QC schema owned by this application was deployed to MySQL as well. `sqlalchemy` can support other DB engines.
+
+### Frontend
+
+The frontend was developed as a single page application using the `Vue 3` framework. Vue was chosen on account of the amount of other Vue applications developed within the institute. A prototype was trialled with Svelte but we had no supportive culture to help fledgling development along. Vue 3 was chosen in preference to Vue 2 in the hope of supporting `typescript` amongst other things, but it proved very difficult for an inexperienced web developer to integrate Vue 3 and typescript together with other features. Attempts to use the OpenAPI spec of the backend to generate a client library were unsuccessful, and the client code lacks a "nice" way to access the data. Further effort might be rewarded, see [fastapi docs](https://fastapi.tiangolo.com/advanced/generate-clients/).
+
+### Environment
+
+The whole application has been containerised with Docker and the development and production instances are hosted on virtual machines in OpenStack.
+
+Deployment has been automated to a degree via OpenStack API and Ansible playbook, see the gitlab-hosted `langqc_deploy` project for deployment instructions.
+
+### Authentication
+
+Authentication is managed by Okta and a reverse-proxy server (Apache) that redirects users as required. The application can therefore be opened to all staff without risk to data integrity, and both backend and frontend components do not need to implement authentication. Authorisation is handled by a manually curated user list in the QC DB. The backend API can then control who is allowed make changes to QC outcomes.
+
+As a consequence, development outside of the institutional network requires an Okta.com dev account. As of 2023 we have not needed granular authorisation.

frontend/README.md

@@ -6,7 +6,8 @@ It is written in Javascript using the Vue 3 framework, and expects to interact w
 
 ## Recommended IDE Setup
 
-[VSCode](https://code.visualstudio.com/) + [Volar](https://marketplace.visualstudio.com/items?itemName=Vue.volar) (and disable Vetur) + [TypeScript Vue Plugin (Volar)](https://marketplace.visualstudio.com/items?itemName=Vue.vscode-typescript-vue-plugin).
+[VSCode](https://code.visualstudio.com/) + [Volar](https://marketplace.visualstudio.com/items?itemName=Vue.volar) (and disable Vetur)
+ESLint, Pylance, isort
 
 ## Customize configuration
 
@@ -34,20 +35,8 @@ restrictions and for the backend host. Good luck with that.
 npm run build
 ```
 
-### Run End-to-End Tests with [Cypress](https://www.cypress.io/)
-
-```sh
-npm run build
-npm run test:e2e # or `npm run test:e2e:ci` for headless testing
-```
-
 ### Lint with [ESLint](https://eslint.org/)
 
 ```sh
 npm run lint
 ```
-
-Note that ESLint, cypress config files and vue3 components do not get on.
-Expect to see warnings like "props is not defined" and similar, and the
-format settings for config.js files versus modules causes one or the other to
-fail. Improve `.eslintrc.json` at your leisure.

frontend/package.json

@@ -1,6 +1,6 @@
 {
   "name": "npg-longue-vue",
-  "version": "1.1.0",
+  "version": "1.2.0",
   "description": "UI for LangQC",
   "author": "Kieron Taylor <kt19@sanger.ac.uk>",
   "license": "GPL-3.0-or-later",
@@ -19,7 +19,7 @@
   },
   "dependencies": {
     "@element-plus/icons-vue": "^2.0.10",
-    "element-plus": "^2.2.18",
+    "element-plus": "^2.3.7",
     "lodash": "^4.17.21",
     "pinia": "^2.0.23",
     "vue": "^3.2.37",

frontend/src/App.vue

@@ -1,16 +1,20 @@
 <script setup>
-import { RouterView } from 'vue-router';
+import { RouterView, useRoute } from 'vue-router';
 import { onMounted, provide, ref } from "vue";
 import { ElMessage } from "element-plus";
+import { Search } from '@element-plus/icons-vue';
 
 import router from "@/router/index.js";
 import LangQc from "@/utils/langqc.js";
 
 let logout_redirect_url = ref(null);
 let input = ref('');
+let searchMode = ref('search');
 let appConfig = ref(null);
 const apiClient = new LangQc();
 
+let route = useRoute()
+
 provide('appConfig', appConfig)
 
 onMounted(() => {
@@ -37,9 +41,36 @@ onMounted(() => {
 
 function goToRun(runName) {
   if (runName != '') {
-    router.push({ name: 'WellsByRun', params: { runName: runName }})
+    if (searchMode.value == 'search') {
+      router.push({ name: 'WellsByRun', params: { runName: [runName] }})
+    } else {
+      compareAnotherRun(runName)
+    }
   }
 }
+
+function compareAnotherRun(supplementalRunName) {
+  if (supplementalRunName != '') {
+    let previousRuns = [...route.params.runName]
+    // Copying runName list to force vue-router to notice a change to
+    // the array
+
+    if (previousRuns.length > 5) {
+      ElMessage({
+        message: "Too many runs",
+        type: "error"
+      })
+    } else if (! previousRuns.includes(supplementalRunName)) {
+      previousRuns.push(supplementalRunName)
+      console.log(`Now ${previousRuns}`)
+      router.push({ name: 'WellsByRun', params: { runName: previousRuns }})
+    }
+  }
+}
+
+function notInWellsByRun() {
+  return route.name == 'WellsByRun' ? false : true
+}
 </script>
 
 <template>
@@ -67,8 +98,20 @@ function goToRun(runName) {
       <el-link type="primary" href="/ui/login">Login</el-link>
       <el-link type="primary" :href="logout_redirect_url">Logout</el-link>
 
-      <el-input v-model="input" placeholder="Run Name" @change="goToRun"/>
-      <el-icon><Search-icon @click="goToRun(input)"/></el-icon>
+      <el-input v-model="input" placeholder="Run Name" @change="goToRun">
+        <template #prepend>
+          <el-tooltip content="Top center" placement="top">
+            <template #content>Display one run (search)<br />Add one more run (also)</template>
+            <el-select v-model="searchMode">
+              <el-option value="search"/>
+              <el-option :disabled="notInWellsByRun()" label="also" value="also"/>
+            </el-select>
+          </el-tooltip>
+        </template>
+        <template #append>
+            <el-button :icon="Search" @click="goToRun(input)"/>
+        </template>
+      </el-input>
     </nav>
     <!-- Header END -->
 
@@ -97,7 +140,15 @@ h2 {
 }
 
 .el-input {
-  width: 15pc;
+  width: 250pt;
+}
+
+.el-select {
+  width: 70pt;
+}
+
+.button {
+  padding: 2pt;
 }
 
 .el-link .el-icon--right.el-icon {

frontend/src/components/QcView.vue

@@ -55,7 +55,7 @@
     const ssLimsStudyIds = computed(() => {
         if (props.runWell.experiment_tracking) {
             return props.runWell.experiment_tracking.study_id.join(", ")
-        } 
+        }
         return ''
     })
 
@@ -84,6 +84,13 @@
         return ''
     })
 
+    const poolName = computed(() => {
+        if (props.runWell.experiment_tracking && props.runWell.experiment_tracking.pool_name) {
+            return props.runWell.experiment_tracking.pool_name
+        }
+        return ''
+    })
+
 </script>
 
 <template>
@@ -140,7 +147,10 @@
                 <!-- TODO: Display a link to the LIMS server web page for a pool here? -->
                 <td v-else>No sample information</td>
             </tr>
-            <!-- Tag sequence info can be displayed below when the tag deplexing info is available -->
+            <tr>
+                <td>Pool name</td>
+                <td>{{ poolName }}</td>
+            </tr>
         </table>
     </div>
 

frontend/src/components/__tests__/QcView.spec.js

@@ -21,7 +21,8 @@ describe('Component renders', () => {
     sample_id: '3456',
     sample_name: 'oldSock',
     num_samples: 1,
-    library_type:['Pacbio_HiFi']
+    library_type:['Pacbio_HiFi'],
+    pool_name: "TRAC-2-3456"
   };
 
   let props_1 = {
@@ -68,6 +69,7 @@ describe('Component renders', () => {
 
     expect(wrapper.html()).toMatch(/17\/01\/2021|1\/17\/2021/) //American style dates in CI
 
+    expect(wrapper.getByText('TRAC-2-3456')).toBeDefined()
   });
 
 

frontend/src/main.js

@@ -15,7 +15,6 @@ const pinia = createPinia();
 app.use(router);
 app.use(ElementPlus); // Configure global element options here
 app.component('ExtLink', ElementPlusIconsVue.Link);
-app.component('Search-icon', ElementPlusIconsVue.Search);
 app.use(pinia);
 
 app.mount('#app');

frontend/src/router/index.js

@@ -16,7 +16,8 @@ const router = createRouter({
       component: WellsByStatus,
     },
     {
-      path: '/run/:runName',
+      path: '/run/:runName+',
+      // + means 1 or more names /run/name1/name2/name3
       name: 'WellsByRun',
       component: WellsByRun,
       props: true

frontend/src/utils/metrics.js

@@ -9,7 +9,7 @@ const CssMapping = {
 // Logical grouping of metrics to display next to each other
 const StatCategories = ({
     MachineInfo: (['binding_kit', 'movie_minutes']),
-    CCS: (['hifi_read_bases', 'hifi_read_length_mean']),
+    CCS: (['hifi_read_bases', 'hifi_read_length_mean', 'percentage_deplexed_reads', 'percentage_deplexed_bases']),
     ControlRead: (['control_num_reads', 'control_read_length_mean']),
     PolymeraseStats: (['polymerase_read_bases', 'polymerase_read_length_mean', 'p0_num', 'p1_num', 'p2_num', 'local_base_rate'])
 });