Updates: Move quantum-espresso to conda, add aiida-plugins, add aiida…

… example scripts (#212

- Update build tool to VirtualBox version 7 (https://www.virtualbox.org/wiki/Changelog-7.0)
- Install quantum-espresso via Conda (rather than direct compilation)
- Add more AiiDA plugins/codes, compatible with aiida-core v2, see #211
- Add `verdi code test` run, after code creation, to test it is working correctly
- Add `aiida-examples` scripts, these run a basic calculation example (with MPI), and test it completes correctly
  - currently only qe and nwchem, then more will be added later

.pre-commit-config.yaml

@@ -18,3 +18,8 @@ repos:
   hooks:
   - id: ansible-lint
     args: [--exclude, roles/]
+
+- repo: https://github.com/psf/black
+  rev: 23.1.0
+  hooks:
+  - id: black

docs/conf.py

@@ -20,7 +20,14 @@
 
 # -- General configuration ---------------------------------------------------
 
-extensions = ["myst_parser", "sphinx_design", "ablog", "sphinx.ext.intersphinx", "sphinxext.rediraffe", "sphinx_subfigure"]
+extensions = [
+    "myst_parser",
+    "sphinx_design",
+    "ablog",
+    "sphinx.ext.intersphinx",
+    "sphinxext.rediraffe",
+    "sphinx_subfigure",
+]
 
 myst_enable_extensions = ["colon_fence", "deflist", "html_image"]
 
@@ -30,7 +37,13 @@
 post_redirect_refresh = 1
 post_auto_excerpt = 2
 fontawesome_included = True
-html_sidebars = {"releases/index": ['ablog/tagcloud.html', 'ablog/archives.html', 'sbt-sidebar-nav.html']}
+html_sidebars = {
+    "releases/index": [
+        "ablog/tagcloud.html",
+        "ablog/archives.html",
+        "sbt-sidebar-nav.html",
+    ]
+}
 
 # The master toctree document.
 master_doc = "index"
@@ -63,4 +76,4 @@
     "use_edit_page_button": True,
 }
 
-rediraffe_redirects = 'redirects.txt'
+rediraffe_redirects = "redirects.txt"

docs/maintainers/release.md

@@ -61,14 +61,18 @@ tox -e update-vagrant
 Follow the [desktop build instructions](../developers/build-vagrant.md).
 
 :::{note}
-Manual modifications required, as of QM 20.03.1:
+Manual modifications required, as of QM 23.04.01, desktop shortcuts mut be manually enabled:
 
-* Double-click on the Desktop symbols to show the icons (see [#150](https://github.com/marvel-nccr/quantum-mobile/issues/150)) or just run in the VM terminal (press F5 after to refresh):
+- First install the file with <https://manpages.ubuntu.com/manpages/focal/en/man1/desktop-file-edit.1.html>:
 
-  ```bash
-  for FILE in ${HOME}/Desktop/*.desktop; do gio set "$FILE" "metadata::trusted" yes; done
+  ```console
+  $ sudo desktop-file-install --delete-original ~/Desktop/homepage.desktop
+  $ sudo desktop-file-install --delete-original ~/Desktop/jupyterlab.desktop
   ```
 
+- Now navigate to the `/usr/share/applications` in the folder app, find the files and drag/drop them to the Desktop
+- Right click on the file on the Desktop and select `Allow Launching`
+
 :::
 
 ### Cloud Edition
@@ -77,7 +81,7 @@ Follow the [cloud build instructions](../developers/build-cloud.md).
 
 * For the server to build the VM, choose e.g. 2 CPUs with 4GB of RAM, with at least 16GB of storage.
   * Your configuration may also be the standard configuration shown to users creating a VM using your image
-  * Currently Ubuntu Server 18.04 LTS
+  * Currently Ubuntu Server 20.04 LTS
   * Expose ports as listed in instructions
   * You won't need it for long, so price for CPU/RAM is of no concern
 * Exemplary metadata:

docs/releases/versions/23.04.03.md

@@ -0,0 +1,138 @@
+---
+blogpost: true
+author: Quantum Mobile
+date: 2023-04-03
+category: Full
+tags: desktop,docker
+---
+
+# v23.04.03
+
+A Virtual Machine for Computational Materials Science
+
+## Installation instructions
+
+Get Quantum Mobile running on your computer in three simple steps:
+
+ 1. Download virtual machine image (5.9 GB)
+    - URL: <https://drive.google.com/file/d/1dhBANfO7KdaFWu_rwkBduKYNjofu8k33/view?usp=sharing>
+    - Filename: `quantum_mobile_23.04.03.ova`
+    - MD5 hash: `3d489840c77b27afe1d2367abb5ac44d`
+ 2. Install Virtual Box 6.1.6 or later (see <https://www.virtualbox.org>)
+ 3. Import virtual machine image into Virtualbox (15.7 GB)
+    File => Import Appliance
+
+Login credentials: username: `max`, password: `moritz`.  
+The default configuration of `2` cores and `1536` MB RAM can be adjusted in the VM settings.
+
+Alternatively, this Quantum Mobile is available as a Docker image: <https://hub.docker.com/r/marvelnccr/quantum-mobile>
+
+## Contact
+
+For issues encountered during installation, please first consult the [FAQ page](https://github.com/marvel-nccr/quantum-mobile/wiki/Frequently-Asked-Questions#virtualbox-installationstartup-issues).
+
+Please direct inquiries regarding Quantum Mobile to the [AiiDA mailinglist](http://www.aiida.net/mailing-list/)
+
+## Changelog
+
+- Update build tool to VirtualBox version 7 (https://www.virtualbox.org/wiki/Changelog-7.0)
+- Install quantum-espresso via Conda (rather than direct compilation)
+- Add more AiiDA plugins/codes, compatible with aiida-core v2 (see <https://github.com/marvel-nccr/quantum-mobile/issues/211>)
+- Add `verdi code test` run, after code creation, to test it is working correctly
+- Add `aiida-examples` scripts folder, these run a basic calculation example (with MPI), and test it completes correctly
+
+### Build Process
+
+- OS: `MacOSX`
+- Ansible: `2.10.17`
+- Vagrant: `2.3.4`
+- Virtualbox: `7.0.6r155176`
+- Base VM Image: `bento/ubuntu-20.04`
+
+### Software Summary
+
+```ini
+[Quantum Mobile]
+version = 23.04.03
+Operating System = Ubuntu 20.04.6 LTS
+Login credentials = max / moritz
+
+[Apt packages]
+grace = 1:5.1.25-7build1
+xcrysden = 1.6.2-3build1
+default-jre = 2:1.11-72
+rabbitmq-server = 3.8.2-0ubuntu1.4
+postgresql-client = 12+214ubuntu0.1
+
+[Conda 'aiida' environment]
+aiida-abinit = 0.4.0-pyhd8ed1ab_0@conda-forge
+aiida-core = 2.2.2-pyh1a96a4e_1@conda-forge
+aiida-core.notebook = 2.2.2-pyh1a96a4e_1@conda-forge
+aiida-cp2k = 2.0.0-pyhd8ed1ab_1@conda-forge
+aiida-nwchem = 2.1.0-pyhd8ed1ab_0@conda-forge
+aiida-pseudo = 1.0.0-pyhd8ed1ab_0@conda-forge
+aiida-quantumespresso = 4.2.0-pyhd8ed1ab_0@conda-forge
+aiida-siesta = 2.0.0-pyhd8ed1ab_0@conda-forge
+ipykernel = 6.22.0-pyh210e3f2_0@conda-forge
+jupyterlab = 3.5.3-pyhd8ed1ab_0@conda-forge
+jupyterlab-spellchecker = 0.7.3-pyhd8ed1ab_0@conda-forge
+jupyterlab-tour = 3.1.4-pyhd8ed1ab_0@conda-forge
+mamba_gator = 5.2.0-pyhd8ed1ab_0@conda-forge
+pip = 23.0.1-pyhd8ed1ab_0@conda-forge
+python = 3.9.16-h2782a2a_0_cpython@conda-forge
+
+[Conda 'abinit' environment]
+abinit = 9.8.3-hd1b6b71_2@conda-forge
+libxc = 4.3.4-h86c2bf4_2@conda-forge
+mpich = 4.0.3-h846660c_100@conda-forge
+
+[Conda 'bigdft' environment]
+bigdft-suite = 1.9.3-mpi_mpich_py311h71b1498_0@conda-forge
+libxc = 4.3.4-h86c2bf4_2@conda-forge
+mpich = 4.0.3-h846660c_100@conda-forge
+
+[Conda 'cp2k' environment]
+cp2k = 9.1.0-py39_openmpi_0@conda-forge
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+
+[Conda 'fleur' environment]
+fleur = 6.1-h005d346_1@conda-forge
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+
+[Conda 'nwchem' environment]
+libxc = 5.2.3-py39hea1df8f_2@conda-forge
+nwchem = 7.0.2-py39hea0d9f8_3@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+
+[Conda 'qespresso' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+qe = 7.0-he8a42d8_1@conda-forge
+
+[Conda 'siesta' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+siesta = 4.1.5-mpi_openmpi_hfab99a0_2@conda-forge
+
+[Conda 'yambo' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+yambo = 5.0.4-h6b7a505_1@conda-forge
+
+[Conda 'wannier90' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+wannier90 = 3.1.0-hb97063f_2@conda-forge
+
+[Conda 'visualise' environment]
+cif2cell = 2.0.0a3-pyhd8ed1ab_0@conda-forge
+gnuplot = 5.4.5-h142138f_1@conda-forge
+jmol = 14.32.10-ha770c72_0@conda-forge
+python = 3.9.16-h2782a2a_0_cpython@conda-forge
+
+[Pseudopotentials]
+SSSP/PBE/efficiency/1.1 = /usr/local/share/pseudo_sssp_PBE_efficiency_1.1
+DOJO/PBE/FR/standard/0.4/psml = /usr/local/share/pseudo_dojo_PBE_FR_standard_0.4_psml
+```

inventory.yml

@@ -11,7 +11,7 @@ all:
   vars:
     # VM Metadata
     vm_name: "Quantum Mobile"
-    vm_version: "23.03.01"
+    vm_version: "23.04.03"
     vm_description: "A Virtual Machine for Computational Materials Science"
     vm_url: "https://github.com/marvel-nccr/marvel-virtualmachine"
     vm_author: "MARVEL NCCR and MaX CoE"

local/modules/apt_show.py

@@ -54,7 +54,9 @@ def _main():
     module = AnsibleModule(argument_spec={"name": {"required": True, "type": "str"}})
     if not module.params["name"]:
         module.fail_json(msg="Package name is empty")
-    _, stdout, _ = module.run_command(["apt", "show"] + module.params["name"].split(), check_rc=True)
+    _, stdout, _ = module.run_command(
+        ["apt", "show"] + module.params["name"].split(), check_rc=True
+    )
     data = {}
     lines = stdout.splitlines()
     index = 0

local/modules/verdi_code.py

@@ -34,6 +34,9 @@
     append_text:
         description: The append text
         required: false
+    run_test:
+        description: Whether to run verdi code test after creating the code (default: true)
+        required: false
 """
 
 EXAMPLES = """
@@ -66,6 +69,7 @@ def _main():
             "description": {"required": False, "type": "str"},
             "prepend_text": {"required": False, "type": "str"},
             "append_text": {"required": False, "type": "str"},
+            "run_test": {"required": False, "type": "bool", "default": True},
         },
     )
     verdi = module.params["verdi"].split() + ["--profile", module.params["profile"]]
@@ -93,6 +97,26 @@ def _main():
 
     _, stdout, _ = module.run_command(command, check_rc=True)
 
+    if "run_test" in module.params and not module.params["run_test"]:
+        # if not running test, then just exit
+        module.exit_json(changed=True, stdout=stdout)
+
+    # test the code actually works, e.g. that the executable exists
+    # otherwise delete
+    test_rc, test_stdout, test_stderr = module.run_command(
+        verdi + ["code", "test", module.params["label"]]
+    )
+    if test_rc != 0:
+        module.run_command(
+            verdi + ["code", "delete", "--force", module.params["label"]]
+        )
+        module.fail_json(
+            msg="verdi code test failed",
+            rc=test_rc,
+            stdout=test_stdout,
+            stderr=test_stderr,
+        )
+
     module.exit_json(changed=True, stdout=stdout)
 
 

local/tasks/aiida-example-run.yml

@@ -0,0 +1,14 @@
+- name: "Copy aiida script: {{ aiida_script }}"
+  ansible.builtin.copy:
+    src: aiida_run_scripts/{{ aiida_script }}
+    dest: "{{ aiida_script_path }}"
+    owner: "{{ vm_user }}"
+    group: "{{ vm_user }}"
+  register: aiida_script_copy
+
+- name: "Run aiida script: {{ aiida_script }}"
+  when: aiida_script_copy.changed
+  shell: |
+    {{ aiida_verdi_cmd }} run {{ aiida_script_path }}
+
+# TODO handle failures (delete the script?)

local/tasks/aiida-examples.yml

@@ -0,0 +1,23 @@
+- name: "Expand {{ aiida_examples_folder }}"
+  become: true
+  become_user: "{{ vm_user }}"
+  chrisjsewell.conda.expandpath:
+    path: "{{ aiida_examples_folder }}"
+  register: aiida_examples_dir
+
+- name: "Create {{ aiida_examples_dir.path }} folder"
+  ansible.builtin.file:
+    path: "{{ aiida_examples_dir.path }}"
+    state: directory
+
+# TODO get list of all files in aiida_run_scripts
+
+- name: "Copy and run aiida script"
+  include_tasks: aiida-example-run.yml
+  vars:
+    aiida_script_path: "{{ aiida_examples_dir.path }}/{{ aiida_script }}"
+  loop:
+  - qe_pw.py
+  - nwchem.py
+  loop_control:
+    loop_var: aiida_script

local/tasks/aiida-pseudo-dojo.yml

@@ -0,0 +1,53 @@
+# download pseudopotials and install them in aiida
+
+- set_fact:
+    pseudo_folder: "{{ aiida_pseudo_base_folder }}/pseudo_dojo_{{ aiida_pseudo_functional }}_{{ aiida_pseudo_relativistic }}_{{ aiida_pseudo_protocol }}_{{ aiida_pseudo_version }}_{{ aiida_pseudo_format }}"
+
+- name: create the folder to download
+  become: true
+  become_user: "{{ root_user }}"
+  file:
+    path: "{{ pseudo_folder }}"
+    state: directory
+    owner: "{{ vm_user }}"
+    group: "{{ vm_user }}"
+    mode: 0755
+
+- name: Download the archive to the download location
+  become: true
+  become_user: "{{ vm_user }}"
+  shell: |
+    {{ aiida_pseudo_cmd }} \
+    install pseudo-dojo --traceback --download-only \
+    --version {{ aiida_pseudo_version | quote }} \
+    --functional {{ aiida_pseudo_functional | quote }} \
+    --protocol {{ aiida_pseudo_protocol | quote }} \
+    --relativistic {{ aiida_pseudo_relativistic | quote }} \
+    --pseudo-format {{ aiida_pseudo_format | quote }} \
+  args:
+    chdir: "{{ pseudo_folder }}"
+    creates: "archive.tar.gz"
+
+- name: Add the archive to the AiiDA profile
+  become: true
+  become_user: "{{ vm_user }}"
+  shell: |
+    {{ aiida_pseudo_cmd }} \
+    --profile {{ aiida_pseudo_profile | quote }} \
+    install pseudo-dojo --traceback \
+    --version {{ aiida_pseudo_version | quote }} \
+    --functional {{ aiida_pseudo_functional | quote }} \
+    --protocol {{ aiida_pseudo_protocol | quote }} \
+    --relativistic {{ aiida_pseudo_relativistic | quote }} \
+    --pseudo-format {{ aiida_pseudo_format | quote }} \
+  register: result
+  failed_when: result.rc != 0 and "already installed" not in result.stderr
+  changed_when: '"already installed" not in result.stderr'
+
+- name: Document potential
+  include_role:
+    name: release_notes
+  vars:
+    section: "Pseudopotentials"
+    option: "DOJO/{{ aiida_pseudo_functional }}/{{ aiida_pseudo_relativistic }}/{{ aiida_pseudo_protocol }}/{{ aiida_pseudo_version }}/{{ aiida_pseudo_format }}"
+    value: "{{ pseudo_folder }}"