Skip to content

desihub/tutorials

Repository files navigation

DESI tutorials

Welcome to the repository designed for tutorials that guide you through the process of simulating and handling DESI data. A host of detailed tutorials can also be found across various DESI code repositories. However, this particular space serves primarily as a platform for:

  • Tutorials that were demonstrated at specific DESI meetings, each segregated under directories marked by the date of the meeting. For example, meetings/Dec2020.
  • Tutorials designed around wide-ranging subjects (like onskydata) or specific packages (such as fiberassign, redrock).

Table of Contents

Environment setup

The tutorials presented here utilize a Jupyter server at NERSC. This facility enables users to access all DESI data at NERSC and work with pre-installed DESI code without necessitating any local downloads or installations.

Please note, only the tutorials located under getting_started are designed to be compatible for non-DESI NERSC users. The remainder of the tutorials demand access to DESI's private data.

To commence, you will need to create a NERSC account. DESI users can follow the instructions detailed on the DESI wiki.

Once you have a NERSC account, you can proceed to log into Perlmutter:

ssh perlmutter.nersc.gov

The role of Jupyter "kernels" is fundamental as they establish a specific set of code versions for utilization. For those who are part of the DESI collaboration, detailed guidance can be found on the collaboration wiki.

If you're not affiliated with the DESI collaboration, please refer to the instructions provided in this repository for further guidance.

To procure a copy of the tutorials on perlmutter.nersc.gov, execute:

mkdir -p $HOME/desi/git
cd $HOME/desi/git
git clone https://github.com/desihub/tutorials

Navigate to https://jupyter.nersc.gov and sign in using your credentials and OTP (One Time Password). Then, select the Perlmutter Shared CPU Node "Start" button. Using the file browser on the left, navigate to the location where you cloned the tutorials repository. ($HOME/desi/git/tutorials in the above commands). Select a tutorial to begin your journey.

If clicking on a tutorial prompts a "Select Kernel" option from a dropdown list, it indicates that the default tutorial kernel on GitHub is not among the kernels already installed on your system. The system is thus asking you to choose from the kernels you do have installed. While most tutorials are compatible with the latest release (for instance, 23.1), some tutorials may require a specific release, which will be indicated at the top of the tutorial. If this is the case, return to perlmutter.nersc.gov, run

source $CFS/desi/software/desi_environment.sh main
$DESIMODULES/install_jupyter_kernel.sh VERSION

for the necessary VERSION, and refresh the Jupyter page.

Menu of tutorials

This repository hosts a selection of tutorials, including:

These links allow you to browse the results of these tutorials from GitHub; to run them yourself follow the instructions in the "Getting started" section above.

Other tutorials

Tutorials hosted in other packages include (but haven't been recently vetted):

External databases

In addition to the database access through NERSC, one can use the following external database services.

Astro Data Lab

The Astro Data Lab serves a copy of the DESI EDR database as desi_edr. This is accessible to users without a NERSC account. Various modes of data access are described here. In particular, there is a Table Access Protocol (TAP) handle that provides a convenient access layer for the DESI catalog database tables. TAP-aware clients (such as TOPCAT) can point to https://datalab.noirlab.edu/tap, select the desi_edr database, and see the database tables and descriptions.

Descriptions of the associated tables can also be found in the Data Lab table browser and on the DESI Data Documentation Database page.

SPARCL

The SPectra Analysis & Retrievable Catalog Lab (SPARCL) contains DESI EDR spectra that were coadded per healpix. It is a searchable database that can be used via the Astro Data Lab JupyterLab Notebook. Alternatively, the Python client can be installed locally using:

pip install sparclclient

The client can be loaded within a Python session or program via:

>>> from sparcl.client import SparclClient 
>>> client = SparclClient()

There are instructions and useful examples in the How-to-use-SPARCL tutorial notebook.

Acknowledgments

Any use of DESI data whether via NERSC or external databases requires DESI Data Acknowledgments.

For authors of tutorials

Include at the beginning of the tutorial what is needed as a prerequisite for running the tutorial, e.g. specific codes, environment variables, datasets. It's OK to link elsewhere for detailed instructions (e.g. for how to install DESI code in general).