Skip to content

Latest commit

 

History

History
462 lines (349 loc) · 22 KB

INSTALL.md

File metadata and controls

462 lines (349 loc) · 22 KB
Raw

Installation Guide

Instruction to install PISA on your local machine

  1. Install the essential dependencies

    1. Anaconda (https://www.anaconda.com/) 
      wget -nc https://repo.anaconda.com/archive/Anaconda3-2023.07-2-Linux-x86_64.sh

bash Anaconda3-2023.07-2-Linux-x86_64.sh
    2. pip (Python package installer) 
      conda install pip
    3. git 
      sudo apt install git

    Other required libraries (listed below) will be installed automatically during the setup (listed in setup.py).

    • fast-histogram >= 0.10
    • h5py
    • iminuit >= 2
    • line_profiler
    • llvmlite
    • matplotlib >= 3.0
    • nlopt
    • numba >= 0.53
    • numpy >=1.17, < 1.23
    • pandas
    • pint <=0.19
    • py-cpuinfo
    • pyarrow
    • scikit-learn <= 1.1.2
    • scipy >= 1.6
    • simplejson == 3.18.4
    • sympy
    • tables
    • tabulate
    • tqdm
    • uncertainties
    • kiwisolver >= 1.3.1
    • cycler >= 0.10
    • packaging >= 20.0
    • fonttools >= 4.22.0
    • python-dateutil >= 2.7
    • pillow >= 8
    • contourpy >= 1.0.1
    • pyparsing >= 2.3.1
    • threadpoolctl >= 2.0.0
    • joblib >= 1.0.0
    • numexpr
    • pytz >= 2020.1
    • tzdata >= 2022.1
    • mpmath >= 0.19
    • blosc2 >= 2.3.0
    • future
    • ndindex >= 1.4
    • msgpack
    • six >= 1.5

  2. Create conda environment/ virtual environment for the installation of PISA. Assume that the name of the environment is pisa_env you can choose your preferred version of python >= 3.10

conda create -n pisa_env python=3.10
  1. Activate the newly created environment (You can see the name of your environment in the bash shell after the activation)
conda activate pisa_env
  1. Clone the PISA repository from github (https://github.com/icecube/pisa.git)
git clone https://github.com/icecube/pisa.git
  1. Install PISA with the following command
pip install -e pisa

Arguments:

  • pisa directory of source code
  • -e (editable) Installs from source located at $PISA (Installed directory - pisa) and allows for changes to the source code within to be immediately propagated to your Python installation.
  • -vvv Be maximally verbose during the install. You'll see lots of messages, including warnings that are irrelevant, but if your installation fails, it's easiest to debug if you use -vvv
  • [develop] Specify optional dependency groups. You can omit any or all of these if your system does not support them or if you do not need them.

Test your installation by running a simple script:

  • Activate your python environment (pisa_env)

    conda activate pisa_env

  • Start python interpretator in terminal

    python

  • Import Pipline modelue and matplotlib.pyplot

    from pisa.core import Pipeline
import matplotlib.pyplot as plt
    << PISA is running in single precision (FP32) mode; numba is running on CPU (single core) >>

  • Load an example pipeline

    template_maker = Pipeline("settings/pipeline/osc_example.cfg")

  • Run the pipeline

    template_maker.run()

  • Get the oscillation probabilities as a map

    outputs = template_maker.data.get_mapset('prob_mu')

  • Plot the oscillogram

    fig, axes = plt.subplots(figsize=(18, 5), ncols=3)
outputs['nue_cc'].plot(ax=axes[0], cmap='RdYlBu_r', vmin=0, vmax=1);
outputs['numu_cc'].plot(ax=axes[1], cmap='RdYlBu_r', vmin=0, vmax=1);
outputs['nutau_cc'].plot(ax=axes[2], cmap='RdYlBu_r', vmin=0, vmax=1);

Instructions to install PISA on Madison working group servers cobalts (current guide from August 2023):

Assuming you already are in the directory where you want to store fridge/pisa source files and the python virtualenv and build pisa. You also need access to github through your account.

Clone into the fridge and pisa (ideally your own fork):

git clone git@github.com:icecube/wg-oscillations-fridge.git ./fridge

git clone git@github.com:USERNAME/pisa.git ./pisa

Load cvmfs environment:

unset OS_ARCH; eval `/cvmfs/icecube.opensciencegrid.org/py3-v4.2.1/setup.sh`

which python should now point to /cvmfs/icecube.opensciencegrid.org/py3-v4.2.1/RHEL_7_x86_64/bin/python

Note: It's not tested whether this works with a cvmfs version newer than py3-v4.2.1.

Create virtual environment:

python -m venv ./YOUR_PISA_PY3_VENV

Start the virtual environment and install pisa from source:

source ./YOUR_PISA_PY3_VENV/bin/activate

(should now show that you are in the environment)

pip install -e pisa

Modify your environment by adding lines to ./YOUR_PISA_PY3_VENV/bin/activate

Every time you want to use pisa now, you need to activate your virtual environment by running source ./YOUR_PISA_PY3_VENV/bin/activate. In order to set some useful environment variables you might want to add the following lines (or more if needed) to the end of the ./YOUR_PISA_PY3_VENV/bin/activate script:

# PISA source
export PISA="/data/user/USERNAME/PATH_TO_PISA"

# set some custom environment variables and load fridge
export PISA_RESOURCES="/data/user/USERNAME/PATH_TO_FRIDGE/analysis/common"
export PISA_RESOURCES=$PISA_RESOURCES:"/data/user/USERNAME/PATH_TO_FRIDGE/analysis"

source "/data/user/USERNAME/PATH_TO_FRIDGE/setup_fridge.sh"

# Add this project to the python path
export PYTHONPATH=$FRIDGE_DIR:$PYTHONPATH

# Madison
export PISA_RESOURCES=/data/ana/LE:$PISA_RESOURCES
export PISA_RESOURCES=/data/ana/BSM/HNL/:$PISA_RESOURCES

export PISA_RESOURCES=$FRIDGE_DIR:$FRIDGE_DIR/analysis:$FRIDGE_DIR/analysis/YOUR_ANALYSIS/settings:$FRIDGE_DIR/analysis/YOUR_ANALYSIS/analysis:$FRIDGE_DIR/analysis/common:$PISA_RESOURCES

export PISA_CACHE=~/cache/
export PISA_FTYPE=fp64
export HDF5_USE_FILE_LOCKING='FALSE'

Install any additional packages that you might want

pip install PACKAGE (for example jupyter)

Quickstart (old guide)

Note that terminal commands below are intended for the bash shell. You'll have to translate if you use a different shell.

  1. Obtain a github user ID if you don’t have one already
    https://github.com
  2. Fork PISA on github so you have your own copy to work from
    https://github.com/IceCubeOpenSource/pisa#fork-destination-box
  3. (optional) Set up shared-key ssh access to github so you don’t have to enter passwords
    https://help.github.com/articles/connecting-to-github-with-ssh
  4. In your terminal, define a directory for PISA sourcecode to live in. For example:
    export PISA="~/src/pisa"
    • Add this line to your ~/.bashrc file so you can refer to the $PISA variable without doing this everytime.
  5. Create the directory
    mkdir -p $PISA
  6. Clone the PISA repo to your local computer (at command line)
    • If you set up shared-key auth above
      git clone git@github.com:<YOUR GITHUB USER ID HERE>/pisa.git $PISA
    • Otherwise
      git clone https://github.com/<YOUR GITHUB USER ID HERE>/pisa.git $PISA
  7. Install the Python 3.7 / 64 bit Anaconda or Miniconda python distribution for either Mac or Linux (as your user, not as root), if you don’t have it already
  8. Create a new conda environment, ideally with a python version compatible with the python requirements below (cf. conda's getting started guide).
  9. Active your new conda environment.
  10. Install PISA including optional packages and development tools (develop), if desired
    pip install -e $PISA[develop] -vvv
  11. Run a quick test: generate templates in the staged mode
    $PISA/pisa/core/pipeline.py --pipeline settings/pipeline/example.cfg --outdir /tmp/pipeline_output --intermediate --pdf -v

See github.com/IceCubeOpenSource/pisa/wiki/installation_specific_examples for users' recipes for installing PISA under various circumstances. Please add notes there and/or add your own recipe if your encounter a unique installation issue. Also, for instructions on running PISA on Open Science Grid (OSG) nodes, see github.com/IceCubeOpenSource/pisa/wiki/Running-PISA-on-GRID-nodes-with-access-to-CVMFS

The following sections delve into deeper detail on installing PISA.

Python Distributions

Obtaining Python and Python packages, and handling interdependencies in those packages tends to be easiest if you use a Python distribution, such as Anaconda or Canopy. Although the selection of maintained packages is smaller than if you use the pip command to obtain packages from the Python Package Index (PyPi), you can still use pip with these distributions.

The other advantage to these distributions is that they easily install without system administrative privileges (and install in a user directory) and come with the non-Python binary libraries upon which many Python modules rely, making them ideal for setup on e.g. clusters.

  • Note: Make sure that your PATH variable points to e.g. <anaconda_install_dr>/bin and not your system Python directory. To check this, type: echo $PATH; to udpate it, add export PATH=<anaconda_install_dir>/bin:$PATH to your .bashrc file.
  • Python 3.7.x can also be found from the Python website python.org/downloads or pre-packaged for almost any OS.

Required Dependencies

To install PISA, you'll need to have the following non-python requirements. Note that these are not installed automatically, and you must install them yourself prior to installing PISA. Also note that Python, HDF5, and pip support come pre-packaged or as conda-installable packages in the Anaconda Python distribution.

  • python — version 3.x
    • Anaconda: built in
  • pip version >= 1.8 required
    • Anaconda:
      conda install pip
  • git
    • In Ubuntu,
      sudo apt install git
  • hdf5 — install with --enable-cxx option
    • In Ubuntu,
      sudo apt install libhdf5-10
  • llvm Compiler needed by Numba. This is automatically installed in Anaconda alongside numba.
    • Anaconda
      conda install numba=0.45.1

Required Python modules that are installed automatically when you use the pip command detailed later:

  • decorator
  • h5py
  • iminuit: Python interface to the MINUIT2 C++ package, used for proper covariance matrices during minimization
  • kde
    • You can install the kde module manually if it fails to install automatically:
      • Including CUDA support:
        pip install git+https://github.com/icecubeopensource/kde.git#egg=kde[cuda]
      • Without CUDA support:
        pip install git+https://github.com/icecubeopensource/kde.git#egg=kde
  • line_profiler: detailed profiling output
    • if automatic pip installation of line_profiler fails, you may want to try conda install line_profiler if you are using anaconda
  • matplotlib>=3.0 >= 3.0 required
  • numba==0.53.1 Just-in-time compilation of decorated Python functions to native machine code via LLVM. This package is required to use PISA pi; also in cake it can accelerate certain routines significantly. If not using Anaconda to install, you must have LLVM installed already on your system (see above).
  • numpy version >= 1.17 required
  • pint>=0.8.1 >= 0.8.1 required
    • if automatic pip installation of pint fails, you may want to try conda install pint if you are using anaconda
  • scipy version >= 0.17 required
  • setuptools version >= 18.5 required
  • simplejson version >= 3.2.0 required
  • tables
  • uncertainties
  • py-cpuinfo retrieve detailed CPU and architecture info for documenting in tests / obtaining results
  • sympy Used for testing nsi_params.py

Optional Dependencies

Optional dependencies. Some of these must be installed manually prior to installing PISA, and some will be installed automatically by pip, and this seems to vary from system to system. Therefore you can first try to run the installation, and just install whatever pip says it needed, or just use apt, pip, and/or conda to install the below before running the PISA installation.

  • LeptonWeighter Required for the data.licloader_weighter service.
  • MCEq Required for flux.mceq service.
  • daemonflux Recuired for flux.daemon_flux service.
  • nuSQuiDS Required for osc.nusquids service.
  • pandas Required for datarelease (csv) stages.
  • OpenMP Intra-process parallelization to accelerate code on on multi-core/multi-CPU computers.
    • Available from your compiler: gcc supports OpenMP 4.0 and Clang >= 3.8.0 supports OpenMP 3.1. Either version of OpenMP should work, but Clang has yet to be tested for its OpenMP support.
  • Photospline Required for the flux.airs service.
  • Pylint: Static code checker and style analyzer for Python code. Note that our (more or less enforced) coding conventions are codified in the pylintrc file in PISA, which will automatically be found and used by Pylint when running on code within a PISA package.
    • Installed alongside PISA if you specify option ['develop'] to pip
  • recommonmark Translator to allow markdown docs/docstrings to be used; plugin for Sphinx. (Required to compile PISA's documentation.)
    • Installed alongside PISA if you specify option ['develop'] to pip
  • ROOT >= 6.12.04 with PyROOT Necessary for xsec.genie, unfold.roounfold and absorption.pi_earth_absorption services, and to read ROOT cross section files in the crossSections utils module. Due to a bug in ROOT's python support (documented here #430), you need at least version 6.12.04 of ROOT.
  • Sphinx version >= 1.3
    • Installed alongside PISA if you specify option ['develop'] to pip
  • versioneer Automatically get versions from git and make these embeddable and usable in code. Note that the install process is unique since it first places versioneer.py in the PISA root directory, and then updates source files within the repository to provide static and dynamic version info.
    • Installed alongside PISA if you specify option ['develop'] to pip
  • black Format your Python code, automatically, with typically very nice results!
    • Note this only works in Python3

Obtain PISA sourcecode

Develop PISA: Fork then clone

If you wish to modify PISA and contribute your code changes back to the PISA project (highly recommended!), fork IceCubeOpenSource/pisa from Github. (How to work with the cake branch of PISA will be detailed below.)

Forking creates your own version of PISA within your Github account. You can freely create your own branch, modify the code, and then add and commit changes to that branch within your fork of PISA. When you want to share your changes with IceCubeOpenSource/pisa, you can then submit a pull request to IceCubeOpenSource/pisa which can be merged by the PISA administrator (after the code is reviewed and tested, of course).

  • Navigate to the PISA github page and fork the repository by clicking on the fork button.
  • Clone the repository into the $PISA directory via one of the following commands (<github username> is your Github username):
    • either SSH access to repo:
      git clone git@github.com:<github username>/pisa.git $PISA
    • or HTTPS access to repo:
      git clone https://github.com/<github username>/pisa.git $PISA

Using but not developing PISA: Just clone

If you just wish to pull changes from github (and not submit any changes back), you can just clone the sourcecode without creating a fork of the project.

  • Clone the repository into the $PISA directory via one of the following commands:
    • either SSH access to repo:
      git clone git@github.com:IceCubeOpenSource/pisa.git $PISA
    • or HTTPS access to repo:
      git clone https://github.com/IceCubeOpenSource/pisa.git $PISA

Ensure a clean install using virtualenv or conda env

It is absolutely discouraged to install PISA as a root (privileged) user. PISA is not vetted for security vulnerabilities, so should always be installed and run as a regular (unprivileged) user.

It is suggested (but not required) that you install PISA within a virtual environment (or in a conda env if you're using Anaconda or Miniconda Python distributions). This minimizes cross-contamination by PISA of a system-wide (or other) Python installation with conflicting required package versions, guarantees that you can install PISA as an unprivileged user, guarantees that PISA's dependencies are met, and allows for multiple versions of PISA to be installed simultaneously (each in a different virtualenv / conda env).

Note that it is also discouraged, but you can install PISA as an unprivileged user using your system-wide Python install with the --user option to pip. This is not quite as clean as a virtual environment, and the issue with coflicting package dependency versions remains.

Install PISA

pip install -e $PISA[develop] -vvv

Explanation:

  • First, note that this is not run as administrator. It is discouraged to do so (and has not been tested this way).
  • -e $PISA (or equivalently, --editable $PISA): Installs from source located at $PISA and allows for changes to the source code within to be immediately propagated to your Python installation. Within the Python library tree, all files under pisa are links to your source code, so changes within your source are seen directly by the Python installation. Note that major changes to your source code (file names or directory structure changing) will require re-installation, though, for the links to be updated (see below for the command for re-installing).
  • [develop] Specify optional dependency groups. You can omit any or all of these if your system does not support them or if you do not need them.
  • -vvv Be maximally verbose during the install. You'll see lots of messages, including warnings that are irrelevant, but if your installation fails, it's easiest to debug if you use -vvv.
  • If a specific compiler is set by the CC environment variable (export CC=<path>), it will be used; otherwise, the cc command will be run on the system for compiling C-code.

Notes:

  • You can work with your installation using the usual git commands (pull, push, etc.). However, these won't recompile any of the extension (i.e. pyx, C/C++) libraries. See below for how to reinstall PISA when you need these to recompile.

Reinstall PISA

Sometimes a change within PISA requires re-installation (particularly if a compiled module changes, the below forces re-compilation).

pip install -e $PISA[develop] --force-reinstall -vvv

Note that if files change names or locations, though, the above can still not be enough. In this case, the old files have to be removed manually (along with any associated .pyc files, as Python will use these even if the .py files have been removed).

Compile the documentation

To compile a new version of the documentation to html (pdf and other formats are available by replacing html with pdf, etc.):

cd $PISA && sphinx-apidoc -f -o docs/source pisa

In case code structure has changed, rebuild the apidoc by executing

cd $PISA/docs && make html

Test PISA

Unit Tests

Throughout the codebase there are test_*.py files and test_* functions within various *.py files that represent unit tests. Unit tests are designed to ensure that the basic mechanisms of objects' functionality work.

These are all run, plus additional tests (takes about 15-20 minutes on a laptop) with the command

$PISA/pisa_tests/test_command_lines.sh

Run a basic analysis

To make sure that an analysis can be run, try running an Asimov analysis of neutrino mass ordering (NMO) with the following (this takes about one minute on a laptop; note, though, that the result is not terribly accurate due to the use of coarse binning and low Monte Carlo statistics):

export PISA_FTYPE=fp64
$PISA/pisa/analysis/hypo_testing.py --logdir /tmp/nmo_test analysis \
    --h0-pipeline settings/pipeline/example.cfg \
    --h0-param-selections="ih" \
    --h1-param-selections="nh" \
    --data-param-selections="nh" \
    --data-is-mc \
    --min-method slsqp \
    --metric=chi2 \
    --pprint -v

The above command sets the null hypothesis (h0) to be the inverted hierarchy (ih) and the hypothesis to be tested (h1) to the normal hierarchy (nh). Meanwhile, the Asimov dataset is derived from the normal hierarchy.

The significance for distinguishing NH from IH in this case (with the crude but fast settings specified) is shown by typing the follwoing command (which should output something close to 4.3):

hypo_testing_postprocess.py --asimov --detector "pingu_v39" --dir /tmp/nmo_test/hypo*