Skip to content

Commit

Permalink
refactor(docs): move from reST to Markdown (reanahub#34)
Browse files Browse the repository at this point in the history
Convert docs from reStructuredText to Markdown so that the changelog
file is compatible with Release Please.
  • Loading branch information
mdonadoni committed Feb 8, 2024
1 parent 32bf7df commit 1d8bbb3
Show file tree
Hide file tree
Showing 8 changed files with 180 additions and 207 deletions.
8 changes: 8 additions & 0 deletions AUTHORS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Authors

The list of contributors in alphabetical order:

- [Daniel Prelipcean](https://orcid.org/0000-0002-4855-194X)
- [Marco Donadoni](https://orcid.org/0000-0003-2922-5505)
- [Tibor Simko](https://orcid.org/0000-0001-7202-5803)
- [Vladyslav Moisieienkov](https://orcid.org/0000-0001-9717-0775)
8 changes: 0 additions & 8 deletions AUTHORS.rst

This file was deleted.

7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Changelog

## 0.1.0 (2019-MM-DD)

<!-- TODO add realease date -->

- Initial public release.
15 changes: 0 additions & 15 deletions CHANGES.rst

This file was deleted.

25 changes: 10 additions & 15 deletions CONTRIBUTING.rst → CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,28 @@
Contributing
============
# Contributing

Issues
------
## Issues

Bug reports, feature requests, and other contributions are welcome. If you find
a demonstrable problem that is caused by the REANA code, please:

1. Search for `already reported problems
<https://github.com/reanahub/reana-demo-cms-reco/issues>`_.
1. Search for [already reported problems](https://github.com/reanahub/reana-demo-cms-reco/issues).
2. Check if the issue has been fixed or is still reproducible on the
latest `master` branch.
3. Create an issue, ideally with **a test case**.

Pull requests
-------------
## Pull requests

If you create a feature branch, you can run the tests to ensure that everything
is operating correctly:

.. code-block:: console
$ ./run-tests.sh
```console
$ ./run-tests.sh
```

Each pull request should preserve or increase code coverage.

Kanban
------
## Kanban

We are using Kanban technique for keeping track of ongoing tasks. Please see our
`Kanban board <https://waffle.io/reanahub/reana>`_ and look for issues that are
labelled as "ready for work".
[Kanban board](https://waffle.io/reanahub/reana) and look for issues that are
labelled as "ready for work".
120 changes: 120 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# REANA example - CMS Reconstruction

[![image](https://img.shields.io/pypi/pyversions/reana-demo-cms-reco.svg)](https://pypi.org/pypi/reana-demo-cms-reco)
[![image](https://img.shields.io/badge/discourse-forum-blue.svg)](https://forum.reana.io)
[![image](https://img.shields.io/github/license/reanahub/reana.svg)](https://github.com/reanahub/reana-demo-cms-reco/blob/master/LICENSE)

## About

This REANA reproducible analysis example demonstrates the reconstruction procedure of the
CMS collaboration from
[raw data](http://opendata.cern.ch/search?page=1&size=20&experiment=CMS&file_type=raw) to
[Analysis Object Data (AOD)](https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookDataFormats#AoD),
for the year `2011` and the data set `DoubleElectron`.

The workflow consists of the steps need for the samples reconstruction, as taken from the
[CMS legacy validation repo](https://github.com/cms-legacydata-validation/RAWToAODValidation/tree/master).

## Reconstruction procedure

### 1 & 2. Input data and Analysis code

Any raw input data from the
[CERN open data platform](http://opendata.cern.ch/search?page=1&size=20&experiment=CMS&type=Dataset&subtype=Collision&subtype=Derived&subtype=Simulated&file_type=raw)
should be valid for reconstruction. In this example, the input is taken from:
`root://eospublic.cern.ch//eos/opendata/cms/Run2011A/DoubleElectron/RAW/v1/000/160/433/C046161E-0D4E-E011-BCBA-0030487CD906.root`

The reconstruction step can be repeated with a configuration file that depends on the
analyzed data, e.g. [this example](http://opendata.cern.ch/record/43), or by creating our
own configuration file (created in a CMS VM) and then changing the script accordingly:

```console
cmsDriver.py reco -s RAW2DIGI,L1Reco,RECO,USER:EventFilter/HcalRawToDigi/hcallaserhbhehffilter2012_cff.hcallLaser2012Filter --data --conditions FT_R_53_LV5::All --eventcontent AOD --customise Configuration/DataProcessing/RecoTLR.customisePrompt --no_exec --python reco_cmsdriver2011.py
```

### 3. Compute environment

In order to be able to rerun the analysis even several years in the future, we need to
"encapsulate the current compute environment", for example to freeze the software package
versions our analysis is using. We shall achieve this by preparing a
[Docker](https://www.docker.com/) container image for our analysis steps.

This analysis example runs within the [CMSSW](http://cms-sw.github.io/) analysis
framework that was packaged for Docker in
[cmsopendata](https://hub.docker.com/u/cmsopendata). The different images corresponds to
data sets taken in different years. Instructions can be found under
[this repo](http://opendata.cern.ch/docs/cms-guide-docker).

Moreover, the re-reconstruction task needs access run-time to the condition database and
inside a
[CMS VM](http://opendata.cern.ch/search?page=1&size=20&q=virtual%20machine&subtype=VM&type=Environment&experiment=CMS),
this is achieved with the commands:

```console
$ ln -sf /cvmfs/cms-opendata-conddb.cern.ch/FT_53_LV5_AN1_RUNA FT_53_LV5_AN1
$ ln -sf /cvmfs/cms-opendata-conddb.cern.ch/FT_53_LV5_AN1_RUNA.db FT_53_LV5_AN1_RUNA.db
```

For REANA, the condition database on CVMFS can be accessed with any container, the only
requirement is that the user should specify the necessary CVMFS volumes to be
live-mounted in the `reana.yaml` resource section, as described
[here](https://reana.readthedocs.io/en/latest/userguide.html#declare-necessary-resources).

### 4. Analysis workflow

First, we have to set up the environment variables accordingly for the
[CMS SW](http://cms-sw.github.io/). Although this is done in the docker image, REANA
overrides them and they need to be reset. This is done by copying the
[cms entrypoint.sh script](https://github.com/clelange/cmssw-docker/blob/master/standalone/entrypoint.sh):

```console
$ source /opt/cms/cmsset_default.sh
$ scramv1 project CMSSW CMSSW_5_3_32
$ cd CMSSW_5_3_32/src
$ eval `scramv1 runtime -sh`
```

The actual commands that are needed to carry out the analysis in the CMS specific
environment are then:

```console
$ ln -sf /cvmfs/cms-opendata-conddb.cern.ch/FT_53_LV5_AN1_RUNA FT_53_LV5_AN1
$ ln -sf /cvmfs/cms-opendata-conddb.cern.ch/FT_53_LV5_AN1_RUNA.db FT_53_LV5_AN1_RUNA.db
$ ls -l
$ ls -l /cvmfs/
$ cmsRun reco_cmsdriver2011.py
```

This demo represents a "workflow factory" script that will produce REANA workflows for
given parameters for the CMS RAW to AOD reconstruction procedure.

Following successful tests (see other branches), we know that REANA is able to run CMS
reconstruction for a variety of RAW samples (e.g. dataset SingleMu) and data-taking years
(e.g. 2011).

### Example

Before running example, you might want to install necessary packages:

```console
$ # create new virtual environment
$ virtualenv ~/.virtualenvs/myreana
$ source ~/.virtualenvs/myreana/bin/activate
$ # install reana-commons and reana-client
$ pip install git+git://github.com/reanahub/reana-demo-cms-reco.git@master#egg=cms-reco
```

After, the following will generate the workflow to run the example for a given record id,
with its metadata retrieved using the
[COD Client](https://github.com/cernopendata/cernopendata-client). This generates a
workflow in a given output directory, where the `reana.yaml` file lives with all
necessary inputs:

```console
$ cernopendata-client get-record --recid 39 | tee cms-reco-config.json
# # use the values from the 'cms-reco-config.json' file
$ cms-reco --create-workflow
Created `cms-reco-SingleElectron-2011` directory.
$ cd cms-reco-SingleElectron-2011
$ reana-client run
```
132 changes: 0 additions & 132 deletions README.rst

This file was deleted.

Loading

0 comments on commit 1d8bbb3

Please sign in to comment.