Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pull Master branch #1528

Draft
wants to merge 92 commits into
base: MS-corr-refactoring
Choose a base branch
from
Draft

Pull Master branch #1528

wants to merge 92 commits into from

Conversation

DonNabla
Copy link
Contributor

Before you submit this PR: make sure to put all operations-related information in a wiki-note, a PR should be about code and is publicly accessible

What does the code in this PR do / what does it improve?

Can you briefly describe how it works?

Can you give a minimal working example (or illustrate with a figure)?

Please include the following if applicable:

  • Update the docstring(s)
  • Update the documentation
  • Tests to check the (new) code is working as desired.
  • Does it solve one of the open issues on github?

Notes on testing

  • Until the automated tests pass, please mark the PR as a draft.
  • On the XENONnT fork we test with database access, on private forks there is no database access for security considerations.

All italic comments can be removed from this template.

dachengx and others added 30 commits July 10, 2024 09:04
@dachengx
Generate only one instance for MongoDownloader (#1398)
58a78b7
@dachengx
Load whole run for VetoIntervals regardless the run length (#1399)
4d6c2b7
@dachengx
Bump to v2.2.5 (#1407)
5622027 
* Update HISTORY.md

* Bump version: 2.2.4 → 2.2.5

* Bug fix to be compatible with latest strax

6a17e3785e942dca703f038c29f6b7544999a2ee
@dachengx
Inherit DEFAULT_CHUNK_SPLIT_NS from strax (#1405)
419f06c 
* Inherit `DEFAULT_CHUNK_SPLIT_NS` from `strax`

* Use latest strax for testing until new strax version release

* Debug
@dachengx
Use pyproject.toml to install straxen (#1408)
85ae52f 
* Use `pyproject.toml` to install straxen

* Try to debug

* Debug again

* Debug again

* Debug again

* Debug

* Move docformatter to pyproject.toml

* debug

* Move docformatter configuration to pre-commit

* Debug

* Debug

* Debug

* Debug

* Install packages test needed

* Debug

* Debug

* Delete requirement.txt

* Try debug

* Debug

* Debug

* Debug

* Debug

* Debug

* Debug

* Add scripts for installation
@pre-commit-ci @dachengx
[pre-commit.ci] pre-commit autoupdate (#1396)
dd9f866 
updates:
- [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0)
- [github.com/pre-commit/mirrors-mypy: v1.10.0 → v1.11.1](pre-commit/mirrors-mypy@v1.10.0...v1.11.1)
- [github.com/pycqa/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@dachengx
Be compatible with new Plugin.run_id (#1410)
3e48a58 
* Be compatible with new `Plugin.run_id`

* Bug fix

* debug for notebook
@cfuselli @pre-commit-ci @dachengx
Small restrax fix - DAQ (#1402)
731d598 
* small restrax fix

restrax did not fail a run in case of problems with the data docs themselves, getting into an infinite loop of try - fail.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* forgot delete line..

* increment n_tries when failing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* initialise only once

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* better db increase by one

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix set restrax failed inc

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* bug

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@dachengx
Make targeted raw_records chunk 500MB (#1412)
70a7d6e
@dependabot @dachengx
Bump actions/setup-python from 5.1.0 to 5.1.1 (#1403)
189c09e 
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.1.0 to 5.1.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v5.1.0...v5.1.1)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@RoiFrankel @pre-commit-ci
@yuema137 @dachengx
fix peak per event plugin (#1400)
1dd8e3e 
* first tutorial

* first tutorial

* Implement new version of peak_per_event plugin

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* peak_per_event.py

Comments were moved from code to description of PR

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove redundant #

* Minor change

---------

Co-authored-by: Roi Frankel <roif@midway3-login3.rcc.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Yue Ma <3124558229@qq.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@tflehmke @pre-commit-ci
@GiovanniVolta @yuema137 @dachengx
Dynamic led window (#1401)
ced9f51 
* Added dynamic LED window

* Changed default values for LED window

* Changed AP plugin version number

* Added dynamic window to LEDCalibration plugin

* Added comments, changed version number

* Added numba import

* Changed right window edges to be exclusive.

* Added hit finding parameters as URL configs

* Added baseline and baseline_rms fields to internal records datatype.

* Added docstrings and improved formatting

* Changed led window width

* Fixed docstrings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* formatting

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* break long line

* Removed magic numbers from window calculation

* Added check if records are sorted

* Changed threshold to min_height_over_noise

* Renamed and clarified variable names

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix get_records argument

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* handle edge case for empty data

* fixed record_length argument in get_records

* moved check for empty "records" input to "get

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Unify naming of `LEDCalibration` and `LEDAfterpulseProcessing`

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Giovanni Volta <38431109+GiovanniVolta@users.noreply.github.com>
Co-authored-by: yuema137 <3124558229@qq.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@juehang @pre-commit-ci @dachengx
Plugins for position reconstruction with conditional normalizing flow (…
b3e775a 
…#1404)

* Add normalizing flow position reconstruction plugin, and modify existing peak position plugin to include the flow plugin.
Committer: Juehang Qin <qinjuehang@rice.edu>
On branch flow-posrec
Changes to be committed:
	modified:   straxen/plugins/peaks/peak_positions.py
	new file:   straxen/plugins/peaks/peak_positions_flow.py

* Added URLConfig protocol for serialized jax functions

* Add flow plugin to peaks __init__.py

* Add flow Event posrec plugins

* Add flow Event posrec plugins to __init__.py

* Restore default_reconstruction_algorithm in peak_positions

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove default_reconstruction_algorithm from EventPosEventPositionUncertainty where it is not used

* fix merge typo

* Add numpy import to peak_positions_flow.py

* Adds check to make sure model file exists in tar.gz resource

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor fix to make sure mypy correctly infers type in jax URLConfig

* another mypy fix

* flake8 style fixes

* fix flow plugin default model URLConfig

* Update documentation for flow plugins

* Added note to docstrings, changed flow to cnf, removed extra var

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change flow to cnf in peak_positions plugin

* Add no-qa for flake8 line length check in docstring URL.

* Add default_reconstruction_algorithm

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* `PeakPositionsCNF` should be inherited from `PeakPositionsBaseNT`

---------

Co-authored-by: Juehang Qin <juehang@midway3-login4.rcc.local>
Co-authored-by: Juehang Qin <qinjuehang@rice.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@dachengx
Remove redundant spaces (#1411)
5362dfa
@dachengx
Stop using self.config because we do not use strax.Option (#1413)
007ede1 
* Stop using `self.config` because we do not use `strax.Option`

* Unify configs of `tpc_internal_delay`
@dachengx
Remove DetectorSynchronization (#1414)
c618480
@dachengx
Remove configuration sum_waveform_top_array from MergedS2s (#1415)
cb34a66 
Close: #1305
@dachengx
Debug for EventwBayesClass because peaks overlapping (#1417)
c1eda3e
@pre-commit-ci @dachengx
[pre-commit.ci] pre-commit autoupdate (#1416)
00ee694 
updates:
- [github.com/pre-commit/mirrors-mypy: v1.11.1 → v1.11.2](pre-commit/mirrors-mypy@v1.11.1...v1.11.2)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@WenzDaniel @pre-commit-ci @dachengx
Refactor nv plugins (#1228)
6ad09a1 
* Remove hitlet entropy, make hitlets more light

* Add summed waveform to NV events

* Small fix, area needs to be defined before compute widths

* Add min/max gap to event

* Move waveform calculation into an extra plugin

* Make extra plugin for waveform, some small chnages

* Bump version

* Change back compressor

* Register plugin by default

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove imports and options

* Small fixes

* Remove not needed files

* Make NV events again time interval

* Added dt field and length to buffer

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@WenzDaniel
Changed NV software trigger (#1388)
ab7f6dd
@dachengx
Assign __version__ of RecordsFromPax (#1418)
61d55f2
@dachengx
Minor debug for the pyproject.toml (#1420)
6d90384 
* Minor debug for the `pyproject.toml`

* Force install
@dachengx
Fix the usage of scripts (#1423)
f436f7f
@dachengx
Deprecate selection_str (#1424)
7f7e0c1 
* Deprecate selection_str

* Ignore untested scripts
@dachengx
Fix singleton pattern for MongoDownloader (#1426)
b12383c
@dachengx
Following AxFoundation/strax#893 (#1432)
6b26525
@dachengx
Be compatible with utilix>0.9 (#1433)
a412b43
@dachengx
Specify available RSE in RucioRemoteBackend (#1435)
35917e7
@yuema137 @dachengx @pre-commit-ci
Add docstring rucio (#1436)
cb27dd4 
* Specify available RSE in `RucioRemoteBackend`

* add docstring for the _get_rse function

* reformat docstring

* add docstring for the frontend class for rses_only

* also add docstring to the backend class

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: dachengx <dx2227@columbia.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
dachengx and others added 28 commits November 16, 2024 21:15
@dachengx
Fix small bug in CNF (#1479)
e222b25
@dachengx
Remove GCN & CNN S2 pos-rec (#1484)
b18d972 
* Remove GCN pos-rec

* Remove CNN

* CNF as default

* Recover MLP as default
@dachengx
Set CNf as the default S2 (x, y) position-reconstruction (#1486)
081af5d
@dachengx @pre-commit-ci
Prototype of peaklets-level (x, y) S2 position reconstruction (#1482)
42908b0 
* Prototype of peaklets-level pos-rec

* Simplify dependency tree

* Add TODO

* Pos-rec for merged_s2s

* Recover all possible data_types

* No 'NT'

* Debug

* Restructure the dependency tree with straxen style

* Still keep CNN and GCN

* Debug

* Debug

* Debug

* Debug

* Renaming

* Debug

* Remove CNN and GCN

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* A minor change

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@dachengx
Rename old PeakletClassification as PeakletClassificationVanilla (#…
1c3bd28 
…1487)
@dachengx
Remove Bayes models (#1488)
0a31fa1
@dachengx
Rename defualt_run_comments -> default_run_comments (#1489)
9813b30
@dachengx
Accelerate Euclidean distance by numba (#1493)
f765c10 
* Accelerate Euclidean distance by numba

* Do not assert in numba
@dachengx
Move set_nan_defaults to be a stand-alone function (#1497)
5704361
@dachengx
Set CNF as the default S2 (x, y) position-reconstruction (#1494)
7138149 
* Set CNF as the default S2 (x, y) position-reconstruction

* Hack into CMT and assign MLP maps when asked for CNF maps

* Hack more

* Fix bug
@GiovanniVolta @dachengx
did pushed (#1499)
951856b 
Co-authored-by: Giovanni Volta <gvolta@midway3-login4.rcc.local>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@dachengx
Move compute_center_times from straxen to strax (#1501)
7fc3dbd 
* Move `compute_center_times` from straxen to strax

* Update version

* Debug
@dachengx
Use numpy and strax native dtypes, not "<i8" or "<f4" (#1502)
4d2c89e 
* Use numpy native dtypes, not `"<i8"` or `"<f4"`

* Use `np.bool_` inside numba decorated function

* Debug
@dachengx
Inherit area_fraction_top, center_time and median_time from pea…
7c8a01c 
…klets (#1503)

* Inherit `area_fraction_top`, `center_time` and `median_time` from peaklets

* Debug

* Minor change

* Add comment

* Debug

* Reform

* Debug

* Debug
@dachengx
Bump version of changed plugins in #1503 (#1504)
3a310e4
@dachengx
Clean unnecessary codes (#1507)
57e0f33
@dachengx
Clean chunk after computing records (#1508)
5bb4e04 
* Clean chunk after computing `records`

* Fix bug

* Minor change
@dachengx
Add a line of comment about memory optimization (#1509)
dfcf36b 
* Add a line of comment about memory optimization

* Minor change
@dachengx
Bump to v3.0.1 (#1510)
9815352 
* Update HISTORY.md

* Bump version: 3.0.0 → 3.0.1

* Update dependency
@dachengx
Collect SOM dtype at one place (#1511)
69265ff
@dachengx
Stop support for list of "take" protocol (#1517)
f993f0b 
* Stop support for list of "take" protocol

* Try to bypass the check of URL

* Debug

* Debug
@dachengx
Add stage flag for RucioRemoteBackend (#1520)
a61fc62
@dachengx
Bump to v3.0.2 (#1522)
dd63518 
* Bump version: 3.0.1 → 3.0.2

* Update HISTORY.md
@dachengx
Rename "algo" to "alg" because they were mixed (#1523)
51a5b52
@WenzDaniel @dachengx
Bump version (#1500)
db52edc 
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@cfuselli @JoranAngevaare
@pre-commit-ci @yuema137 @dachengx
[DAQ] Bootstrax dynamic adapt targets for AmBe high rate (#1492)
0909068 
* fix_resources

* up to peak monitor

* only raw records

* only peaklets

* ambe do all

* new era

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* long line

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: cfuselli <cfusellu@nikhef.nl>
Co-authored-by: JORAN <jorana@nikhef.nl>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Yue Ma <3124558229@qq.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
@dachengx
Track position reconstruction algorithm in EventPatternFit plugin (#…
21b5e34 
…1524)
@dachengx
Still use np.bool_ in numba decorated function (#1527)
4c215d3
@DonNabla DonNabla marked this pull request as draft January 17, 2025 15:14
@dachengx
First and last channel inside peak(let)s (#1525)
6e0a79d 
* First and last channel inside peak(let)s

* Debug

* Update Peaklets version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.