-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathREADME.qmd
232 lines (155 loc) · 11.2 KB
/
README.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
---
title: 'ABC random forests for model choice and parameters estimation'
link-citations: true
bibliography: [ref.bib]
toc: true
toc-depth: 1
format:
gfm:
wrap: none
---
<!-- pandoc -f markdown README-ORIG.md -t gfm -o README.md --citeproc -s --toc --toc-depth=1 --webtex -->
[](https://pypi.python.org/pypi/pyabcranger)
[](https://github.com/diyabc/abcranger/actions?query=workflow%3Aabcranger-build+branch%3Amaster)
Random forests methodologies for :
- ABC model choice [@pudlo2015reliable]
- ABC Bayesian parameter inference [@raynal2016abc]
Libraries we use :
- [Ranger](https://github.com/imbs-hl/ranger) [@wright2015ranger] : we use our own fork and have tuned forests to do "online"^[The term "online" there and in the code has not the usual meaning it has, as coined in "online machine learning". We still need the entire training data set at once. Our implementation is an "online" one not by the sequential order of the input data, but by the sequential order of computation of the trees in random forests, sequentially computed and then discarded.] computations (Growing trees AND making predictions in the same pass, which removes the need of in-memory storage of the whole forest)^[We only use the C++ Core of ranger, which is under [MIT License](https://raw.githubusercontent.com/imbs-hl/ranger/master/cpp_version/COPYING), same as ours.].
- [Eigen3](http://eigen.tuxfamily.org) [@eigenweb]
As a mention, we use our own implementation of LDA and PLS from [@friedman2001elements{81, 114}], PLS is optimized for univariate, see [-@sec-plsalgo]. For linear algebra optimization purposes on large reftables, the Linux version of binaries (standalone and python wheel) are statically linked with [Intel’s Math Kernel Library](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/api-based-programming/intel-oneapi-math-kernel-library-onemkl.html), in order to leverage multicore and SIMD extensions on modern cpus.
There is one set of binaries, which contains a Macos/Linux/Windows (x64 only) binary for each platform.
There are available within the "[Releases](https://github.com/fradav/abcranger/releases)" tab, under "Assets" section (unfold it to see the list).
This is pure command line binary, and they are no prerequisites or library dependencies in order to run it. Just download them and launch them from your terminal software of choice. The usual caveats with command line executable apply there : if you're not proficient with the command line interface of your platform, please learn some basics or ask someone who might help you in those matters.
The standalone is part of a specialized Population Genetics graphical interface [DIYABC-RF](https://diyabc.github.io/), presented in MER (Molecular Ecology Resources, Special Issue), [@Collin_2021].
# Python
## Installation
```bash
pip install pyabcranger
```
## Notebooks examples
- On a [toy example with $MA(q)$](https://github.com/diyabc/abcranger/blob/master/notebooks/Toy%20example%20MA(q).ipynb), using [@JMLR:v19:17-374] as graph-powered engine.
- [Population genetics demo](https://github.com/diyabc/abcranger/blob/master/notebooks/Population%20genetics%20Demo.ipynb), data from [@Collin_2021], available [there](https://github.com/diyabc/diyabc/tree/master/diyabc-tests/MER/modelchoice/IndSeq)
# Usage
```text
- ABC Random Forest - Model choice or parameter estimation command line options
Usage:
../build/abcranger [OPTION...]
-h, --header arg Header file (default: headerRF.txt)
-r, --reftable arg Reftable file (default: reftableRF.bin)
-b, --statobs arg Statobs file (default: statobsRF.txt)
-o, --output arg Prefix output (modelchoice_out or estimparam_out by
default)
-n, --nref arg Number of samples, 0 means all (default: 0)
-m, --minnodesize arg Minimal node size. 0 means 1 for classification or
5 for regression (default: 0)
-t, --ntree arg Number of trees (default: 500)
-j, --threads arg Number of threads, 0 means all (default: 0)
-s, --seed arg Seed, generated by default (default: 0)
-c, --noisecolumns arg Number of noise columns (default: 5)
--nolinear Disable LDA for model choice or PLS for parameter
estimation
--plsmaxvar arg Percentage of maximum explained Y-variance for
retaining pls axis (default: 0.9)
--chosenscen arg Chosen scenario (mandatory for parameter
estimation)
--noob arg number of oob testing samples (mandatory for
parameter estimation)
--parameter arg name of the parameter of interest (mandatory for
parameter estimation)
-g, --groups arg Groups of models
--help Print help
```
- If you provide `--chosenscen`, `--parameter` and `--noob`, parameter estimation mode is selected.
- Otherwise by default it's model choice mode.
- Linear additions are LDA for model choice and PLS for parameter estimation, "--nolinear" options disables them in both case.
# Model Choice
## Example
Example :
`abcranger -t 10000 -j 8`
Header, reftable and statobs files should be in the current directory.
## Groups
With the option `-g` (or `--groups`), you may "group" your models in several groups splitted . For example if you have six models, labeled from 1 to 6 `-g "1,2,3;4,5,6"
## Generated files
Four files are created :
- `modelchoice_out.ooberror` : OOB Error rate vs number of trees (line number is the number of trees)
- `modelchoice_out.importance` : variables importance (sorted)
- `modelchoice_out.predictions` : votes, prediction and posterior error rate
- `modelchoice_out.confusion` : OOB Confusion matrix of the classifier
# Parameter Estimation
## Composite parameters
When specifying the parameter (option `--parameter`), one may specify simple composite parameters as division, addition or multiplication of two existing parameters. like `t/N` or `T1+T2`.
## A note about PLS heuristic
The `--plsmaxvar` option (defaulting at 0.90) fixes the number of selected pls axes so that we get at least the specified percentage of maximum explained variance of the output. The explained variance of the output of the $m$ first axes is defined by the R-squared of the output:
$$Yvar^m = \frac{\sum_{i=1}^{N}{(\hat{y}^{m}_{i}-\bar{y})^2}}{\sum_{i=1}^{N}{(y_{i}-\hat{y})^2}}$$
where $\hat{y}^{m}$ is the output $Y$ scored by the pls for the $m$th component.
So, only the $n_{comp}$ first axis are kept, and :
$$n_{comp} = \underset{Yvar^m \leq{} 0.90*Yvar^M, }{\mathop{\text{argmax}}}$$
Note that if you specify 0 as `--plsmaxvar`, an "elbow" heuristic is activiated where the following condition is tested for every computed axis :
$$\frac{Yvar^{k+1}+Yvar^{k}}{2} \geq 0.99(N-k)\left(Yvar^{k+1}-Yvar^ {k}\right)$$
If this condition is true for a windows of previous axes, sized to 10% of the total possible axis, then we stop the PLS axis computation.
In practice, we find this $n_{heur}$ close enough to the previous $n_{comp}$ for 99%, but it isn't guaranteed.
## The signification of the `noob` parameter
The median global/local statistics and confidence intervals (global) measures for parameter estimation need a number of OOB samples (`--noob`) to be reliable (typlially 30% of the size of the dataset is sufficient). Be aware than computing the whole set (i.e. assigning `--noob` the same than for `--nref`) for weights predictions [@raynal2016abc] could be very costly, memory and cpu-wise, if your dataset is large in number of samples, so it could be adviseable to compute them for only choose a subset of size `noob`.
## Example (parameter estimation)
Example (working with the dataset in `test/data`) :
`abcranger -t 1000 -j 8 --parameter ra --chosenscen 1 --noob 50`
Header, reftable and statobs files should be in the current directory.
## Generated files (parameter estimation)
Five files (or seven if pls activated) are created :
- `estimparam_out.ooberror` : OOB MSE rate vs number of trees (line number is the number of trees)
- `estimparam_out.importance` : variables importance (sorted)
- `estimparam_out.predictions` : expectation, variance and 0.05, 0.5, 0.95 quantile for prediction
- `estimparam_out.predweights` : csv of the value/weights pairs of the prediction (for density plot)
- `estimparam_out.oobstats` : various statistics on oob (MSE, NMSE, NMAE etc.)
if pls enabled :
- `estimparam_out.plsvar` : variance explained by number of components
- `estimparam_out.plsweights` : variable weight in the first component (sorted by absolute value)
# Various
## Partial Least Squares algorithm (univariate) {#sec-plsalgo}
#. $X_{0}=X ; y_{0}=y$
#. For $k=1,2,...,s$ :
#. $w_{k}=\frac{X_{k-1}^{T} y_{k-1}}{y_{k-1}^{T} y_{k-1}}$
#. Normalize $w_k$ to $1$
#. $t_{k}=\frac{X_{k-1} w_{k}}{w_{k}^{T} w_{k}}$
#. $p_{k}=\frac{X_{k-1}^{T} t_{k}}{t_{k}^{T} t_{k}}$
#. $X_{k}=X_{k-1}-t_{k} p_{k}^{T}$
#. $q_{k}=\frac{y_{k-1}^{T} t_{k}}{t_{k}^{T} t_{k}}$
#. $u_{k}=\frac{y_{k-1}}{q_{k}}$
#. $y_{k}=y_{k-1}-q_{k} t_{k}$
**Comment** When there isn’t any missing data, stages $2.1$ and $2.2$ could be replaced by $w_{k}=\frac{X_{k-1}^{T} y_{k-1}}{\left\|X_{k-1}^{T} y_{k-1}\right\|}$ and $2.3$ by $t_{k}=X_{k-1}w_{k}$
To get $W$ so that $T=XW$ we compute :
$$\mathbf{W}=\mathbf{W}^{*}\left(\widetilde{\mathbf{P}} \mathbf{W}^{\*}\right)^{-1}$$
where $\widetilde{\mathbf{P}}_{K \times p}=\mathbf{t}\left[p_{1}, \ldots, p_{K}\right]$ and $\mathbf{W}^{*}_{p \times K} = [w_1, \ldots, w_K]$
# TODO
## Input/Output
- [X] Integrate hdf5 (or exdir? msgpack?) routines to save/load reftables/observed stats with associated metadata
- [ ] Provide R code to save/load the data
- [X] Provide Python code to save/load the data
## C++ standalone
- [X] Merge the two methodologies in a single executable with the (almost) the same options
- [ ] \(Optional) Possibly move to another options parser (CLI?)
## External interfaces
- [ ] R package
- [X] Python package
## Documentation
- [ ] Code documentation
- [ ] Document the build
## Continuous integration
- [X] Linux CI build with intel/MKL optimizations
- [X] osX CI build
- [X] Windows CI build
## Long/Mid term TODO
- methodologies parameters auto-tuning
- auto-discovering the optimal number of trees by monitoring OOB error
- auto-limiting number of threads by available memory
- Streamline the two methodologies (model choice and then parameters estimation)
- Write our own tree/rf implementation with better storage efficiency than ranger
- Make functional tests for the two methodologies
- Possible to use mondrian forests for online batches ? See [@lakshminarayanan2014mondrian]
# References
This have been the subject of a proceedings in [JOBIM 2020](https://jobim2020.sciencesconf.org/), [PDF](https://hal.archives-ouvertes.fr/hal-02910067v2) and [video](https://relaiswebcasting.mediasite.com/mediasite/Play/8ddb4e40fc88422481f1494cf6af2bb71d?catalog=e534823f0c954836bf85bfa80af2290921) (in french), [@collin:hal-02910067].
::: {.references}
:::