Skip to content

Releases: SolarArbiter/solarforecastarbiter-core

1.0.13 Release

11 Feb 21:26
d4d85a3
Compare
Choose a tag to compare
Metrics: make CRPS handle obs outside the fx support (#781)

* tests for CPRS with obs outside forecast support (WIP)

* CRPS: handle obs outside forecast support

Revise the CRPS calculation to handle the case where the observation is
outside the forecast support. For example, if Prob(Power <= 10 MW) = 0%
but then the observation is 9 MW. Or if Prob(Power <= 30 MW) = 100% but
ten the observation is 31 MW. This change required some subtle changes
to how the vectorized calculation is performed. Also, this commit
switches the integration from the rectangular rule to a quadrature rule,
which seems result in more accurate CRPS calculations when the number of
forecast CDF intervals is low (e.g., 10 or less).

This commit also updates the docstring of the CRPS function and the
tests, including comparisons against examples where the forecast is
defined by a continuous parametric distribution that allows calculating
the CRPS analytically.

Note: this branch still needs to validate the CRPS skill score
calculation and related tests. Also, it would be good to include some
"simpler" CRPS calculation examples (e.g., with 3 or 5 CDF intervals,
but that may not be practical.

* CRPS: simplify integration using np.trapz

Simplify the integration using the numpy trapezoidal function. The
result is identical to the prior code (since underneath it's the same
operations), but using np.trapz() makes it clearer what method is being
used and allows users to read up on the numpy documentation regarding
the integration method.

* Add tests for CRPS and CRPSS (skill score)

Add additional tests for the CRPS and CRPSS (CRPS skill score)
functions, including "simple" examples with 3 CDF intervals to help show
the logical of the trapezoidal integration.

* CRPS: allow numpy broadcasting with n >= 2

Modify how the forecast CDF is expanded and the observation indicator
function is calculated to support numpy broadcasting when the number of
samples is greater than or equal to two (i.e., n >= 2).

* tests/probabilistic: use np.array for obs

Revert to assuming observations are provided as numpy arrays, including
in the case where the observation is for a single sample. This matches
the previous logic and helps prevent issues in other parts of the code
base.

* tests/calculator: update CRPS examples

Note that since the CRPS calculation now uses a more generalized
numerical integration setup, some of the example CRPS values had to be
adjusted as the "simple" CRPS examples in the calculator tests are not
very realistic (e.g., only 3 forecast CDF intervals). Rather than
complicate the examples in these tests, I instead corrected the CRPS
values for the given examples.

Also, this commit corrects a mispelling of the Brier score name in a
code comment.

* Add whatsnew/1.0.13.rst with CRPS revision

* whatsnew: correct name

* Address reviewer comments and suggested revisions.

1.0.12 Release

11 Jan 20:21
881fbab
Compare
Choose a tag to compare
add xfails for posting forecasts (#768)

* add xfails for posting forecasts

* add xfails to all realapisession tests that rely on database consistency

* set 1.0.12 release date in whatsnew

Co-authored-by: lboeman <lboeman@email.arizona.edu>

1.0.11 Release

13 Dec 18:35
47b5aef
Compare
Choose a tag to compare
Orca cleanup (#767)

* whatsnew

* remove orca test fixtures, notes, cli command

1.0.10 Release

09 Dec 20:46
dde1c50
Compare
Choose a tag to compare
whatnewdate (#766)

1.0.9 Release

03 Dec 19:29
f5976a4
Compare
Choose a tag to compare
set release date (#761)

* set release date

* Update docs/source/whatsnew/1.0.9.rst

Co-authored-by: Will Holmgren <william.holmgren@gmail.com>

Co-authored-by: Will Holmgren <william.holmgren@gmail.com>

1.0.8 Release

04 Nov 23:28
262d4a2
Compare
Choose a tag to compare
add 108 whatsnew (#749)

1.0.7 Release

28 Oct 16:19
1453093
Compare
Choose a tag to compare
1.0.7 whatsnew (#746)

1.0.6 Release

08 Oct 19:29
24a3aa6
Compare
Choose a tag to compare
finalize 1.0.6 whatsnew (#740)

1.0.5 Release

22 Sep 23:15
f3d3f62
Compare
Choose a tag to compare
Finalize 1.0.5 whats new (#731)

* add date, @williamhobbs

* missing doc requirements

* fix pr number

1.0.4 Release

18 Aug 16:59
e02d458
Compare
Choose a tag to compare
update whatsnew for v1.0.4 (#714)