Custom forecast confidence

[autoSteve]

I am toying with the idea of creating a custom forecast 'confidence' level in the integration.

This may get legs, or it may take off like a wrought iron hang glider...

Input appreciated.

We receive pv_estimate10, pv_estimate and pv_estimate90 from Solcast, and I've heard it said a few times something like, "I usually get values somewhere between pv_forecast10 and pv_forecast. I'm going to try and make a set dampening automation to adjust values down..."

What if we could calculate our own pv_estimate_custom, or adjust?

Configure this as a variable confidence level in config flow and via an action of between 10 and 90. Create a new detailedForecast key of pv_estimate_custom, and populate it with proportional values in between the Solcast confidence levels. Make it selectable as default by varying the select value. Include it in sensor attribute breakdowns so it could be visualised.

Sounds a bit complicated.

It is.

My preferred alternative would be to retain the three estimates that we have, and add a config flow entry/action to "adjust" pv_50 up or down. This would internally use the proportional values calculated in between the Solcast confidence levels.

Much simpler. And less Urdu translation.

Interest?

[autoSteve]

@ChirpyTurnip and @gcoan, you may both possibly be interested and have a view.

[autoSteve]

Or think "wrought iron ultralight" in your case, @gcoan.

[gcoan]

Definitely not wrought iron, there's a whole load of weight / strength balancing going on.
Aircraft grade (pure) Aluminium then yes please

This sounds very much like what Predbat does for my solar battery forecast.

You choose the base forecast

And then mix in a weighted amount of the pv10 forecast https://springfall2008.github.io/batpred/customisation/#solar-pv-adjustment-options

And optionally can turn on a cloud simulation mode that modulates the individual 5 minute estimates https://springfall2008.github.io/batpred/customisation/#cloud-coverage-and-load-variance

(I have turned this latter bit off as I was getting some weird charge and discharge behaviour).

Your suggestion could be useful for newcomers who don't want to create custom sensors so yes, worth exploring.
An alternative can be achieved with setting the solcast efficiency and integration dampening and may be more guidance on setting up these factors?

For me I don't think I'll use it as predbat is already doing this in the half hour battery plan and I don't really need another different forecast

[autoSteve]

Best of both worlds, but. An Energy dashboard more in line with your reality, and predbat fed with reduced confidence.

[autoSteve]

And by "proportional" I really mean inverse log of the difference. Confidence 40 would be just below confidence 50. Confidence 30 way closer to confidence 50 than confidence 10. Confidence 20 would be well below 50 and approaching 10.

It'd be an approximation of the in-between mean, which to do properly would need the original sample set, which we don't have.

[BJReplay]

I suggested something like that to oziee back in the day, but it was shot down as not something that he wanted to do, but I should feel free to template it if I felt so inclined.

I'd probably move my slider (it's gotta be a slider, and ideally rendered as an old school balance linear potentiometer 🤣) towards 40. The limits are obviously 10, centre detent at 50, and 90.

The problem, as I see it, is the probably inevitable complaints that 40 isn't right - it should be closer to 50 in one person's opinion, but closer to 10 in another.

[BJReplay]

What came to mind was both a bell curve and Gaussian distribution - at a guess that is what the 10% and 90% figures are. If the values were 5% / 95% they'd both be 2sd, and all but useless because so far away from the mean forecast.

In energy forecasting, 10%POE (the Solcast 90 forecast) and 90%POE (the Solcast 10% forecast) are really common outer band forecasts, and still expected to be right most of the time.

In terms of the function to blend the two forecasts, would it be an inverse log, or a RMS? I have forgotten all my stats. Gut says RMS.

I have to find out if I have years of forecasts with the 10 and 90 versions stored, or if I threw them all away (and have to burn some API calls to get some historical forecasts) to run some numbers.

[autoSteve]

I don't know if this would be the best way to go about it, but... Confidence heads towards plummeting as PV10 approacheth (using simple generation values for PV50 and PV10 as the same as those confidence designations.) "Basic" math, and would execute quickly, but is as dumb as a bagful of hammers without more input. =B$3 + (LOG((A5-B$3),((B$2-B$3))) * (B$2-B$3)), which could probs be simplified and tweaked.

[autoSteve]

(Imagine the other side of the 'confidence curve' extending upwards towards PV90...)

[autoSteve]

a bell curve

My thought is that this kind of adjustment should not do that. It should greatly sway at the extremes for a forecast (much down, or up to an extreme), reflecting a massive (edit: sway) in confidence at the outer data points.

[BJReplay]

Replying from my phone, so you will have to make do with imagined curves: a bell curve, or a sigmoid if opened up and flipped to represent cumulative probability (or, in this case, the movement between three known curves) would have the effect of being sticky at the three published curves, but moving rapidly away towards the other sticky nodes as you moved the slider.

Which might be a good desired effect - "light" steering near top dead center, where larger inputs didn't produce larger outputs, but as you move more, the outputs are amplified.

Let me put together both some simple and worked examples.

Finally, I don't think that this matters very much on the 90 side, as they are rarely of much value anyway, other than as surprise good news targets on cloudy days.

[ChirpyTurnip]

So one wonders...dare I say it....if a Bayesian approach work better? You already have a forecast with probability bands, you can also ask the user to supply the HA sensors that register inverter outputs (both power + energy).

It has been a long, long, long, time since I did this all at Uni....so I can't do this modelling anymore as I have forgotten more than I can now remember. I do recall though that of the modelling methods we always seemed to do better with Bayesian statistics as errors/inaccuracy/uncertainty aren't always neatly distributed, especially if they were compounding.

You already have a forecast with probability bands, you can also ask the user to supply the HA sensors that register inverter outputs (both power + energy). A Bayesian method could track the actual generation and adjust the forecast accordingly - a re-calc every 15 minutes could covers the near term until a new Solcast update makes weather adjustments. You would only need to do this for the 'today' forecast as you would only have actual for the present time.

Over the longer term it would potentially be possible to build an adjustment matrix of "forecast output" to "likely output" based on the past fit/adjustment of the forecast. Depending on how accurate you want to be this could be generic, or more accurate by creating multiple matrices for times of day, and for fixed time periods (i.e. per quarter, per month, or per week) so that it is seasonally accurate. There is a limit at which point returns rapidly diminish....working out where is half the trick.

It is a deep rat hole and you can easily get lost in it - but there is a lot to be said to tuning with actuals as that will passively account for shading, bird poop, grid conditions, and so on...

[autoSteve]

I am abandoning thoughts of this unless others come up with a compelling reason.

It is a forecast. It is going to be right, or wrong. 50% confidence is okay for me, with an eye on the 10 percent.

Yes, it could be tweaked, but to what benefit? Right now I can come up with no serious benefit, so am moving on, which saddens me because there is a horse race here that has resulted in a public holiday tomorrow. Instead I will do weeding.

Given your suggestion to Oziee, BJ, I am all ears if you present a model and a compelling...

[BJReplay]

Yeah, fair enough. I've been distracted with something else, and the inability to focus clearly on a screen and type at the same time to do the analysis that I wanted to do, but having had a look at the code base in some detail over the last couple of days...

It's complex enough, already!