Troubleshooting FAQ

[autoSteve]

Q: I have a Solcast API limit of 50 calls. Why is the integration now limiting me to 10?

Can't happen now, given Solcast removed an API call. To answer a question with a question, is the API quota set correctly in the integration configuration? If not, then set it to 50.

[autoSteve]

Q: You asked for DEBUG logs to be provided when raising issues. How do I get these?

We can't usually do anything without them, and it's almost always the first question that will be asked: "Could you provide debug logs, please?"

When set for debug you see all kinds of fascinating stuff about what's going on under the covers. In configuration.yaml:

 logger:
   default: warn
   logs:
     custom_components.solcast_solar: debug

(Make the default info/warn/whatever, as we don't care. We just want the debug goodness.)

Reviewing logs is quite simple, but debug logs cannot be reviewed from the UI. The file /homeassistant/home-assistant.log must be viewed. I find the handiest way is to use the less command from the HAOS command line to do this.

less /homeassistant/home-assistant.log
then...
/solcast_solar to search for and highlight lines relating to the integration
&/solcast to only display lines containing 'solcast'
PgUp/PgDn/Up/Down to hunt around.
F key to follow live (capital F)
Ctrl-C to exit follow mode

If you are running HA under Docker, then the command to get the logs might be similar to

docker compose logs -n 500 -f homeassistant

This will display and follow the last 500 lines of logs from the hommassistant service in your docker compose stack.

[autoSteve]

Q: Is submitting debug logging going to expose my API key or location to the world?

Yeah, nah. Definitely not now. All sensitive information, including API keys, and your home address via latitude/longitude are now redacted. Make sure you're on the latest version.

[autoSteve]

Q: What polls to Solcast happen, when do they happen, and are they important?

When the integration starts for the very first time, several important things happen, and these involve Solcast polls.

1. The rooftop sites information is gathered for each of the solcast API keys specified.

This is super important information, and the integration cannot function with out this. The return includes the rooftop ID(s), which are used in subsequent calls, plus other data that isn't used, like location, azimuth and panel tilt.

If the API call does not work for some reason, then this integration utilises a cache that will recall the data from the last successful call. If the cache doesn't exist yet, then the integration won't work, and it will continuously restart until this call succeeds.

This call will not use up any precious API call quota.

2. When "estimated actuals" is not yet available.

This call happens theoretically once, and only for a new install or if the solcast.json file is deleted (which is an action that can be requested via Developer tools, or by directly deleting the file).

This occurs for each rooftop ID, so if you have two Solcast sites defined, then two calls are made.

This will use up double the API quota than a regular forecast update.

3. When a forecast update is requested.

Auto-update is enabled, or an automation is created by you in Home Assistant to trigger how often solar forecasts are gathered, and when this triggers the service solcast_solar.update_forecasts it will update all of the rooftop IDs for all of the accounts.

This will use up API quota, and if you have two sites configured then it'll use up two calls.

Should this call not be successful, then it will be re-tried ten times. (A failure almost always does not use quota. It may if the failure happens due to a bug.) The retry mechanism is designed using a back-off mechanism that will retry at delays of 15, 30, 45, 60 and so on seconds, plus a random number of seconds between zero and fifteen for each retry. You'll see this activity in the log as warnings if it happens.

If all retries are exhausted, then a 429/Solcast too busy error will be logged.

But don't panic and raise an issue. It's almost certainly them, not you or us, and the next forecast acquisition will likely be successful.

That's all the API calls there are!

Sometimes the API gets so swamped with requests that it asks users to retry. This is the well seen HTTP 429 response where quota has not yet been reached. It's not an error as such, but more a "we heard you, but go away we're busy, try later" notification.

This integration does its level best to cooperate with Solcast, and retry in a sensible manner.

Q: A follow-up question: If I restart the integration will it use API quota?

The integration has a cache of the last successful forecast call response data and the sites data. The sites are loaded on startup if requesting it from Solcast fails. Then the forecast history loads. This does not use API quota. Forecast is only requested, and API usage incremented when auto-update fetches, or you ask the integration to do so, normally with an automation.

So the answer is definitely no. But this becomes a definite maybe as of v4.2.5.

If you have auto-update enabled in v4.2.5+ then some strange things can seem to happen. If you restored Home Assistant from a backup that pre-dated the last auto fetch then the integration will initiate a fetch because stale data. API call(s) used.

And if you re-started HA immediately before an auto-update was scheduled to fire then that update will fire on integration start. API call(s) will be used, but they would have been used anyway, just weren't previously.

So depending on circumstance, the v4.2.5+ answer is possibly, but probably not.

[autoSteve]

Q: Why does the Solcast integration sometimes take a long time to start?

The only thing that could cause this to happen when using the latest (possibly beta) release is if the forecast cache file solcast.json failed to load, as well as the Solcast API failed to return all site details.

This would be super rare.

The Solcast integration should load super quick every time.

If it doesn't, and you cant work out why, then please raise an issue including much log.

If this is the very first time that the integration is being loaded for a new installation, then it is also possible that initial startup occurs at precisely the same moment that the Solcast API happens to be really busy. If this is the case then the integration will fail to load, but will then repeatedly attempt a start until things work out. You can see this happening either in the logs, or by opening the integration at Settings | Devices & Services | Integrations.

[autoSteve]

Q: Why have my historical forecasts disappeared from the energy dashboard? I now only see 10/14 days!

At some point, your /config/solcast.json file has gone missing, and was recreated. This contains the history.

First ask yourself, what use are historic forecasts to me anyway? A dashed line that extends back as far as since this integration was first installed is really only visually pleasing, and not really of value.

What temperature was forecast on the 3rd of December 2021, and was it right? Who cares? We know the actual answer now. Solcast would care about improving forecast accuracy, but I'm pretty sure they would not use history to do so. They would compare the predictions of present and proposed forecast models against actuals over time.

The "good" news is that these forecasts will be retained for a couple of years from here on, so your dashed line will get longer, even if it is of almost zero value.

Or do you really want to fix it?

I hope you have a backup from the day when the history vanished. You do back up, right?

The fix involves "merging" the contents of two solcast.json files, which is not as simple as just concatenating the files.

Inside the json structure there is a forecasts key for each rooftop site, which holds an array. What you need to do is get the older values from backup for this array, and carefully (making sure there is the required comma between array elements) insert these forecast elements into the current file. Do this for each rooftop ID, then restart the Solcast integration. (Having duplicates of the timestamped entries won't hurt anything, and they will be cleaned up.) Please, take a backup of solcast.json before attempting this...

(A footnote. I believe I know why this happened, so v4.1.4 includes a change that should prevent it in future. It hasn't just been BJReplay versions that have done this, as folks have encountered, but we're trying to stop it from ever happening again!)

[autoSteve]

Q: Why are certain sensors Watt, while others are Watt-hour or kilo-Watt-hour? Shouldn't these be the same? Why?

The power sensors, with a unit of measurement of Watt represent an instantaneous forecast power at a point in time. Given Solcast forecasts in half-hourly increments these can be thought of as an average power that is expected to be generated for a period (or the value expected half way through each interval).

All values received from Solcast are instantaneous power, or Watts.

The values for Watt-hour/kWh are calculated by the integration from the power numbers, and are power over time, or energy. An example of this is expected solar production for the remainder of the day. For these, the period averages are summed, and then divided by two because the unit is for a whole hour, yet intervals are half-hourly. For some of these, like remaining for the day, a portion of the calculated first period is used because some sensors are updated every five minutes.

@ProphetOfDoom drew up a great annotated representation of an actual forecast chart overlaid with the underlying values that had been received from Solcast.

[autoSteve]

Q: I get a timeout connecting to api.solcast.com.au!!! What the heck is happening? (...raises issue...)

Honestly, it's not us, but you.

This kind of thing can cause you to tear your hair out, but we might be able to put it back. A timeout is a timeout. We got nothing, so we timeout. Period. We got nothing, so we can't go on. But...

You have to check your networking. It may seem right, but it's not. It may seem solid but...

Try a curl from your Home Assistant server first. (This will not use up API call quota.)

curl -X GET -H 'content-type: application/json' https://api.solcast.com.au/rooftop_sites?api_key=YOURAPIKEYHERE

Do you get an instant, and pleasing response of your sites data? (Or even a 429/Solcast too busy) Then, great! Move on. Because more networking bear traps can be laid... It is possible for the command line curl to receive an IPv4 address, and not the address that Home Assistant actually gets from making the same DNS query.

WT?

Check for IPv6 weirdness. Is IPv6 enabled in Home Assistant? If yes, then triple check that HA can actually talk IPv6 to the Internet... If in doubt, then disable IPv6 in the HA network config. Or triple check your router config. This IPv6 stuff is new and scary, but please get your head around it, or disable it if you don't get it.

If that's not the problem, and curl works, then I've got nothing without knowing more. But it's a network issue... Try a more generic networking issue Google search relating to Home Assistant.

[autoSteve]

Q: Does the integration cope with daylight savings time / Summer time transitions?

It does now.

If it is logging odd things for you in debug level logs, and you're getting multiple "Sunday" forecasts (if that's your day of transition) then you need to upgrade to at least v4.2.5.

The transition to daylight time results in Solcast varying the number of half-hourly forecast intervals for the day of transition. When transitioning to daylight time there will be only 46 intervals, and not the usual 48. This is because 2AM will no longer exist for that day. When transitioning from daylight time we get a sleep-in, and there are two 2AMs and a total of 50 intervals.

The integration was messing up the UTC time of period start and end, and using a fixed number of 48 intervals. Now it does not.

[autoSteve]

Q: I just restored Home Assistant from backup, and when it started the Solcast integration updated the forecast! Why?

Auto-update is enabled. The backup that you restored from was prior to the last auto update.

The integration records the date and time of the latest update attempt in solcast.json. On start, it calculates the auto-update intervals, plus the date and time of the most recent auto-update. It then compares the most recent auto-update time with that recorded in solcast.json, and if the forecast cache is out of date, or "stale" then it will do a forecast update.

It has no way of knowing that you have restored from backup.

This is an unusual situation to arise, so we have no plans to alter the integration behaviour. At worst, there will be one instance of API quota exhaustion on update for the day before the next UTC midnight reset of the used count happens.

As an aside, the reason that it does this check is to cover a far more likely scenario. Re-starting Home Assistant.

If HA gets restarted just before a scheduled auto-update is going to happen then that update will be cancelled. If the check for stale forecast data were not done on start then that update would be missed entirely.

So short-term pain on restore from backup. Long-term gain on having auto-update operate reliably for you.

Don't like the behaviour? Send feedback in a discussion, and revert to using an HA automation to update the forecast.

Another aside.

There was an alternate implementation considered, but disregarded by the way. An integration can hold up the restart until it has completed everything it needs to do, so our integration could do that should an update be imminent. This was initially disregarded quickly because it is super-annoying when HA takes a long time to restart for whatever reason. The integration queues pending updates up to five minutes ahead of time, and that would be a super-unreasonable time to wait.

And should a super-annoyingly long restart happen right at the time that auto-update should have done its thing (for no reason to do with the integration), if the update hadn't even been queued yet then the stale forecast check will have that eventuality covered too.

But that alternate idea was also well discarded after futher thought.

A stale check will also work for a VM 'reset' event, or a power failure missing the forecast update instead of a clean shutdown.

Stale check for the win.