[Test build] Testers for new upcoming build (X-mas '24)

[TD-er]

I've just uploaded the latest GH Actions build to the web flasher

So I would like to summon the very appreciated testing crew :)
@fly74 @ghtester @chemmex @alex-kiev @iz8mbw @Wookbert @Dickesplumpel @thomastech

I had planned to also have the complete network/WiFi rewrite in this build, but it is taking quite a bit longer than expected.
This is also because of other reasons which prevent me from working on ESPEasy for hours at a time. Since the rewrite takes quite a lot of time to get 'into it' again each time I work on it, it doesn't make sense to do anything on it when I only have an hour to spare.

Anyway there have been quite a lot of fixes and other changes since the last build, so let's prepare for some X-mas build.

To make clear what the differences are between SPIFFS and LittleFS:

• ESP32-xx LittleFS builds are based on ESP-IDF 5.3 code
• ESP32-xx SPIFFS builds are based on ESP-IDF 5.1 code

As can be seen, both do not differ a lot anymore and on the next build this difference might even be completely gone as I can hopefully then use pioarduino with the custom SDK flags made by Jason2866 and Staars, both from Tasmota.

Just as a reminder for those who want to upgrade from an April build:

There are lots and lots of changes, but the main tricky one is that it might be impossible to OTA update larger builds from the > April release as there was a bug in the Arduino code which did not properly use timeouts when handling HTTP requests.

If this is an issue, to update any "max" builds, I will add intermediate builds for those typically 16M and 8M builds which do not contain any plugin or controller to make them as small as possible.
This way it is more likely you can successfully OTA update such units without the need to hook them up via serial.

So please let me know if this is indeed an issue (and maybe also for "4M builds LittleFS").
N.B. this does only apply to "LittleFS" builds when OTA updating from the April release. (or test builds from around that time)

Found issues so far:

• Solo1 builds bootloop
• Fetching HTTP calls in quick succession is often failing.
• Unable to save rules while connected via Ethernet (see forum )

[iz8mbw]

@TD-er instead to use the latest GH Actions build, may I build by myself a Custom Build using the latest commit available now on Github?

[TD-er]

Sure
It is the same code base and you may find other issues which may not show when using the standard builds.

[iz8mbw]

OK, going to build and test!

[fly74]

Test activated for from all.zip:

.
├── ./ESP32-D0WDQ5_neopixel_ESP32_4M316k_ETH
│   └── ./ESP32-D0WDQ5_neopixel_ESP32_4M316k_ETH/ESP_Easy_mega_20241210_neopixel_ESP32_4M316k_ETH.bin
├── ./ESP32-D0WDQ6_neopixel_ESP32_4M316k
│   └── ./ESP32-D0WDQ6_neopixel_ESP32_4M316k/ESP_Easy_mega_20241210_neopixel_ESP32_4M316k.bin
├── ./ESP32-D0WDQ6_normal_ESP32_4M316k
│   └── ./ESP32-D0WDQ6_normal_ESP32_4M316k/ESP_Easy_mega_20241210_normal_ESP32_4M316k.bin
├── ./ESP8266_climate_ESP8266_4M1M
│   └── ./ESP8266_climate_ESP8266_4M1M/ESP_Easy_mega_20241210_climate_ESP8266_4M1M.bin
├── ./ESP8266_minimal_core_312_1M_OTA_Domoticz_MQTT
│   └── ./ESP8266_minimal_core_312_1M_OTA_Domoticz_MQTT/ESP_Easy_mega_20241210_minimal_core_312_ESP8266_1M_OTA_Domoticz_MQTT.bin
├── ./ESP8266_neopixel_ESP8266_4M1M
│   └── ./ESP8266_neopixel_ESP8266_4M1M/ESP_Easy_mega_20241210_neopixel_ESP8266_4M1M.bin
└── ./ESP8266_normal_ESP8266_4M1M
   └── ./ESP8266_normal_ESP8266_4M1M/ESP_Easy_mega_20241210_normal_ESP8266_4M1M.bin

[iz8mbw]

All OK here on ESP32 boards (connected on WiFi) using Custum Builds "ESP_Easy_mega_20241210_custom_ESP32_4M316k_LittleFS_ETH"!!!
Thanks @TD-er

[Dickesplumpel]

OTA update from ESP_Easy_mega_20230623_normal_ESP32s2_4M316k_CDC to ESP_Easy_mega_20241210_normal_ESP32s2_4M316k_CDC
Fresh Webui install with ESP_Easy_mega_20241210_normal_ESP8266_4M1M on 12F and ESP_Easy_mega_20241210_normal_ESP32_4M316k on ESP32-WROOM_32

No issues so far. In the past I had often trouble setting up wifi settings with putty, now it seems working pretty good.

[fly74]

Test activated for from all.zip:

.
├── ./ESP32-D0WDQ5_neopixel_ESP32_4M316k_ETH
│   └── ./ESP32-D0WDQ5_neopixel_ESP32_4M316k_ETH/ESP_Easy_mega_20241210_neopixel_ESP32_4M316k_ETH.bin
├── ./ESP32-D0WDQ6_neopixel_ESP32_4M316k
│   └── ./ESP32-D0WDQ6_neopixel_ESP32_4M316k/ESP_Easy_mega_20241210_neopixel_ESP32_4M316k.bin
├── ./ESP32-D0WDQ6_normal_ESP32_4M316k
│   └── ./ESP32-D0WDQ6_normal_ESP32_4M316k/ESP_Easy_mega_20241210_normal_ESP32_4M316k.bin
├── ./ESP8266_climate_ESP8266_4M1M
│   └── ./ESP8266_climate_ESP8266_4M1M/ESP_Easy_mega_20241210_climate_ESP8266_4M1M.bin
├── ./ESP8266_minimal_core_312_1M_OTA_Domoticz_MQTT
│   └── ./ESP8266_minimal_core_312_1M_OTA_Domoticz_MQTT/ESP_Easy_mega_20241210_minimal_core_312_ESP8266_1M_OTA_Domoticz_MQTT.bin
├── ./ESP8266_neopixel_ESP8266_4M1M
│   └── ./ESP8266_neopixel_ESP8266_4M1M/ESP_Easy_mega_20241210_neopixel_ESP8266_4M1M.bin
└── ./ESP8266_normal_ESP8266_4M1M
   └── ./ESP8266_normal_ESP8266_4M1M/ESP_Easy_mega_20241210_normal_ESP8266_4M1M.bin

No issues at all.

[Dickesplumpel]

ESP_Easy_mega_20240331_normal_ESP32c3_4M316k_CDC to ESP_Easy_mega_20241012_normal_ESP32c3_4M316k_CDC on ESP32-c3 super mini
I'm missing ESP_Easy_mega_20241210_climate_ESP32solo1_4M316k_LittleFS, ESP_Easy_mega_20241210_climate_ESP32solo1_4M316k_LittleFS_ETH doesn't work on shelly-plus1, bootloop.

[TD-er]

Check!
Not sure if ETH is the issue here.
Do other solo1 builds work?

[Dickesplumpel]

Update from ESP_Easy_mega_20231223_climate_ESP32solo1_4M316k_LittleFS to ESP_Easy_mega_20241210_climate_ESP32solo1_4M316k_LittleFS_ETH gone wrong. A flash with ESP_Easy_mega_20241210_climate_ESP32solo1_4M316k_LittleFS_ETH.factory doesn't work ether. I will try it again tomorrow with normal build.

[TD-er]

Check!
I will later this evening update the ESP-IDF version for the solo1, and will post here when it is ready.

[chemmex]

Ready to test specifically local AP restart after loss of main WiFi connection.

ESP_Easy_mega_20241210_normal_ESP32c3_4M316k_LittleFS_CDC_ETH.factory.bin, doesn't restart so far

[TD-er]

@Dickesplumpel
Here is a build running for the solo1 update: https://github.com/letscontrolit/ESPEasy/actions/runs/12260984392

@chemmex
Correct, as that is part of the WiFi/network rewrite which is not done yet

[chemmex]

Correct, as that is part of the WiFi/network rewrite which is not done yet

Oh, I see, I mistakenly thought you are doing it incrementally
Will wait for that fully armed.

[iz8mbw]

and after the WiFi rewrite there will be a surprise/present made by @TD-er : the Bluetooth Low Energy support in ESPEasy :-)

[Dickesplumpel]

ESP_Easy_mega_20241210_collection_G_ESP8266_4M1M no issues.
Update from ESP_Easy_mega_20231225_climate_ESP32solo1_4M316k_LittleFS to ESP_Easy_mega_20241210_normal_ESP32solo1_4M316k_LittleFS_ETH does not work, stays on 20231225, doing a clean flash with factory.bin later.

[TD-er]

I will have a look at it later. Not sure which one of my Shelly devices is a single core though, so have to check.

I know there have been issues with the recent solo1 platform builds, but I thought those were fixed by Jason2866.
I will ask him about it

[Dickesplumpel]

Fresh clean install with Bin-ESP32SOLO1-normal_ESP32solo1_4M316k_LittleFS_ETH-20241210_PR_5175_12260984392.factory.bin, serial says this block in a loop:

rst:0x8 (TG1WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 188777542, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:268
ho 0 tail 12 room 4
load:0x40078000,len:13056
load:0x40080400,len:4
load:0x40080404,len:2724
entry 0x40080534

Same with CollectionG.

[TD-er]

Talked with Jason2866 and he pointed out the error I made.
So this build should hopefully fix the solo1 crashing: https://github.com/letscontrolit/ESPEasy/actions/runs/12273878227

[fly74]

Is it only for the solo1 oder a globally change?

[TD-er]

Is it only for the solo1 oder a globally change?

I also updated to the same (newer) ESP-IDF code for other LittleFS builds, so stricktly speaking also other LittleFS builds will differ from the build I put on the web flasher page.
If this is working with the solo1 boards, I will also put that build on the web flasher page.

[Dickesplumpel]

No, ESP_Easy_mega_20240822_normal_ESP32solo1_4M316k_LittleFS_ETH works fine, upgrade to Bin-ESP32SOLO1-normal_ESP32solo1_4M316k_LittleFS_ETH-20241211_PR_5175_12273878227 or fresh install with factory.bin gets same result as earlier described.

[TD-er]

Jason also confirmed this build is failing on his Solo1.
So have to dig further to see why this is failing.

[fly74]

It looks like we have an issue with sending values with http eg.

/usr/bin/curl http://esp/control?cmd=TaskValueSet,7,1,40
curl: (56) Recv failure: Connection reset by peer

often results in this error. Also with IP without DNS name.

[TD-er]

Which build?
LittleFS or SPIFFS?

[fly74]

ESP_Easy_mega_20241210_normal_ESP32_4M316k.bin

[TD-er]

Is it possible to test it on some (other) node using a LittleFS build?
This fix is not included in the SDK used for SPIFFS builds: espressif/arduino-esp32#10640

[fly74]

Back to ESP_Easy_mega_20240822_normal_ESP32_4M316k works fine.

[fly74]

Is it possible to test it on some (other) node using a LittleFS build? This fix is not included in the SDK used for SPIFFS builds: espressif/arduino-esp32#10640

no running ESP with SPIFFS atm, but can you build one for this bin?

[TD-er]

Built a Custom Build on ESP32 and after some time the Web Interface does not respond:

What is set in the Custom.h?
When was the build made? Yesterday there has been an important fix which for sure is needed or else all kinds of network related issues will occur.

[TD-er]

Just uploaded the latest build with fixes for the Solo1 builds (with help from and tested by @Jason2866 )
https://td-er.nl/ESPEasy/latest/

@Dickesplumpel Can you test with your Solo1 board?

[fly74]

Only Solo1 new?

[TD-er]

Yep.
The build I made on the 16th fixed the networking issues for all other ESP32-xx variants and this one fixes the bootloop on the Solo1, with no changes in code for other builds.

[fly74]

Then the x-mas build is coming ?

[iz8mbw]

Built a Custom Build on ESP32 and after some time the Web Interface does not respond:

What is set in the Custom.h? When was the build made? Yesterday there has been an important fix which for sure is needed or else all kinds of network related issues will occur.

After a reset it's now up since 16 hours.
So for now better to ignore what happened to me

[TD-er]

Then the x-mas build is coming ?

Yep, looks like it :)
Last weekend I didn't think it would, but now the major blockers are fixed and I have to say that one in Arduino's NetworkClient class was a nasty one as it was about uninitialized variables which makes it really hard to reproduce.
Glad it is fixed now :)

[Dickesplumpel]

Update from 20140822 to ESP_Easy_mega_20241218_climate_ESP32solo1_4M316k_LittleFS_ETH and fresh install with ESP_Easy_mega_20241218_normal_ESP32solo1_4M316k_LittleFS_ETH works but configuration changes except rules are not saved, rules are not shown unless marked again.

[TD-er]

Can you show the part of the sysinfo page where it displays info about the file system and partition layout?

You didn't by any chance flash it using the ESPHome flasher?

I guess the filesystem isn't 316k, but probably more like 64k, which isn't even enough to store the main settings.
Could also be damaged by all these bootloops.

If the file system is too small, you should erase the entire flash.
Using the web flasher, you can check the 'erase entire flash' checkbox when flashing.

Or flash a 'blank' bin file to it first.

[Dickesplumpel]

I have erased and flashed with web flasher.
Storage
Flash Chip ID: Vendor: 0x20 Device: 0x4016 (Embedded)
Flash Chip Real Size: 4096 [kB]
Flash IDE Size: 4096 [kB]
Flash Chip Speed: 80 [MHz]
Flash IDE Speed: 40 [MHz]
Flash IDE Mode: DIO
Flash Writes: 0 daily / 0 boot
Sketch Size: 1465 [kB] (1856 kB free)
Max. OTA Sketch Size: 1856 [kB] (1900544 bytes)
Little FS Size: 316 [kB] (16 kB free)
Page size: 256 [byte]
Block size: 8192 [byte]
Number of blocks: 39

[Dickesplumpel]

Flashed with flash download tool 3.9.6 did it (first blanked with 4MB.bin). As far as I can tell everything works.

[Dickesplumpel]

Clean install with web flasher also works, don`t know what happened the first time ;-)

[TD-er]

What have you stored on the file system?
It reported to only have 16k free, which is not enough to properly work.

[Dickesplumpel]

Nothing. I have set up wifi, mqtt, ntp, dst and a few rules.
That was from the flash that was not storing anything but wifi settings, now it has 148kB free:
Storage
Flash Chip ID: Vendor: 0x20 Device: 0x4016 (Embedded)
Flash Chip Real Size: 4096 [kB]
Flash IDE Size: 4096 [kB]
Flash Chip Speed: 80 [MHz]
Flash IDE Speed: 40 [MHz]
Flash IDE Mode: DIO
Flash Writes: 0 daily / 0 boot
Sketch Size: 1465 [kB] (1856 kB free)
Max. OTA Sketch Size: 1856 [kB] (1900544 bytes)
Little FS Size: 316 [kB] (148 kB free)
Page size: 256 [byte]
Block size: 8192 [byte]
Number of blocks: 39

[TD-er]

OK, then there was something really corrupt in the previous file system you had, which does explain why you had those issues.

[fly74]

I don't want to hurry up, but x-mas is coming.

[Dickesplumpel]

In my opinion it is better to have a rock stable build than a x-mas build.

[TD-er]

What else is there to fix then?
I was planning on starting the releasbot script today

[tonhuisman]

I have a couple of useful fixes in the PR I linked a few comments above... 👼

[TD-er]

I just put up a last test build before release here: https://td-er.nl/ESPEasy/latest/
It contains the latest changes Ton made with respect to parsing of system variables for controllers.

The order of parsing these matters or else some %..% notation parts could already have been replaced.

However I do feel better if these could be tried first by more than only Ton and the person who reported the issue.

So if y'all could do some final test, please, then I will make a release tomorrow so there is no confusion about the date in builds.

[TD-er]

I found another issue myself...
On (some) ESP32-C3 boards you can now select flash pins, which should not be allowed.
At least not on those which are 100% for sure connected to flash....

[chemmex]

Clean install with ESP8266 is OK now, all settings are active. Thanks @tonhuisman!

No issues with parsing variables in controller settings, at least with C005.

[fly74]

New test with:

ESP_Easy_mega_20241221_climate_ESP8266_4M1M
ESP_Easy_mega_20241221_minimal_core_312_ESP8266_1M_OTA_Domoticz_MQTT
ESP_Easy_mega_20241221_neopixel_ESP32_4M316k
ESP_Easy_mega_20241221_neopixel_ESP8266_4M1M
ESP_Easy_mega_20241221_normal_ESP32_4M316k
ESP_Easy_mega_20241221_normal_ESP8266_4M1M

[fly74]

New test with:

ESP_Easy_mega_20241221_climate_ESP8266_4M1M ESP_Easy_mega_20241221_minimal_core_312_ESP8266_1M_OTA_Domoticz_MQTT ESP_Easy_mega_20241221_neopixel_ESP32_4M316k ESP_Easy_mega_20241221_neopixel_ESP8266_4M1M ESP_Easy_mega_20241221_normal_ESP32_4M316k ESP_Easy_mega_20241221_normal_ESP8266_4M1M

No issues so far.

[Dickesplumpel]

ESP_Easy_mega_20241221_normal_ESP32s2_4M316k_LittleFS_CDC_ETH
ESP_Easy_mega_20241221_normal_ESP8266_4M1M
ESP_Easy_mega_20241221_climate_ESP32c3_4M316k_CDC
ESP_Easy_mega_20241218_normal_ESP32solo1_4M316k_LittleFS_ETH
running about 2,5 hours. no issues so far.

[thomastech]

OTA Flashed: ESP_Easy_mega_20241221_collection_B_ESP32_4M316k
No issues observed.

• Thomas

[iz8mbw]

No issues on ESP32 boards with Custum Builds "ESP_Easy_mega_20241221_custom_ESP32_4M316k_LittleFS_ETH"!!

[TD-er]

Everyone thanks for testing.
It resulted in some really nasty bugs finally getting fixed, so an extremely productive session :)

And merry X-mas to y'all