EEPROM.commit() crash with large structure

[Sercurio]

When I try to commit a large structure (3984 bytes), my application crashes.

It's a multicore application where core0 handles button inputs, potentiometers, and the display, while core1 generates a PWM signal using an interrupt.

Previously, I managed each piece of data individually using variables and addresses, but I thought using a structure would be easier since I wouldn't need to handle each variable's address separately.

I tried placing the structure on the heap by initializing it in a function, but the problem persists.

When I slimmed down the structure and kept only a small amount of data (e.g., an array of 127 chars), the commit function no longer caused a crash.

I’ve been thinking: committing a structure writes every byte, which could reduce the lifespan of the flash memory, right?

Previously, I had a diff-based system that only updated the data that had changed:

if (data != EEPROM.read(DATA_ADDRESS)) {
    EEPROM.write(DATA_ADDRESS, data);
    diff = diff || 1;
}
if (diff) {
    EEPROM.commit();
}

I’d like your opinion: should I stick with my current system, or should I investigate why committing the structure doesn't work?

[earlephilhower]

Moved to a discussion since it's not really a core problem and more of a general question...

You have a 4K total stack, so if you try and allocate a 3900 byte structure on it (i.e. as a local in a function), you will end up going boom unless you're very lucky. So the heap space allocation/global is the way to do something like that.

As for the wear, remember that the Pico doesn't really have an EEPROM so we're just emulating one with a 4K flash sector. Basically every EEPROM.commit() will cause that 4K sector to be erased and the new data to be written as one-chunk. So whether you update 1 byte or 100 bytes, it is the same wear if you only commit once.

Theoretically the flash on the real Pico has something like 100K write cycles (it's NOR nor NAND) so if you do this commit once a day then you'll retire the device before the flash will be an issue. If you try and commit every few seconds, though, things may not be so good. FWIW, the EEPROM does verify that data was written before this erase/rewrite so your smart check is actually already implemented by the library:

arduino-pico/libraries/EEPROM/src/EEPROM.cpp

Lines 110 to 116 in 633faa1

	bool EEPROMClass::commit() {
	if (!_size) {
	return false;
	}
	if (!_dirty) {
	return true;
	}

The other option if you're really concerned about wear is LittleFS. You can write a file with the structure (and alternate between new/old) so that even if you lose power in the middle of an update you won't lose data. (If you're unlucky enough, it's possible to lose power after the flash erase and before the flash write using EEPROM.)

There are other, more clever things you can do for specific use cases, but you'd need to manually work with the flash and Pico SDK's APIs to implement them...

[Sercurio]

Sorry for opening an issue instead of a discussion.
Thank you for addressing most of my questions (such as clarifying that modifying a value with a write operation will overwrite the entire 4k bytes, not just the targeted address).

Regarding the lifespan of the chip's flash memory, I have implemented a mechanism to commit changes only when the user interacts with the interface and then doesn't touch it again for 5 seconds. This should help preserve the flash a bit longer.