feat: add some initial cargo-fuzz test for queues

[lulf]

I played with using cargo-fuzz for testing the queue behavior. I wanted to reuse the mock flash for that, but that would require gating it behind some hidden feature, not sure if you have a preferred way.

[diondokter]

Hmmm, cool!
Dario's suggestions are good.

Also, seems like the fuzz target folder, corpus and artifact folder are not gitignored

[lulf]

@diondokter I'm trying to have the fuzzer determine whether or not a push() operation should fail due to out of capacity, and for that I need to calculate the expected overhead of the metadata.

From what I can infer, the per-item overhead is 6 bytes, but aligned to word boundary, and at the start(?) of each page there is a byte controlling which state the page is in?

[diondokter]

From what I can infer, the per-item overhead is 6 bytes, but aligned to word boundary, and at the start(?) of each page there is a byte controlling which state the page is in?

1. There's two words that control the page state. The first and the last word. So the total space in a page is the page - 2 words
2. An item must fully fit on one page, they cannot be split over two pages. When an item doesn't fit in the remaining space of a page, the page is closed and the next page is used to write the item into.
3. The layout of an item is documented here:

   sequential-storage/src/item.rs

   Lines 1 to 20 in 9ffb7b6

   	//! Module that implements storing raw items in flash.
   	//! This module is page-unaware.
   	//!
   	//! Memory layout of item:
   	//! ```text
   	//! ┌──────┬──────┬──────┬──────┬──────┬──────┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐
   	//! │ : │ : │ : │ : : : : : │ : : : : : : : : : │ : : : : : │
   	//! │ Length │ Length' │ CRC │Pad to word align│ Data │Pad to word align│
   	//! │ : │ : │ : │ : : : : : │ : : : : : : : : : │ : : : : : │
   	//! └──────┴──────┴──────┴──────┴──────┴──────┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴───┴──┴──┴──┴─┴──┴──┴──┴──┴──┴──┘
   	//! 0 1 2 3 4 5 6 6+padding 6+padding+length 6+padding+length+endpadding
   	//! ```
   	//!
   	//! An item consists of an [ItemHeader] and some data.
   	//! The header has a length field that encodes the length of the data, a crc of the length (`Length'`)
   	//! and a crc field that encodes the checksum of the data.
   	//!
   	//! If the crc is 0, then the item is counted as being erased.
   	//! The crc is calculated by [crc16] which never produces a 0 or 0xFFFF value on its own.
   	//!

   
   There's this helper function too:

   sequential-storage/src/item.rs

   Lines 150 to 152 in 9ffb7b6

   	/// Calculates the amount of bytes available for data.
   	/// Essentially, it's the given amount minus the header and minus some alignment padding.
   	pub fn available_data_bytes<S: NorFlash>(total_available: u32) -> Option<u32> {

   
   Just give it the space left in the page and it'll tell you how much data can still be stored in the page. It's None if no data can be stored anymore.

[lulf]

@diondokter believe it has found one off-by-one bug in next_free_item (but please check it, see a521750).

Another fuzz test to add (which I'd like to add in a follow-up PR) is to take an existing 'image' that may be corrupted in random ways and see what happens.

Regarding the distribution, we can do that by using a custom type and implementing the Arbitrary trait. However, I think what's there is pretty good already.

Finally, there is a question of semantics and another potential bug. At the moment a push of an empty item will fail with StorageFull in some cases, and some cases not (depending on what happened before). I'd like to get your view on what the behavior should be if there is space for the aligned header but not the empty item. My thinking is that it should be allowed (could be useful as some kind of external marker?)

[diondokter]

Nice! I'll review later.

As for the 0 length, that is allowed. It didn't use to, so the weird behavior might be from that still. So yeah, I do consider that a bug

[lulf]

@diondokter Just a quick update, fixed a bug in the new find_max_fit() handling 0 length correctly.

[diondokter]

Alright, looks pretty good! Only a couple of small things.

[diondokter]

Btw, I just added a simple CI script on master.
We could add a small fuzz step that fuzzes for only like 10 seconds.

Not sure how much that would bring us, but at least the fuzzer would run every change, even if it's brief.

For statime we do that (though that still uses the old actions-rs): https://github.com/pendulum-project/statime/blob/main/.github/workflows/rust.yml#L101-L124

[diondokter]

Thanks a bunch!