Deduplicate OnDisk Corpus

[BAGUVIX456]

addresses #2434

Original PR (now closed) #2802

[tokatoka]

this is a lot ...
can you wait a bit

[tokatoka]

Hey @BAGUVIX456.
I discussed with others, but I think the previous direction is acceptable/better.
can you force push f06ed97 to the branch then we can merge after reviewing the remaining part

sorry for the confusion!

[tokatoka]

can you run ./scripts/fmt_all.sh and fix the errors in ci?

[BAGUVIX456]

yep, working on it

[tokatoka]

it seems it doesn't pass some tests

thread 'schedulers::queue::tests::test_queuecorpus' has overflowed its stack
error: test failed, to rerun pass `-p libafl --lib`

Caused by:
  process didn't exit successfully: `D:\a\LibAFL\LibAFL\target\debug\deps\libafl-6609fdc76bef418d.exe` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)
note: test exited abnormally; to see the full output pass --nocapture to the harness.

[BAGUVIX456]

Seems like fs2 is having trouble with windows, i'll have a look

[BAGUVIX456]

Any method that modifies the name of a testcase (rename_testcase() of InMemoryOnDisk corpus or Testcase::with_filename()) runs the risk of breaking deduplication. Would just adding a warning work for now or should I do something else?
The worst case scenario is that they will end up with duplicate testcases so it is not really a critical issue, but it needs to be considered nonetheless