[parallel] If not num_threads, use queue instead of tail recursion

[mpsijm]

Previously, running bt fuzz --jobs 0 would crash with a stack overflow after 244 iterations (±4 stack frames per iteration). This happened because after every GeneratorTask, a new task was started by calling Parallel.put, which in turn called its lambda, which called the new GeneratorTask, which called Parallel.put again in finish_task, etc.

I've rewritten the task handling in the case of not num_threads by moving it all to Parallel.done. Parallel.put now puts the task in the same priority queue as in the case where we do want parallelism (num_threads > 0). The fuzzer should now be able to run more than 244 iterations again 😄

I created a PR instead of simply pushing to master, since I haven't touched all of this parallel stuff that often, so some extra eyes would be nice 🙂

To quickly reproduce the bug with `bt fuzz -j0`, I added a minimal generator to the test problem "hellowholeworld", which makes 244 iterations take about 2 minutes:

diff --git a/test/problems/hellowholeworld/generators/gen.py b/test/problems/hellowholeworld/generators/gen.py
new file mode 100644
index 0000000..829afc9
--- /dev/null
+++ b/test/problems/hellowholeworld/generators/gen.py
@@ -0,0 +1,6 @@
+#!/usr/bin/env python3
+import random
+import sys
+
+random.seed(sys.argv[1])
+print(random.randint(1, 100))
diff --git a/test/problems/hellowholeworld/generators/generators.yaml b/test/problems/hellowholeworld/generators/generators.yaml
new file mode 100644
index 0000000..2f90cce
--- /dev/null
+++ b/test/problems/hellowholeworld/generators/generators.yaml
@@ -0,0 +1,9 @@
+solution: /submissions/accepted/test-hello.py
+data:
+  sample:
+    data:
+      "1":
+        in: "1"
+  secret:
+    data:
+      - "": gen.py {seed}

[mpsijm]

Better late than never 😄

While trying to resolve your feedback, I came to the conclusion that having one class that's doing very different things in the sequential vs. the parallel case is not very maintainable. So, I decided to split the two functionalities into two subclasses of a common abstract superclass.

The consequence of this is that @mzuenni's fix for the fuzzer in #327 is no longer necessary: the user of parallel.create can always transparently treat it as if it was a parallel queue, even though it's secretly a sequential queue when args.jobs == 0 😄

[RagnarGrootKoerkamp]

Mostly LGTM; just some small comments.

Still not entirely convinced we actually need this but happy to merge either way.