dataset quality

[KiaraGrouwstra]

• to increase dataset diversity, sample instead of using hard limits
• keep not just the function with the fewest number of AST nodes, but the more generic one (most type variable names/occurrences) if possible - I made an attempt in this direction, but decided that for generation this shouldn't matter much (maybe ditch dupes of validation/test set tasks but otherwise keep training dupes), while for synthesis this should probably be part of the reward function...
• I now assume all type vars are of kind *, but I should check this in findTypeVars and return like (HashMap String Int) there! - this ends up manifesting as a lack of type_fn_instantiations for e.g. Foldable t => t a -> Bool, i.e. test that for instantiateTypes.
• allow type variable constraints

out of scope:

• dedupe out only functions equivalent to those in validation/test sets, having redundancy within training seems okay
• pick generic blocks amenable to strong polymorphic typing, constructing task functions from those
• allow generating new type vars
• type variable name generation
• [padding] with unit type () according to arity probably doesn't scale to monads' arity=2 as () is kind *
• randomFnType: ensure each type var occurs at least twice
• point-free tools like function composition (.) and param flipping (flip) if not allowing doing point-ful style (variables/lambdas #8)
  • is my hole variant thing even supportive of passing in-program arguments to functions calculated by combinators like flip?