notes about flepimop incremental use / operational runs / etc

[pearsonca]

Notes about supporting enabling more "building" style in flepimop - that is, running models / inference of increasing complexity.

• config files need to do more:
  • support non-date time setting. something like "duration", and then any two of start_date, end_date, duration are required.
• config files need to do less:
  • control via environment variables is a bit wild. makes more sense to support multiple config files or arguments that override config file values
• "spatial_setup" isn't really spatial, its patch structure
  • perhaps this should be a population key, and then support either just a number (for non-patch models) or the dictionary keys for sizes, mixing
  • the select key needs more documentation
• support use of patching and references in config files
  • patching: allow -c config1.yml, config2.yml, ... - each subsequent file is "patching" the previous file. adding / overwriting the keys from from previous files. need to understand deep vs shallow patches?
  • references: using reference variables for certain arguments, to point to other config files that provide the details. kind of like patching?

[jcblemai]

I don't necessarily agree with the config nee to do more (or maybe, I perhaps do, but I think the bit on duration is less relevant to this discussion). However, yes, ENVAR are wild and these are a sources of error. The general philosophy should be "everything is on the config: where you resume from, if it's stochastic, if we discard chimeric, ..." AND some envar can override some config options if needed (e.g, you specify your # of chain and iterations, but you want to run a smaller test, ..).

I think spatial_setup is called subpop_setup now, but if you have seen it there must be a leak somewhere. Let us know. I would add that we should support population and subpop names as list inside the config and not only as external files.

For your last point, our config module already supports loading several config with each one shadowing similar keys to the other, so it's probably some very easy step to support (found this out while I was writing test, if I don't use config.reset() then all my configs from previous test get added). So we might be very close to that compared to reference/patching.

[pearsonca]

I think spatial_setup is called subpop_setup now, but if you have seen it there must be a leak somewhere. Let us know. I would add that we should support population and subpop names as list inside the config and not only as external files.

This seems to just be a documentation error, which I've now fixed.