Refactor Ersilia test command (#1224)

* Enable Ersilia to serve multiple models simultaneously (#1201)

* WIP Add session management utils

* Create a session at ersilia cli initialization

* Remove unused code

* start an ersilia session in a dedicated sessions dir that is mapped to the parent process' id which ran the given ersilia command, this will generally be a shell process inside a terminal, but it can also be a process from a bash script

* declare session specific defaults

* Run all ersilia commands within a single process during standard example run, otherwise ersilia run command does not find a served model bec of running in a different process and therefore in a different session

* Move the currently served model's pid to its dedicated session directory

* WIP Logging

* Redirect tmp logs to model's session logs

* catch permission error

* Redirect tmp logs to model's session logs

* don't use ersilia exception management because we don't exactly want the ersilia process to exit

* copy relevant files

* read from 3rd column, logging statements bash output file

* run_bash path edit, conda.sh current failed fix

* add new size methods, clean directory size method

* add additional logging of file types and count

* delete dead function

* fix run path

* uncomment testing

* fix bash output file opening (arg1)

* change var names for clarity and add debug prints

* addtional logging statements + updated read_csv method

* fixed updated_read_csv

* Delete comments and dead code

* clean up read_csv

* fix comparison method

* add prints

* model size calculation update

* delete comments, update state variables

* add time stamp

* add debugging print for thresholds

* update threshold debugging statements

* implement MRAE + Spearman's

* bash output file path log

* fix divide by 0 mrae calculation

* Update threshold

* bash script logging statements

* bold message debug

* generalize read_csv for different column numbers

* uncomment

* specify output

* include scipy dependecy in pyproject.toml

* delete dead function + comment

* delete unused var

* fairyfloss bckgrnd

* clarity code mrae calculation

* adjust control flow example method

* update metric calculations, delete comments

* Update comment wording

* remove prints and add log statements

* end test command statement

* fix log format + delete echo and prints

---------

Co-authored-by: Dhanshree Arora <DhanshreeA@users.noreply.github.com>

.devcontainer/devcontainer.json

@@ -36,7 +36,8 @@
                 "ms-python.python",
                 "ms-python.vscode-pylance",
                 "mohsen1.prettify-json",
-                "redhat.vscode-yaml"
+                "redhat.vscode-yaml",
+                "mintchipleaf.fairyfloss"
             ]
         }
     }

ersilia/io/input.py

@@ -267,10 +267,14 @@ def predefined_example(self, file_name):
                 return False
 
     def example(self, n_samples, file_name, simple, try_predefined):
-        predefined_done = False
+        predefined_available = False
         if try_predefined is True and file_name is not None:
             self.logger.debug("Trying with predefined input")
-            predefined_done = self.predefined_example(file_name)
-        if not predefined_done:
+            predefined_available = self.predefined_example(file_name)
+        elif predefined_available:
+            with open(file_name, "r") as f:
+                return f.read()
+        if not predefined_available:
             self.logger.debug("Randomly sampling input")
-            self.random_example(n_samples=n_samples, file_name=file_name, simple=simple)
+            return self.random_example(n_samples=n_samples, file_name=file_name, simple=simple)
+