Single machine testing (#422)

* add new collection for single machine test

* modify test_study to add single machine collection

* Added new test versions for single machine testing

These test versions drop devices that cause exceptions if they are listed but not actually available in the test environment.

* minor cleanup in client.py

* work-in-progress on booting correctly configured servers

* Delete nb_table_exporter.py

* Update .gitignore

ignore server_pids.txt, which is created in the repo folder when running from system tests against the local development environment

* Update gui.py

fix typo in comment

* documentation updates

* Create add_test_user.sql

* added logging statement to gui

* Delete test_server.py

* Allow metadator to skip SSH tunneling if db is running on localhost

* Update gui.py

* Update server_acq.py

* Update server_stm.py

* update config to work on single machine

* Update single_machine_testing.md

* add support for localhost as db host

* update client and server

* Update single_machine_testing.md

* Avoid recreating existing Windows Scheduled Tasks

* Update gui.py

* Update gui.py

* Update gui.py

* cleanup docs

* Update .gitignore

* Update single_machine_testing.md

* Update single_machine_testing.md

* update docs and configs

* Update single_machine_testing.md

* Update single_machine_testing.md

.gitignore

@@ -70,3 +70,6 @@ venv-*
 
 # Logs
 *.log
+
+# Misc
+server_pids.txt

docs/single_machine_testing.md

@@ -0,0 +1,65 @@
+# Single-Machine Testing
+
+This document describes the steps required to run Neurobooth on a single machine for testing purposes. There is currently no support for running a production neurobooth on a single machine.
+
+Note: You must be running Windows 10 or greater to run Neurobooth on your machine.
+
+## Configuration
+To run on a single machine you must modify the neurobooth config file, typically "neurobooth_os_config.json". 
+The following changes should be applied:
+- The entries for acquisition, control, and presentation must have name, user, and password all set to  "".
+- The entries for acquisition, control, and presentation must use different ports since they will all be on the same host.
+- TODO: Describe how to set the monitor width, etc.
+
+If you're running the db on the same machine as the neurobooth code, the following additional changes are required:
+- The database host value should be set to the loopback address "127.0.0.1"
+- You will likely also need to update some of your other database params (e.g. user and password)
+
+For more information on configuration settings, please see: [system_configuration.md](system_configuration.md) 
+
+## Servers
+ACQ and STM servers are started via the GUI using WMI. See the [WMI instructions](enable_WMI_instuctions.txt) document to get started. 
+WMI invokes the server_acq and server_stm batch file scripts to run the servers. The first the scripts are invoked this way
+they are added to the Windows Task Scheduler and then run almost immediately afterward. You can view the tasks in the scheduler for troubleshooting.
+
+Please Note: When the tasks are first added to the scheduler, they are created with a number of default settings, one of which
+causes the task to not run if the machine is not running on AC power.  You should probably change that if you plan to work 
+on a laptop and may run on battery. 
+
+## Database
+You can run the Neurobooth servers on one machine without also running the database on that machine. 
+Sometimes however, it is helpful to also run the database on the same machine. For example, you may want to avoid using your VPN or to perform testing while disconnected from the Net. 
+
+The first step is to install a working copy of PostgreSQL and PGAdmin.  While PGAdmin isn't strictly necessary, the steps described here will assume it is used.
+
+1. Create a database on the local server
+2. Make a schema-only backup of a current Neurobooth database, from either production or another test environment.
+3. Run a SQL script to add a single user. The file extras/add_test_user.sql contains SQL to add the user.
+
+When running on one machine, we do not use an SSH tunnel to connect to the database. 
+
+## Studies and Task Collections
+Since your local device is unlikely to have the standard devices, a new task collection called "test_no_eyelink" 
+has been added to the "test_study" study config in the examples folder. 
+Several new tasks were added to this collection. These tasks: 
+- altern_hand_test_obs_1
+- foot_tapping_test_obs_1
+- sit_to_stand_test_obs
+are variations on real tasks, but have fewer device requirements. 
+This collection is currently the only one that will work without real devices being attached to the machine used for the test environment.
+
+## Testing your code in PyCharm
+The easiest way to run the system on a single machine is to create a Run Configuration in PyCharm that executes gui.py. WIth this 
+approach, any changes made to the code can be tested with a mouse click. 
+
+Note: PyCharm should be started as Administrator. This will ensure that the PyCharm interpreter will have adequate permissions.
+
+When running from your development environment, Neurobooth will add a server_pids.txt file to the neurobooth_os folder. This can  be ignored and will be ignored by git. 
+
+### Debugging in PyCharm
+In order to run the GUI from within PyCharm, we must add PYDEVD_USE_FRAME_EVAL=NO to the list of environment variables specified in the PyCharm run configuration. 
+This is due to a known issue that occurs when running the PyCharm/python debugger on a QT based GUI.
+The issue causes a python interpreter crash with the error message "Process finished with exit code -1073741819 (0xC0000005)"
+
+Currently, only the gui process can be run in PyCharm's interactive debugger
+

docs/system_configuration.md

@@ -10,7 +10,7 @@ Consult the release notes for any modifications to the example files, as these m
 
 ## Config files
 At least one config file (neurobooth_os_config.json) should be modified as necessary and placed in the folder named in the NB_CONFIG environment variable. (See below)
-Any additional config files required by the system in the future will also be placed in this folder or in a sub-folder. It is expected that device-specific config files will be placed in a sub-folder called "devices".
+Any additional config files required by the system will also be placed in this folder or in a sub-folder. It is expected that device-specific config files will be placed in a sub-folder called "devices".
 
 ## Environment Variables
 Several environment variables must be setup to run neurobooth. An example Windows batch file can be found that creates all the required variables (or updates them if they already exist). 

examples/configs/collections/test_no_eyelink.yml

@@ -0,0 +1,7 @@
+collection_id: test_no_eyelink
+is_active: true
+task_ids:
+- altern_hand_test_obs_1
+- foot_tapping_test_obs_1
+- sit_to_stand_test_obs
+arg_parser: iout.stim_param_reader.py::CollectionArgs()

examples/configs/studies/test_study.yml

@@ -4,6 +4,7 @@ collection_ids:
 - testing
 - et_calib_test
 - test_mvp_030
+- test_no_eyelink
 irb_protocol_number: 0
 protocol_version: null
 consent_version: null

examples/configs/tasks/altern_hand_test_obs_1.yml

@@ -0,0 +1,14 @@
+task_id: altern_hand_test_obs_1
+feature_of_interest: motor
+stimulus_id: altern_hand_mov_task1
+instruction_id: alt_hand_1
+device_id_array:
+- Intel_D455_1
+- Intel_D455_2
+- Intel_D455_3
+- Mbient_LH_2
+- Mbient_RH_2
+- Mbient_RF_2
+- Mbient_LF_2
+- Mbient_BK_1
+arg_parser: iout.stim_param_reader.py::RawTaskParams()

examples/configs/tasks/foot_tapping_test_obs_1.yml

@@ -0,0 +1,14 @@
+task_id: foot_tapping_test_obs_1
+feature_of_interest: foot movement
+stimulus_id: foot_tapping_task_1
+instruction_id: foot_tap_1
+device_id_array:
+- Intel_D455_1
+- Intel_D455_2
+- Intel_D455_3
+- Mbient_LH_2
+- Mbient_RH_2
+- Mbient_RF_2
+- Mbient_LF_2
+- Mbient_BK_1
+arg_parser: iout.stim_param_reader.py::RawTaskParams()

examples/configs/tasks/sit_to_stand_test_obs.yml

@@ -0,0 +1,14 @@
+task_id: sit_to_stand_test_obs
+feature_of_interest: body movement
+stimulus_id: sit_to_stand_task_1
+instruction_id: '1'
+device_id_array:
+- Intel_D455_1
+- Intel_D455_2
+- Intel_D455_3
+- Mbient_LH_2
+- Mbient_RH_2
+- Mbient_RF_2
+- Mbient_LF_2
+- Mbient_BK_1
+arg_parser: iout.stim_param_reader.py::RawTaskParams()

examples/environments/staging/neurobooth_os_config.json

@@ -17,15 +17,15 @@
         "name": "stm-staging",
         "user": "STM",
         "password": "5519",
-	    "port": 12347,
+	    "port": 12348,
         "local_data_dir": "C:/neurobooth/neurobooth_data/",
         "bat": "%NB_INSTALL%/neurobooth_os/server_stm.bat"
     },
     "control": {
         "name": "ctr-staging",
         "user": "CTR",
         "password": "5519",
-	    "port": 12347,
+	    "port": 12349,
         "local_data_dir": "C:/neurobooth/neurobooth_data/"
     },
     "database": {

extras/add_test_user.sql

@@ -0,0 +1,3 @@
+INSERT INTO public.subject(
+	subject_id, first_name_birth, middle_name_birth, last_name_birth, date_of_birth_subject, country_of_birth, gender_at_birth, birthplace, guid, old_subject_id, redcap_event_name, test_subject_boolean)
+	VALUES ('100001', 'Taylor', 'Alison', 'Swift', '1979-07-18', 1, 2, 'Allentown', '', '', 'enrollment_arm_1', true);