Transfer Processed data to Delta Lake Repository

[j-knudson]

Parent Feature: #25
Prerequisite Ticket: #26

Technical Details

Brainwave data donated for use in training Machine Learning algorithms must be processed with the file_shuffler app for privacy. This data then needs to be housed in a Delta Lake Repository. There are two versions of the file shuffling script: https://github.com/3C-SCSU/Avatar/blob/main/server/file-shuffler_linux.py and https://github.com/3C-SCSU/Avatar/blob/main/server/file-shuffler_win.py

The data is currently stored directly on the VPS with path: /home/Avatar/Deployment/data

Either

1. create a connection from this directory to the Delta Lake repository, or
2. create a direct transfer of processed data to the Delta Lake repository.

Of those two choices, the second option is preferred and the long term goal. If necessary, it could be implemented on a future ticket.

Acceptance Criteria

• Processed data can be transferred to a Delta Lake Repository.
• In creating the method for transferring the data to the repository, consider the long term goal of completing the transfer from the GUI