Develop the Oracle of DISCOVR

Team: Shockwave Surfers 🔗

NASA Space Apps Challenge Tacoma Location 🔗

---

Data Collection

NASA DSCOVR Experimentation Data

To download and extract the experimentation data, perform the following.

In the shell

Install the dependencies

pip install -r requirements.txt

Acquire the data, only once (if data is already there no need to repeat)

python3 01_preprocessing/01-aqcuire.py

Unzip the data, only once (if data is already there no need to repeat)

python3 01_preprocessing/02-unzip-data-files.py

Optionally delete the zip files

python3 01_preprocessing/03-delete-zip-files.py 

Canada Carisma Magnetometer Network

Data from Canada's magnetometer network can be downloaded from this site for the 2016 and 2017 years to validate the readings of the magnetic field vector from NASA's experimentation data. The .tar files will need to be downloaded to your personal system due to large file size.

Once the desired .tar files have been downloaded, you can run the following to extract the files from the .tar encryption. Be sure to change directory to the directory housing your tar file for extraction.

python3 file_path_to/tar_extract.py your_tar_file_name desired_directory_for_extracted_file

After the .tar file contents have been extracted to a directory for the year, nesting directories for month and days, then you'll need to extract the data from the gzip files.

From your terminal you can run gz_extract.py with the proper parameters to extract the data from the gzip files.

Utilizing code within carisma_preprocess_to_csv.ipynb, the text files extracted from the gzip files could then be cleaned, condensed, and put into a csv for ease of use.

Finally, carisma_preprocesses_data_selected_range.ipynb could then be used to select a specific range of data for assessment.

Analysis

'NASA_2016-2023_all_data.ipynb' 'NASA_2016-2023_condensed_data.ipynb' 'NASA_2016.ipynb' 'NASA_2017.ipynb' 'NASA_2023.ipynb' 'Untitled.ipynb'

Model

kP Index data: 'kP_index_data.csv' 'kp_predict_tf.ipynb' 'experimental.ipynb'

---

References

NASA and Canadian Space Agency

• develop-the-oracle-of-dscovr challenge description 🔗

• Experimentation Data 🔗

• Faraday Cup | Wikipedia 🔗

• DSCOVR portal 🔗

• Glossary: 🔗

• SME discussion | github 🔗

• https://donnees-data.asc-csa.gc.ca/dataset/98466021-2q1w-5g2d-677zwru214wx68

• https://omniweb.gsfc.nasa.gov/form/dx2.html

• https://carisma.ca/carisma-data/fgm-data-format

• https://science.nasa.gov/mission/dscovr/

• https://hpde.io/NASA/NumericalData/OMNI/PT1H

• https://omniweb.gsfc.nasa.gov/coho/form/helios1_f.html

• https://www.earthdata.nasa.gov/engage/open-data-services-and-software/data-information-policy/data-levels

• https://www.swpc.noaa.gov/products/planetary-k-index

  • -> Products and Data
    • -> Observations - Boulder magnetometer
      • https://www.swpc.noaa.gov/products/boulder-magnetometer
        • -> Data https://www.usgs.gov/programs/geomagnetism
          • -> Data -> Programmatic Access to Geomagnetism Data
            • https://code.usgs.gov/ghsc/geomag/geomag-algorithms (Geomag Algorithms is an open source library for processing Geomagnetic timeseries data. It includes algorithms and input/output factories used by the USGS Geomagnetism Program to translate between data formats, generate derived data and indices in near-realtime, and research and develop new algorithms.)

• papers

  • CCMC, Shreeya Khurana 🔗

  • Level one requirements, pg. 9 🔗

Space Weather

• Space Weather Prediction Center (SWPC) | NOAA 🔗

Python

• python-download-file-from-url realpython.com 🔗