Skip to content

Sprint 27

Drew Rotheram edited this page Oct 31, 2024 · 7 revisions

Sprint Notes

VRRC Dashboard

  • Temporary button to update coherence_matrix/baseline/Insar_pairs csv files - Chloe
  • Learn and redeploy InSAR Dashboard - remove - Chloe
  • Observation Log UI - backlog - Drew/Chloe
  • Test latest dash version - backlog - Mandip
  • Bilingual dashboard - backlog\
  • Change status to 'unassigned' after threshold

VRRC API

  • Implementing a observation log table 50% complete - backlog - Tim
    • Define observation data model -Tim /Chloe
  • Implement coherence matrix route - backlog - Tim
  • Implement baselines route - backlog - Tim
  • Implement potential pairs route - backlog - Tim
  • Populate legacy processed interferograms - backlog - Tim
  • Improvements to interferograms GET requests - Tim

DUAP-InSAR-Processor

  • CDK for running processor in Batch - backlog - Tim
  • Add displacement map rastertiles - new colour table required - backlog
  • relocate rastertiles to correct s3 bucket - Mandip
  • Develop manual time-series workflow - Mandip/Drew
  • Reprocessing Reykjanes with 0 or minimal baseline optimization - Mandip/Drew
  • Optimize Compute resources for DUAP-InSAR processing run - Mandip
  • Check for downlinked orbits and run RCM_orb_filt.py - Backlog - Mandip

VRRC-InSAR-Hub

  • Create Batch job in AWS to start processing - Tim

SAR Data Sources

System Architecture

  • Dedicated AWS (Staging) Environment - backlog - Tim/Drew

  • Automate CDK deployment

    • PosGreSQL
    • PostGIS - manually implemented on RDS
    • VRRC API
    • VRRC InSAR-Hub
    • S3 Buckets
    • VRRC-Dashboard
    • Batch process for Duap Insar Processor - Tim
  • Optimize S3 storage classes - Tim

    • Mohamed Amoura AWS Rep
    • Reorganize S3 files - Tim
      • Consolidate raw data bucket everything into A4200_Volcano folders
  • Deploying environment to new SSC LZA environment

  • Develop rough cost estimate for API and DB - Tim

    • Running $140 USD/month as of Jan 29, 2024
    • Running $106 USD/month as of Feb 26, 2024
    • Running $145 USD/month as of March 18, 2024
    • Running $154.67 USD/month as of April , 2024
    • Running $209.46 USD/month as of May , 2024
    • Running $248.00 USD/month as of June , 2024
    • Running $288.00 USD/month as of July , 2024
    • Running $354.17 USD/Month as of August, 2024
    • Running $437.00 USD/Month as of September, 2024
    • Running $772.00 USD/Month as of October, 2024

Documenting & Reporting

  • Milestone Report due end of Fiscal - Drew
  • 1-pager for vrrc review - Mandip/Chloe

Administrative

  • Reviewing AWS ProServe proposal - in progress
  • Reviewing requirements for LZA - Not urgent
  • Kickoff engagement with SSC Science Cloud Enablement
  • Request access to AWS LZA environment - Drew/Tim
  • Reid/Charles meeting re:operationalization
  • Cloud Business Requirements (CBR) update - Drew

Demos

  • Mandip - Optimization of Compute resources
  • Chloe - Dashboard matrix update button
  • Tim - ???
  • Drew - Reykjanes time-series (Downlinked vs definitive vs no-baseopt)

Scrum Notes:

October 8th, 2024 Chloe - Focussing OpenDRR - Temporary button issue vrrc-dashboard - Leave InSAR Pairs all issue for later

Mandip - AWS Conference Tomorrow (Wednesday) - GSC Field Trip Ottawa (Thursday) - Moving Raster Tiles to correct bucket

Drew - Working on VRRC 1-pager - Milestone/Final Report - Time-series Reykjanes no-baseopt

October 10th, 2024 Chloe - added new temporary button to fetch latest csv files

Drew - Reviewing Dashboard and Duap_insar_processor PRs - NISAR Early Adopters workshop

October 22, 2024 Tim - Back from leave - looking into InSAR Hub getting auto processing working - Look into cloudwatch logs

Mandip - Transferring rastertiles s3 to s3 - finished slides for securewatch - investigating right sized ec2 instance

Chloe - Finished AI detecting research articles for AI detection

Drew - Working - Report writing - Batch jobs costs - reaching out to Mohammed Amoura

October 24, 2024 Tim - Working on changes on InSAR hub - create IFG record - Change API/HUB/CDK to trigger auto processing

Chloe - Running AI detection script

Mandip - Migrating Rastertiles in S3 buckets - Using larger instances with cloud9 - EC2 instance cost optimization exercise for Batch

Drew - Roughly AWS batch optimization - Report writing, paper writing - AWS proserve/operationalization coordination

Clone this wiki locally