-
Notifications
You must be signed in to change notification settings - Fork 0
Sprint 22
- Push PR for landing page changes
- Implement local and S3 raster tiles - Drew
- Develop concept for storing static masks - Tim - backlogged
- Describe required masks - Nick/Drew/Mandip
- Store glacial and waterbody footprint maps
- Add DEM entry to targets table - Tim - backlogged
- Storing of DEM data - Tim
- Implementing a observation log table - Tim
-
single command average coherence matrix production- backlog - Mandip
- args: target, method
-
Gacos atm correction - Mandip - On Pause - develop workflow to apply a single atmospheric correction - Mandip - [ ] Reprocess 2-3 beam stacks to get files for atm correction analysis: Nick - backlog - incidence angle in map coords - IFG in RDC - DEM in RDC - Open conversation with Gacos Dev team
-
RAiDER Atmospheric Correction - Mandip
- Investigate use - Mandip
- Generate test correction - Mandip
- Document Weather model requirements - Mandip
- ECMWF Costs
- blocked by weather model access
-
Raster Tiles - Mandip
- Test on Dash leaflet - Drew/Mandip
- Test on S3 directly - Drew/Mandip
- Geoserver bug crashing unexpectedly
- Investigate why it's crashing - Tim/Drew/Nick
- Migrate S3 IFG storage to EFS - Drew/Tim/
- Investigate the use of pre-cached raster tiles
- Implement EODMS product upload workflow
- create database record in SLC table
- Implement EODMS Ordering/Download
- Flag to Ignore SpecialHandlingData
- Download and move image product to S3
- Create SQS job in AWS to start processing/
- Implement HyP3 processing/download for test site - Edgecumbe - Mandip - backlog
- Try running on eodms-cli ec2 instance
- RCM MOU - Review backlog - Drew
-
Dedicated AWS (Staging) Environment - backlog - Tim/Drew
-
Automate CDK deployment
-
PosGreSQL
-
PostGIS - manually implemented on RDS
-
VRRC API
-
VRRC InSAR-Hub - backlog
-
S3 Buckets - backlog
-
VRRC-Dashboard - backlog
-
InSAR Hub - backlog
-
IAM Roles, Security Groups etc., cloudwatch events etc. - backlog
-
-
Develop rough cost estimate for API and DB - Tim
- Running $140 USD/month as of Jan 29, 2024
- Running $106 USD/month as of Feb 26, 2024
- Running $145 USD/month as of March 18, 2024
- Flow chart for deformation checking and escalation 50% - Drew
- Milestone Report due end of Fiscal - Drew/Nick
- Defining Global Volcano watch shapefile for CSA - Drew/Nick
- High Frequency -
- Medium Frequency - on hold
- Low Frequency - on hold
- MOU
- Reach out to Hugo Drouin re: EODMS VAP API
- Draft MOU with U-Bristol Research Team/COMET
- INSAR Data sharing EULA - Melanie/Drew
- Mandip -
- Tim -
- Drew -
- Nick -
Scrum Notes
Tuesday March 19, 2024
Mandip - Fixing InSAR sharing release script
- Fixing 5M18
- Focusing on Raider next
Tim - Get InSAR hub running in Dev with changes, acquiring EODMS metadata - Working with Squawk for EEW
Nick - Rolling out Seiscom
Drew - Interviewing Summer students - Dashboard - Alerting flowchart
Thursday March 21, 2024
Tuesday March 26, 2024 Tim - Working on EODMS updating running in dev environment - pushed change, ability to handle queries with dates - ability to fetch all studies - important for deployment
Drew - Mandip duap_processor - Approving raster tiles PR - Issues getting raster tiles working in Dash
Mandip - working on Raider - fixed weather model issue - getting access to EMCWF - generating test correction - drafting Reykjanes OHS order
Thursday March 28, 2024 Tim - Working on InSAR hub, same status as Tuesday
Mandip - Sent OHS order for Iceland, 3 beam modes only 5M4, 5M17, 5M13 (Ascending) - Working on Raider - Observatory field trip today
Drew - Duap_insar_processor PR approved - VRRC Dashboard changes working. Fixing linting errors - Bug getting rastertiles to work - RE: Hugo running EODMS VAP via API
Thursday April 4, 2024 Tim - Working on InSAR Hub running, fetching metadata on a regular basis - All SLC meta data back to 2020
Nick - Trying out new workbench, PR review - Not a ton of time to contribute
Drew - Coordindate AWS Services - - Potential IT student
Mandip - Finishing up detailed instructions for duap_insar_processor deployment - Finish updates to Geoserver instructions - Continuing working on Raider -
Tuesday April 8, 2024 Tim - Reviewing Dashboard PR - Prepping for AWS Pro Serve Meetings - Prepping for Whiteboarding session - Starting Acquiring EODMS Data - Implement Observation log table
Mandip - Working on Year end Assessment - Continue on with Raider - EODMS CLI PR (Adding Kilauea, Mexico City, and Merapi, Iceland 5M23) - Try linear regression model
Drew - Prepping for the whiteboarding - RCM orders Standard coverage - Prepping AWS pro serve meeting - Push sharing EULA - Meeting SFU student- planning to share 5M3 data - Pushed changes to Dashboard
Tuesday April 11, 2024 Mandip - finished performance review - Working on Raider - Afternoon full of meetings - working on all SLC pair tomorrow
Tim - Looking into what data InSAR hub is pulling compared to landmass sandbox -
Drew - Earth Observation Forum 2024 - Probably not going to get approved - Giovanni Fusina Presentation - Prep AWS Meeting - Making Simply Science Figures - Old AWS pro serve proposal
April 16, 2024 Tim - Starting to work on EODMS ordering and downloading working - Python side pretty simple - Made some changes to the InSAR-hub, missing some images - overlap bug
Drew - Prepping for the meeting AWS pro-services define scope - deformation alerting workflow - Presented to DRDC on Friday
Mandip - generating SLC - using basecalc - full stack - fixed barren island
Nick - 90% of time on seiscomp - dusted off VRRC dashboard for review -
April 16, 2024 Tim - Working on changes associated in VRRC API associated with EODMS ordering and downloading - Implementing "PUT" method for SLC route in API so that SLC records can be modified - This is needed, for example, so the insar-hub can modify the "status" of SLCs and input S3 URIs for image storage locations - At the same time, will implement query filters on SLCs to make querying SLCs easier (for example, ability to query SLC records from API filtered by beam mode or target)
Mandip - Working on potential IFG pairs script - Wrote two more Issues/Tasks, 1st is automating DEM generation for a specific site and 2nd is an issue with orbit state vectors - Will start with automating DEM generation for a specific site - Figuring out how to request OHS to reprocess L1 data so from MLC to SLC, if possible - Generated insar results using HyP3 (latest data Apr 15 scene) of the shallow earthquake insar candidate in Northeast BC, waiting on latest ztd image to apply atm corrections using GACOS due to high atm noise in apr 15 scene
April 23, 2024 Mandip - PR for IFG pair matrix pending - Looking into DEM automation - To look into RasterTiles in Dash
Tim - Changes to API for EODMS Acquisitions - Ability to modify SLC records - Ability to filter SLC requests - back to InSAR Hub
Drew - Writing up feasibility - Standing Coverage Working Group Adding Mt Baker - Playing AWS Batch duap_insar_processor - Waiting AWS pro-serve
April 25, 2024
Tim
- Figuring out order/download methods of py-eodms-rapi. Successfully ordered and downloaded from data using py-eodms-rapi from the Python console.
- Continuing to work on vrrc-api and vrrc-insar-hub to get automated order/download up and running.
Mandip - Fixing duap_insar_processor script PR review comments - DEM generation automation almost working - Raider - progressing but need full day focus to make progress - May 24 DM and ADM visit - Getting familiar with Dash - Working on coherence linear regression model
Drew - Working Earthquake risk applications - Working on presentations for NRCan Emergency Management - Deploy the updated dashboard
April 30, 2024
Tim - Working on EODMS order download - How to move SAR raw-data bucket - Bucket replication
Mandip - Working with Dash-leaflet - implementing raster tiles - managed to show a single image using imageoverlay
Drew - Yesterday gave a presentation DEOC NRCan - Schedule AWS Well architected review - Report writing on Milestone - Coherence Linear regression model
May 2, 2024 Tim - Moved entire SLC archive over to vrrc-dev - started on EODMS order command
Mandip - working on gdal2tiles - Getting tiles running in dash
Drew - Push approvals for EO forum - Making Deformation response plan - working milestone report - modelling coherence using linear-regression or polynomial fit
May 7, 2024 Tim - Working through EODMS ordering/download - save record id and collection id - next step is to start issuing orders - eodms downloads
Mandip - rastertiles working in basic dash script - Kiluea InSAR results - processing issues with baselines - presenting to ADM
Drew - Working Describing InSAR system
- Travel to EO Forum
- Scheduling 4 week sprints
- duap_insar_processing Batch deployment
May 9, 2023
Tim - working on order download system - ordering now working - soon image download should be working
Mandip - ADM visit preparations - Added 5M16 to geoserver and vrrc-api - For Kilauea - Draft order for 5M11 and 5M16 4-day repeat
Drew - Prepping - NISAR slides - Deployed latest of dashboard - Reach out Michelle IMO GNSS - Re Mike Poland
May 14, 2024
Mandip - DM/ADM visit preparations, Temporarily will overlay earthquakes over InSAR in dashboard for Brianna - Implementing rastertiles into VRRC dashboard
Tim
- InSAR HUB image download work in progress
- Now able to download images from EODMS and upload to S3 via hub
- Next step: Uploading downloaded image metadata to VRRC API
May 16, 2024 Mandip - Added earthquakes to dashboard site view - Reprocessing Edgecumbe InSAR - Working with Dash
Tim - Working on EODMS downloads - Download from EODMS and push to S3 - Next, inform API of download
Drew - Working on Reykjanes time-series - Deploying DUAP InSAR Batch
Nick - Still saturated with seismic work
May 21, 2024 Tim - Chipping away at EODMS order download - Hub can now download data from EODMS - Next steps, code clean up - Progressing through Well Architected Review Questionnaire
Mandip - Reprocessed some of Edgecumbe IFGs - RasterTiles - Earthquakes in Site View
Drew - Reykjanes time-series - M2 report - CodeDeploy
May 23, 2024
Mandip - Working on Dash Rastertiles - Works with local tiles - testing on s3 tiles - testing latest version of Dash
Tim - Slow dev last few days - W.A.R. and Peformance review
Drew - Working Reykjanes InSAR time-series - Prepping for new summer IT student - EO Forum
May 28, 2024
Mandip - Working on Dash Rastertiles - Successfully managed to display S3 hosted png files in our dashboard - Intermittently off this afternoon (May 28) from 1-3 pm EDT
Tim - Working on getting EODMS order/download running in dev environment via CDK