Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New data interface ingest #46

Merged
merged 99 commits into from
Feb 9, 2024
Merged

New data interface ingest #46

merged 99 commits into from
Feb 9, 2024

Conversation

mfisher87
Copy link
Contributor

No description provided.

@mfisher87 mfisher87 marked this pull request as ready for review January 3, 2024 18:10
mfisher87 and others added 26 commits January 3, 2024 11:11
States, and all other region types, now come with full context from the
supercomputer.
* Refactor some code structure
* Ingest in to WIP directories
* Put everything in 3 more intuitive top-level `${STORAGE_DIR}` subdirs:
  `incoming`, `ingest-wip`, `live`.
* Update schemas for colormap/variables static files
* Cleanup:
  * "cogs", "points", "plots" dir placeholders
  * "shapes" dir and all shapes inside (these will be sent from
    supercomputer, and there will be too many regions to reasonably
    store in a Git repo (unless LFS)
* Migrate "legends" static data files to new static dir
* Add new "variables.json" and "colormaps.json" static data files, these
  are probably the only pieces that will be static. There may be a
  couple more...
* Add README to static dir
…c repo files

Migrated tasks into their own module
Ruff's isort ruleset doesn't want to combine "as" imports and regular
imports (even with `combine-as-imports` enabled). Maybe I need to
upgrade?
@nsidc-service-admin nsidc-service-admin force-pushed the new-region-ingest branch 11 times, most recently from d0fff1d to 4134bae Compare January 26, 2024 03:50
@mfisher87 mfisher87 changed the title New region ingest New data interface ingest Feb 9, 2024
@mfisher87 mfisher87 merged commit 68dd0dd into main Feb 9, 2024
2 of 3 checks passed
@mfisher87 mfisher87 deleted the new-region-ingest branch February 9, 2024 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants