Skip to content

Commit

Permalink
Update climate_data_processing_with_dask_extremly_slow.md
Browse files Browse the repository at this point in the history
  • Loading branch information
cozzolinoac11 authored Jun 6, 2024
1 parent ad15225 commit f2eff9f
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The scope is to process 1 year of hourly climate data (around 14GB in NetCDF) to
This problem causes execution time and resource consumption to increase exponentially (up to 10x in some cases).

## Affected component of FAIRiCUBE-Hub
Storage, CPU, RAM and Network
FAIRiCUBE Lab (Storage, CPU, RAM and Network)

## Proposed solution
Traditional file formats (e.g. tiff, netCDF) cause a lot of network traffic and slow down the computation when the file resides on the cloud. <br> Cloud-optimized format like COG, zarr are designed to overcome this problem.
Expand Down

0 comments on commit f2eff9f

Please sign in to comment.