ADD: Auto Read Function

[syedhamidali]

• Closes #xxxx
• Tests added
• Changes are documented in history.md

Description: Enhanced Radar File Reading Functionality with Timeout Support

Summary:

This PR introduces the auto_read.py module, which provides an enhanced read function designed to improve the radar file reading process in the xradar library. The read function iterates over all available file-opening functions within xradar.io to attempt to read radar files, with added support for georeferencing and verbose output. A key feature of this function is its ability to handle timeouts, ensuring that long-running file reads do not block execution indefinitely.

Key Features:

• Automatic Format Detection: The read function attempts to open the provided radar file using all available open_ functions in the xradar.io module, automatically selecting the correct file format.
• Timeout Support: The function now includes an optional timeout parameter. If specified, the file read operation is wrapped in a timeout handler, raising a TimeoutException if the read operation exceeds the allowed time.
• Georeferencing: The function can automatically georeference the radar data by adding x, y, z coordinates if the georeference parameter is set to True.
• Verbose Mode: A verbose option is available to print detailed processing information, aiding in debugging and providing insights into which file-opening method was successful.
• Custom Exception Handling: The function raises a ValueError if the file cannot be opened by any supported format, providing clear feedback in cases of unsupported file types or corrupted files.

Usage:

• Basic Usage:

  dtree = xd.io.read(file_path)

• With Timeout and Verbose Output:

  dtree =  xd.io.read(file_path, verbose=True, timeout=10)

Other tests

import os
from open_radar_data import DATASETS

# Dictionary to track which prefixes have been processed
prefix_processed = {}

# Iterate through the files in the registry
for file in DATASETS.registry_files:
#     print(file)

    # Skip files with .gz extension as they are not supported by xradar
    if file.endswith('.gz'):
        continue

    # Extract the prefix of the file (up to the first underscore, dot, or space)
    prefix = file.split('_')[0].split('.')[0].split()[0]

    # Check if a file from this prefix group has already been processed
    if prefix in prefix_processed:
        continue  # Skip the file if a file from this group has already been processed

    try:
        # Attempt to fetch the file path
        filepath = DATASETS.fetch(file)
        print(f"Reading: {os.path.basename(file)}")
        # Attempt to read the radar data
        radar = xd.io.read(filepath, timeout=10)
        
        # If no exception is raised, print success
        print('+++++++++++')
        print('| SUCCESS |')
        print('+++++++++++\n')

        # Mark this prefix as processed
        prefix_processed[prefix] = True

        # Delete the radar object after processing
        del radar

    except Exception as e:
        # Print the error and continue with the next file
        print(f"Error processing file {os.path.basename(file)}: \n{e}")
        print('+++++++++++')
        print('| FAILED |')
        print('+++++++++++\n')
        continue

Reading: 110635.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: 2013051000000600dBZ.vol
+++++++++++
| SUCCESS |
+++++++++++

Reading: 20220628072500_savevol_COSMO_LOOKUP_TEMP.nc
Error processing file 20220628072500_savevol_COSMO_LOOKUP_TEMP.nc: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: 71_20181220_060628.pvol.h5
+++++++++++
| SUCCESS |
+++++++++++

Reading: DES_VOL_RAW_20240522_1600.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: DWD-Vol-2_99999_20180601054047_00.h5
+++++++++++
| SUCCESS |
+++++++++++

Reading: KATX20130717_195021_V06
+++++++++++
| SUCCESS |
+++++++++++

Reading: KLBB20160601_150025_V06
+++++++++++
| SUCCESS |
+++++++++++

Reading: MLA2119412050U.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: MLL2217907250U.003.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: RAW_NA_000_125_20080411181219
+++++++++++
| SUCCESS |
+++++++++++

Reading: SUR210819000227.RAWKPJV
+++++++++++
| SUCCESS |
+++++++++++

Reading: T_PAGZ35_C_ENMI_20170421090837.hdf
+++++++++++
| SUCCESS |
+++++++++++

Reading: User1_184_20240601_013257.hpl
+++++++++++
| SUCCESS |
+++++++++++

Reading: Z__C_RJTD_20230801200000_RDR_JMAGPV_RS47937_Gar0p250km0p35deg_PRkdp_N11_ANAL_cfrad.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: cfrad.20080604_002217_000_SPOL_v36_SUR.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: convsf.19990811.221202.cdf
Error processing file convsf.19990811.221202.cdf: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: cor-main131125105503.RAW2049
+++++++++++
| SUCCESS |
+++++++++++

Reading: corcsapr2cmacppiM1.c1.20181111.030003.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: csapr.nc
Error processing file csapr.nc: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: ddop.910815.212303.cdf
Error processing file ddop.910815.212303.cdf: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: ddop.910815.213931.cdf
Error processing file ddop.910815.213931.cdf: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: ddop.910815.214454.cdf
Error processing file ddop.910815.214454.cdf: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: ddop.910815.215545.cdf
Error processing file ddop.910815.215545.cdf: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: example_plot_ppi_single_sweep.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: gucxprecipradarcmacppiS2.c1.20220314.021559.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: houkasacrcfrM1.a1.20210922.150006.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: sample_rainbow_5_59.vol
+++++++++++
| SUCCESS |
+++++++++++

Reading: sgpkasacrcrrasterC1.a1.20130419.012153.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: sgpkazrgeC1.a1.20190529.000002.cdf
Error processing file sgpkazrgeC1.a1.20190529.000002.cdf: 
File could not be opened by any supported format in xradar.io.
+++++++++++
| FAILED |
+++++++++++

Reading: sgpxsaprcfrvptI4.a1.20200205.100827.nc
+++++++++++
| SUCCESS |
+++++++++++

Reading: swx_20120520_0641.nc
+++++++++++
| SUCCESS |
+++++++++++

[codecov]

Codecov Report

Attention: Patch coverage is 93.33333% with 5 lines in your changes missing coverage. Please review.

Project coverage is 92.08%. Comparing base (7ff84a7) to head (4c4616c).

Files with missing lines	Patch %	Lines
xradar/io/auto_read.py	93.24%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #205      +/-   ##
==========================================
+ Coverage   91.95%   92.08%   +0.13%     
==========================================
  Files          23       24       +1     
  Lines        4562     4637      +75     
==========================================
+ Hits         4195     4270      +75     
  Misses        367      367              

Flag	Coverage Δ
notebooktests	79.03% <49.33%> (+0.73%)	⬆️
unittests	90.25% <92.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB