TypeError: Can't broadcast (smallbaselineApp.py and gmtsar)

[marcontk]

Description of the problem

I am using gmtsar and when running smallbaselineApp.py it stops with some file coherence. The output is shown below

Full script that generated the error

smallbaselineApp.py 

Full error message

smallbaselineApp.py LebuSenDT156.txt                                   

___________________________________________________________

  /##      /## /##             /##     /#######           
 | ###    /###|__/            | ##    | ##__  ##          
 | ####  /#### /## /#######  /######  | ##  \ ## /##   /##
 | ## ##/## ##| ##| ##__  ##|_  ##_/  | #######/| ##  | ##
 | ##  ###| ##| ##| ##  \ ##  | ##    | ##____/ | ##  | ##
 | ##\  # | ##| ##| ##  | ##  | ## /##| ##      | ##  | ##
 | ## \/  | ##| ##| ##  | ##  |  ####/| ##      |  #######
 |__/     |__/|__/|__/  |__/   \___/  |__/       \____  ##
                                                 /##  | ##
                                                |  ######/
   Miami InSAR Time-series software in Python    \______/ 
          MintPy 1.4.1, 2022-08-15
___________________________________________________________

--RUN-at-2022-11-28 11:29:27.403542--
Current directory: /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy
Run routine processing with smallbaselineApp.py on steps: ['load_data', 'modify_network', 'reference_point', 'quick_overview', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_SET', 'correct_troposphere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'google_earth', 'hdfeos5']
Remaining steps: ['modify_network', 'reference_point', 'quick_overview', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_SET', 'correct_troposphere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'google_earth', 'hdfeos5']
--------------------------------------------------
Project name: LebuSenDT156
Go to work directory: /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy
read custom template file: /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/LebuSenDT156.txt
update default template based on input custom template
No new option value found, skip updating /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/smallbaselineApp.cfg
copy LebuSenDT156.txt     to inputs   directory for backup.
copy smallbaselineApp.cfg to inputs   directory for backup.
copy LebuSenDT156.txt     to pic      directory for backup.
copy smallbaselineApp.cfg to pic      directory for backup.
read default template file: /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/smallbaselineApp.cfg


******************** step - load_data ********************

load_data.py --template /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/smallbaselineApp.cfg /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/LebuSenDT156.txt --project LebuSenDT156
processor : gmtsar
SAR platform/sensor : Sen
--------------------------------------------------
prepare metadata files for gmtsar products
prep_gmtsar.py /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/LebuSenDT156.txt
writing  /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/inputs/data.rsc
write file: /Users/marcont/Insar/Processing/descending/LEBU_CONS/topo/dem_ll.grd.rsc
[==================================================] 20221022_20221103    0s /     0s
Done.
--------------------------------------------------
updateMode : True
compression: None
multilook x/ystep: 1/1
multilook method : nearest
--------------------------------------------------
searching geometry files info
input data files:
height          : /Users/marcont/Insar/Processing/descending/LEBU_CONS/topo/dem_ll.grd
--------------------------------------------------
create HDF5 file /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/inputs/geometryGeo.h5 with w mode
create dataset /height             of <class 'numpy.float32'>   in size of (3910, 2690) with compression = lzf
geocoded input, use contant value from metadata INCIDENCE_ANGLE
create dataset /incidenceAngle     of <class 'numpy.float32'>   in size of (3910, 2690) with compression = lzf
prepare slantRangeDistance ...
geocoded input, use contant value from metadata SLANT_RANGE_DISTANCE
create dataset /slantRangeDistance of <class 'numpy.float32'>   in size of (3910, 2690) with compression = lzf
Finished writing to /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/inputs/geometryGeo.h5
--------------------------------------------------
searching interferogram pairs info
input data files:
unwrapPhase     : /Users/marcont/Insar/Processing/descending/LEBU_CONS/F2/intf_all/20*/unwrap_ll.grd
coherence       : /Users/marcont/Insar/Processing/descending/LEBU_CONS/F2/intf_all/20*/corr_ll.grd
number of unwrapPhase     : 14
number of coherence       : 14
--------------------------------------------------
create HDF5 file /Users/marcont/Insar/Processing/descending/LEBU_CONS/mintpy/inputs/ifgramStack.h5 with w mode
create dataset /unwrapPhase of <class 'numpy.float32'>   in size of (14, 3910, 2690) with compression = None
[==================================================] 20221022_20221103   15s /     1s
create dataset /coherence   of <class 'numpy.float32'>   in size of (14, 3910, 2690) with compression = None
[===>                     7%                       ] 20220904_20220916    0s /     0sTraceback (most recent call last):
  File "/Users/marcont/opt/miniconda3/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 1291, in main
    app.run(steps=inps.runSteps)
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 1047, in run
    self.run_load_data(sname)
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 356, in run_load_data
    mintpy.load_data.main(iargs)
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/mintpy/load_data.py", line 966, in main
    stack_obj.write2hdf5(
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/mintpy/objects/stackDict.py", line 216, in write2hdf5
    ds[i, :, :] = data
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 980, in __setitem__
    mspace = h5s.create_simple(selection.expand_shape(mshape))
  File "/Users/marcont/opt/miniconda3/lib/python3.8/site-packages/h5py/_hl/selections.py", line 264, in expand_shape
    raise TypeError("Can't broadcast %s -> %s" % (source_shape, self.array_shape))  # array shape
TypeError: Can't broadcast (4070, 2690) -> (3910, 2690)

System information

• Operating system: macOS (12.6.1 , Monterey)
• Python environment: conda
• MintPy version: MintPy version 1.4.1, date 2022-08-15
• Your custom / default template file (if the bug is related to a specific dataset):
  LebuSenDT156.txt

[welcome[bot]]

👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute!
Make sure you read our contributing guidelines.

[yunjunz]

The error message indicates that for 20220904_20220916, the coherence size is (4070, 2690), which is different from the rest. Please confirm and fix it by re-generating this pair or removing it.

[marcontk]

I am working with a subswath ( without cuts). If indeed, the coherence file and unwrapad have different size and this makes them gmtsar, I discover the reason why they are of different size. In a second attempt I cut inside the subswath and Mintpy gives me the same errors. Inside each interferogram folder will it be possible to cut the coherence according to the dimensions of the unwrapped ?

[yunjunz]

Same size is required by MintPy for all pairs: unwrapped phase and coherence. You might want to ask in the gmtsar forum on how to do that.

[marcontk]

well thank you very much, greetings