High uncertainties by seeding spread

[camines]

Dear all,

Im having an issue that when generating the analysis can bring me lost of errors.

Im seeding the particle with these seeding scheme:

#Port Spain

for i in range(num_steps+1):
    o.seed_elements(lon=-61.6, lat=10.5, radius=500, number=18,
                    time=start_time + i*time_step, origin_marker=0)

and this is the run
o.run(steps=(24*365*11), time_step=timedelta(hours=1), time_step_output=timedelta(days=7), outfile='Double_Tons.nc',export_variables=['age_seconds', 'origin_marker','lon','lat'])

This as my second run, was to compare by adding double of particles (from 9 to 18), and because of memory problems I changed the time_step_output= form 1 to 7 days.

Now the results are making me crazy, because after lots and lots of runs, is the first time I see this seeding behaviour.

Abnormal behaviour

Common behaviour until now

Do you have an idea of what could be happening, and such a drastic change?

[knutfrode]

Hi,

The apparent seeding locations here (green dots) are a bit misleading. This is not the actual seeding locations, but rather the locations of elements at their first output time step.
If all particles were seeded at the same starting time, this locations would correspond to the seeding time.
But when seeding over time, as in this case, positions of particles that are released within an "output time interval" will not be recorded until the end of that time step, here 7 days, during which they may have moved a lot.

So although this looks wrong, the calculations are correct, and elements are released at the correct time and location.
Presently, the actual seeding locations can be re-calculated from metadata in the output netCDF-files (as well as from your invoking script), but in the near future, the actual seeding locations shall be stored explicitly in the netCDF file, and not only their first recorded position.

Note that if you specify seeding time as [time_start, time_end], elements will be released evenly throughout this interval, without the need for the for-loop.

[camines]

Hello dear Knut!

Thanks as always. for the fast and nice reply. Now that makes a lot of sense, I'm looking at the seeding, seen from the output time record!

Can you explain me how to store the actual seeding location explicitly in the netCDF?

I'm really interested on your last comment of the loop. The thing is that I calculated the particles that should be input per month, and that's my main input information.

So, what you would recommend to have an even seeding distribution, would be something like:

#Port Spain
    
o.seed_elements(lon=-61.6, lat=10.5, radius=500, number=18*12*11, time= [ datetime(2010,01,01), datetime(2020,12,31)], origin_marker=0)

The 18*12*11 means: 18 particles * 12 months * 11 years

I think without the loops, it can save me some of computational cost.

Thanks as always!

[knutfrode]

The actual seeding locations are not stored when seeding over a time interval as here, but you can get this from your script after seeding:

o.elements_scheduled.lon
o.elements_scheduled.lat

Then you could e.g. save this to a separate netCDF-file, if necessary.

Yes, I believe you can do seeding like that. The seeding is quite fast, so you will probably not save a lot of time, but script becomes simpler.
And as a difference, elements are here spread evenly in time, and not in "bursts" e.g. every day. So it depends also what you want.

[camines]

Dear Knutt,

I really like the approach of equally distributed particles, because its a better representation of the reality. But its taking me at least 4 times more. And at the end it just send me an error. I put the output so every 20 days, but it still breaks.

The periodic seeding example (30 days) complete the run, but even when the output was every 10 days, it kill the kernel when .plot()
(It reduced the output from 4 GiB to just 600 MiB. )

I have this error with the evenly distributed:

----------------------------------------------------------------
TypeError                      Traceback (most recent call last)
<ipython-input-24-b699c819b8bc> in <module>
      3 
      4 #Here the steps are 11 years in one hour per step of 3600 sec
----> 5 o.run(steps=(24*365*11), time_step=timedelta(hours=1), time_step_output=timedelta(days=20), outfile='Double_new_seeding.nc',export_variables=['age_seconds', 'origin_marker','lon','lat'])
      6 #o.run(steps=(24*365*11), time_step=3600)
      7 #o.run(outfile='PlastDrift3.nc', time_step= 3600, time_step_output=3600*6, duration=timedelta(weeks=480))

/data/share/lehre/unix/oz-m-theo2-uebungen/PBWS2020/G6_wbc/opendrift/opendrift/models/basemodel.py in run(self, time_step, steps, time_step_output, duration, end_time, outfile, export_variables, export_buffer_length, stop_on_error)
   2985             if hasattr(self, 'environment_profiles'):
   2986                 del self.environment_profiles
-> 2987             self.io_import_file(outfile)
   2988 
   2989         self.timer_end('cleaning up')

/data/share/lehre/unix/oz-m-theo2-uebungen/PBWS2020/G6_wbc/opendrift/opendrift/export/io_netcdf.py in import_file(self, filename, times, elements, load_history)
    329                     np.arange(len(index_of_last)), index_of_last]
    330         kwargs['ID'] = np.arange(len(elements)) + 1
--> 331         self.elements = self.ElementType(**kwargs)
    332         self.elements_deactivated = self.ElementType()
    333     else:

/data/share/lehre/unix/oz-m-theo2-uebungen/PBWS2020/G6_wbc/opendrift/opendrift/elements/elements.py in __init__(self, **kwargs)
    131                 array_lengths[i] = 1  # scalar is given
    132         if len(set(array_lengths) - {1}) > 1:
--> 133             raise TypeError(
    134                 'Input arrays must have same length. Lengths given: '
    135                 + str(array_lengths))

TypeError: Input arrays must have same length. Lengths given: [96208, 96208, 96208, 96208, 96208, 96568]

And I guess because that error, and as it doesn't complete the run, the output netCDF have less particles

Periodic (30 day run):

#Port Spain

for i in range(num_steps+1):
    o.seed_elements(lon=-61.6, lat=10.5, radius=500, number=18,
                    time=start_time + i*time_step, origin_marker=0)
o.run(steps=(24*365*11), time_step=timedelta(hours=1), time_step_output=timedelta(days=10), outfile='Double_normal.nc',export_variables=['age_seconds', 'origin_marker','lon','lat'])

===========================
Model:	PlastDrift     (OpenDrift version 1.8.3)
	1238 active PlastElement particles  (96118 deactivated, 0 scheduled)
Time:
	Start: 2010-01-01 00:00:00
	Present: 2020-12-24 00:00:00
	Calculation steps: 96360 * 1:00:00 - total time: 4010 days, 0:00:00
	Output steps: 402 * 10 days, 0:00:00

The evenly distributed:

#Port Spain

o.seed_elements(lon=-61.6, lat=10.5, radius=500, number=18*12*11, time= [ datetime(2010,1,1), datetime(2020,12,31)],
                origin_marker=0)
o.run(steps=(24*365*11), time_step=timedelta(hours=1), time_step_output=timedelta(days=20), outfile='Double_new_seeding.nc',export_variables=['age_seconds', 'origin_marker','lon','lat'])

===========================
Model:	PlastDrift     (OpenDrift version 1.8.3)
	1619 active PlastElement particles  (94949 deactivated, 56 scheduled)
Time:
	Start: 2010-01-01 00:00:00
	Present: 2020-12-14 00:00:00
	Calculation steps: 96360 * 1:00:00 - total time: 4000 days, 0:00:00
	Output steps: 201 * 20 days, 0:00:00

Do you have any glance why it give me an error. I guess the more important is the analisis and comparison of the runs, but without the same partiles, it complicates a bit.

Thanks as always for your help

[paulabirocchi]

Hi @camines
I have seen that you applied an approach for equally distributed particles. Could you give me more details about how you did that?
In my case, I have an irregular grid for velocity data, but I want to calculate the residence time in the estuary, so it is important to release particles equally distributed in space.

[knutfrode]

I cannot see why evenly seeding should take more time than seeding in groups.

Anyway, there was in fact a bug leading to your crash: this occurs in the rare case that there are unseeded elements (here since run is slightly shorter than your seeding interval) a the end of the simulation, and time_step_interval is not fulfilled at the last time step.

This is now fixed, so that it shall hopefully work after an update. (Anyway it would be better to run simulation until all elements are seeded)

[camines]

Im still having an error before the model ends up. Usually was on the equitative seeding scheme on time, which takes about to 4 hours until it breaks, but I want to ask why I'm getting different number of particles at the end.

How can I assure that all the particles are seeded? Or what could be the issue here?

[knutfrode]

Did you update to the latest version (git pull), but still experience this error?

In any case, to avoid unseeded elements at the end, you should make sure that the seeding duration is shorter than the simulation duration.
E.g. I see that you are simulating over 365*24*number_of_years hours, but remember that some years have 366 days, so that the simulation may stop before 31st Dec, thus with unseeded elements.