OPT estimated vs simulated

[IsaDio]

Hi,

I started running OPT for a problem where we want to optimise managed aquifer recharge rates (minimise MAR rates) to achieve a given flow in a creek at the end of MAR (greater than constraints).

It looks like it is working (yay!) but OPT seems to have difficulties finding a solution. The estimated constraints/objective usually look great but the actuals (simulated) are often quite different and lower (constraints not satisfied). Is this due to the non linearity of the solution, and OPT is struggling to find the answer? Or could it be something else? Is there something that can be done to help with this (initial conditions of parameters, OPT settings..)?

Thank you!

[jtwhite79]

Sure sounds like nonlinearity - thats why opt writes the estimated and simulated results. opt can only tolerate the mildest nonlinearity...are you iterating with opt at all? Also what size perturbations on those MAR rates are you using and are your model solver tolerances pretty tight? Typically the decision variable perturbations would be larger than the parameter perturbations that are used to fill a jco...

[IsaDio]

Thanks! Yes that is what I understand.
Yes I am iterating with opt. It does help but it doesn't guarantee it will find an optimal solution for each scenarios I am running.
The water balance looks good with little mass balance error, but good thinking. I might try with a smaller tolerance.
I have gathered that the decision variable perturbations are quite important. I have tested different values and also tried to constrain the bounds of the decision variables. It looks like it is helping.
I am setting up a run with MOU also, as it should solve that issue hopefully?

[jtwhite79]

Seems like you are across the issues! MOU is a different beast - its a global solver so in theory it cope with wicked nonlinearity, but that comes with a much larger computational cost. Keen to hear how this goes for you...

[IsaDio]

I have improved things for sure but it still can be a bit unpredictable. Do you have a rule of thumb or some default values you like to use for the decision variable perturbations to calculate the derivatives based on the range? Absolute or relative does it make a difference? I thought a small increment and relative would help with the non-linearity but the opposite happened. Thanks!

[jtwhite79]

I usually use a 10-15% perturbation when filling the response matrix for opt - this is because you want to make sure you get a good strong representative response at the constraints (that is in excess of any solver tolerance noise). We took a concept from GWM where you can have the perturbation decrease across iterations (there is a ++ arg for this), with idea being that as you near the optimal solution, you might want to use progressively smaller (ie more local) perturbations...I also usually start with strongly feasible solution (so in your case with high MAR rates so that the stream constraints are satisfied)

[IsaDio]

Awesome. This is around the range I have been using, maybe a bit smaller. I do start with a feasible solution, just not optimised.
An decrease in perturbation is exactly what I was thinking could help.
Thank you again for your help, greatly appreciated :)