Respecting causality is all you need for training physics - informed neural networks

[riccardotomada]

Hi everyone. It's great to see the Discussion section of this repository! I'm a master student at Politecnico di Milano and DeepXDE is very helpful for my studies.

I have just read the paper I reported in the title and I found it really interesting. I was wondering if it is possible to implement their suggestions in DeepXDE. In particular I'm referring to the re-formulation of the training loss to force the optimization algorithm to respect the causality of the problem, since the improvement in the performances is amazing.

In particular, I have understood how the weighting mechanism works: it just reduces the "importance" of a collocation point residual if the residuals measured at points which are located previously in time are still high. The problem for me is how to implement it in practice.

The proposed modified MLP should be easy to implement instead by simply define a custom net with apply_output_transform.

If anyone has already implemented this re-formulation of the training loss in DeepXDE or has any idea on how doing it, please let me know. I will need to study vortex-shedding of a cylinder wake at low Re and the results I previously obtained were not promising.

PS: if this proposed method works for temporal causality, the same should apply for the spatial one? I was thinking about a sort of analogy with upwind schemes for solving, for example, advection-diffusion problems.

[sheikhu28]

Hi

Did you ever figure this out? I am having trouble with figuring out how to apply different weights to losses on different collocation points based on the different temporal losses at the present iteration. The algorithm itself seems simple but it seems like its implementation in the context of DeepXDE might be tricky.

Thank you

[lululxvi]

• In DeepXDE, point-wise weight is tricky, although it is doable, see https://doi.org/10.1137/21M1397908
• Also, point-wise weight is equivalent to nonuniform point sampling, see https://doi.org/10.1016/j.cma.2022.115671

[sheikhu28]

Dear Lu Lu

Thank you for your response. The first reference is certainly a good way to approach this problem. I guess the thing I am struggling to achieve is to actively readjust and apply the weights applied to the different training points at each iteration (as in the respecting causality paper). I have gone through the internals of the code but I can't seem to find where exactly the losses are being computed (both with respect to data and the residual). If you could point me to that it would be really helpful to me.

Thank you

[lululxvi]

Which code do you refer to?

[sheikhu28]

I am referring to the DeepXDE code.

[lululxvi]

See model.py