Difficulty in Predictions with Extended Time Domain

[YoungBaeChoi]

Dear Dr. Lu Lu,

Firstly, I would like to express my gratitude for your contributions and the development of the library.

I have been working on solving a system of ODEs using DeepXDE, and I encountered a challenge. Specifically, I noticed that when the time domain is extended from

https://deepxde.readthedocs.io/en/latest/demos/pinn_forward/ode.system.html

geom = dde.geometry.TimeDomain(0, 10) 
-> geom = dde.geometry.TimeDomain(0, 30)

the prediction accuracy diminishes. This issue persists even after adjusting parameters such as domain points, activation functions, and the number of layers.

Initially, I considered this problem to be related to the scaling issues. By gradually reducing the dimensions for testing, I recognized that the problem persists. This issue appears with longer time domains and higher output values.

Despite testing different approaches, including using the ResNet architecture, changing domain points, and applying hard constraints, I have not been able to achieve accurate results. For example, in a simple ODE problem as described below:

def ode_system(x, y):
    y1, = y[:, 0:1],
    dy1_x = dde.grad.jacobian(y, x, i=0)
    return [dy1_x - (0.4 * (1 - tf.exp(-y1)))]

# True solution for comparison
def true_solution(t):
    return np.log(0.64872 * np.exp(0.4 * t) + 1)

def boundary(_, on_initial):
    return on_initial


geom = dde.geometry.TimeDomain(0, 30)
ic1 = dde.icbc.IC(geom, lambda x: 0.5, boundary, component=0)

Same as np.log(0.64872 * np.exp(-0.4 * t) + 1) not correct

(correct prediction)

When the time domain is extended, the PINN solution aligns with np.log(0.64872 * np.exp(-0.4 * t) + 1) (with the negative sign changed). While adjusting model architecture, layers, and other parameters can somewhat extend the accurate prediction time or output value (approximately up to 5), there is still a limit.

I have tried various methods, including ResNet, changing activation functions, and learning rates. However, I have not been able to find a fundamental solution to this problem.

Could you please provide some insights or suggestions on why this occurs and how to address it?

Thank you very much for your time and assistance.

Best regards,