Deep Ritz Method

[pescap]

Dear all,
I was trying to add a simple example using DRM, in order to compare the results with PINNs.

For example, refer to Eqs. (3) and (4) in Sobolev Acceleration and Statistical Optimality for
Learning Elliptic Equations via Gradient Descent

So far, I couldn't obtain convergence for the DRM.
My code, inspired by: this 1D example.

Changes: The loss in DRMpde, and the usage of "MAE" loss.

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import matplotlib.pyplot as plt

import deepxde as dde
from deepxde.backend import tf


def PINNpde(x, y):
    dy_xx = dde.grad.hessian(y, x)
    f = 8 * tf.sin(8 * x)
    for i in range(1, 5):
        f += i * tf.sin(i * x)

    return -dy_xx - f


def DRMpde(x, y):
    dy_xx = dde.grad.hessian(y, x)
    f = 8 * tf.sin(8 * x)
    for i in range(1, 5):
        f += i * tf.sin(i * x)

    dy_x = dde.grad.jacobian(y, x)

    return 1 / 2.0 * tf.abs(dy_x) ** 2 - f * y


def func(x):
    sol = x + 1 / 8 * np.sin(8 * x)
    for i in range(1, 5):
        sol += 1 / i * np.sin(i * x)
    return sol


geom = dde.geometry.Interval(0, np.pi)

# PINNdata = dde.data.PDE(geom, PINNpde, [], 15, 0, "uniform", solution=func, num_test=100)
DRMdata = dde.data.PDE(geom, DRMpde, [], 15, 0, "uniform", solution=func, num_test=100)

layer_size = [1] + [20] * 3 + [1]
activation = "tanh"
initializer = "Glorot uniform"
net = dde.maps.FNN(layer_size, activation, initializer)


def output_transform(x, y):
    return x + tf.math.tanh(x) * tf.math.tanh(np.pi - x) * y


net.apply_output_transform(output_transform)

# PINNmodel = dde.Model(PINNdata, net)
DRMmodel = dde.Model(DRMdata, net)

# PINNmodel.compile("adam", lr=0.001, metrics=["l2 relative error"], loss = "MSE")
DRMmodel.compile("adam", lr=0.001, metrics=["l2 relative error"], loss="MAE")

# losshistory, train_state = PINNmodel.train(epochs=5000)
losshistory, train_state = DRMmodel.train(epochs=5000)

[lululxvi]

Why

1 / 2.0 * tf.abs(dy_x) ** 2 - f * y

[pescap]

According to: The Deep Ritz method: A deep learning-based
numerical algorithm for solving variational problems we seek to minimize (see Eqs. 7 and 8):

\int_D (1/2 * |\nabla u |^2 - f u)

over the set of admissible functions.

I also tried to reproduce the 2D LShape case in Section 3.1 (see https://github.com/xdfeng7370/Deep-Ritz-Method/blob/61aefb642cd0431b4f61358f13d42d2abe5f59ab/ex1/deep_ritz1.py#L173), but I couldn't obtain convergence.

[lululxvi]

OK

[lululxvi]

The loss should not be "MAE". It should be just the mean, as in Eq. 8, there is no abs for g.

[pescap]

Current version (still not working). I'll edit it as soon as I understand what is wrong here.

"""Backend supported: tensorflow.compat.v1, tensorflow, pytorch, paddle"""
import deepxde as dde
import matplotlib.pyplot as plt
import numpy as np
from deepxde.backend import tf

def custom_loss(y_true, y_pred):
    return tf.reduce_mean(y_true - y_pred)

def pde(x, y):
    dy_x = dde.grad.jacobian(y, x)
    f = np.pi ** 2 * tf.sin(np.pi * x)
    return 1/2. * dy_x ** 2 - f * y

def boundary(x, on_boundary):
    return on_boundary

def func(x):
    return np.sin(np.pi * x)


geom = dde.geometry.Interval(-1, 1)
bc = dde.icbc.DirichletBC(geom, func, boundary)
data = dde.data.PDE(geom, pde, bc, 16, 2, solution=func, num_test=100)

layer_size = [1] + [50] * 3 + [1]
activation = "tanh"
initializer = "Glorot uniform"
net = dde.nn.FNN(layer_size, activation, initializer)

model = dde.Model(data, net)
model.compile("adam", lr=0.001, metrics=["l2 relative error"], loss = [custom_loss, "MSE"])

losshistory, train_state = model.train(epochs=5000)
dde.saveplot(losshistory, train_state, issave=False, isplot=True)

x = geom.uniform_points(1000, True)
y = model.predict(x, operator=pde)
plt.figure()
plt.plot(x, y)
plt.xlabel("x")
plt.ylabel("PDE residual")
plt.show()

[lululxvi]

You may need more points than 16?

[pescap]

It didn't change the results.. Sounds like something else is wrong in how I define the loss function.

[lululxvi]

emmm... Not sure what goes wrong here.

[bonneted]

Dear @pescap ,
Did you finally achieve convergence with the DRM?
I'm also trying to apply a variational formulation instead of the PDE (on solid mechanics) but it doesn't work: L-BFGS crashed and Adam doesn't converge.

[pescap]

Hi! Unfortunately I did not achieve convergence with the DRM... I'll let you know if I solve this issue.