Efficient and Scalable Physics-Informed Deep Learning and Scientific Machine Learning on top of Tensorflow for multi-worker distributed computing

The examples in the doc use the latest code of master branch but the library on Pypi is still the version in May. Can you build the lib and update the version on Pypi?

It is possible to define a batch size and this will be applied to the calculation of the residual loss function, in splitting the collocation points in batches during the training.

Dear Levi,

I tried to make a Pull Request on this repository using PyCharm, and I received the following message:

Although you appear to have the correct authorization credentials, the tensordiffeq organization has enabled OAuth App access restrictions, meaning that data access to third-parties is limited. For more information on these restrictions, including how to whitelist this app, visit https://help.github.com/articles/restricting-access-to-your-organization-s-data/

I would kindly ask you to authorize PyCharm to access your organization data to use the GUI to make future pull requests.

Best Regards

Fix bug on the method def get_sizes(layer_sizes) of utils.py. The method was only allowing neural nets with an identical number of nodes in each hidden layer. Which was making the L- BFGS optimization to crash.

Sometimes, it's useful to save the model for later use. I couldn't find a .save method and pickle (and dill) didn't let me dump the object for later re-use. (example of error with pickle: Can't pickle local object 'make_gradient_clipnorm_fn..').

Is it currently possible to save the model? Thanks!

Add model.save and model.load_model to CollocationSolverND class ref #3

Will be released in the next stable.

currently this can be done by using the Keras integration via running model.u_model.save("path/to/file"). This change will allow a direct save by calling model.save() on the CollocationSolverND class. Same with load_model().

The docs will be updated to reflect this change.

Hello @levimcclenny and thanks for recommending this library!

I have modified the 1D burger example to be in 2D, but I did not get good comparison results. Any suggestions?

import math
import scipy.io
import tensordiffeq as tdq
from tensordiffeq.boundaries import *
from tensordiffeq.models import CollocationSolverND

Domain = DomainND(["x", "y", "t"], time_var='t')

Domain.add("x", [-1.0, 1.0], 256)
Domain.add("y", [-1.0, 1.0], 256)
Domain.add("t", [0.0, 1.0], 100)

N_f = 10000
Domain.generate_collocation_points(N_f)


def func_ic(x,y):
    p =2
    q =1
    return np.sin (p * math.pi * x) * np.sin(q * math.pi * y)
    

init = IC(Domain, [func_ic], var=[['x','y']])
upper_x = dirichletBC(Domain, val=0.0, var='x', target="upper")
lower_x = dirichletBC(Domain, val=0.0, var='x', target="lower")
upper_y = dirichletBC(Domain, val=0.0, var='y', target="upper")
lower_y = dirichletBC(Domain, val=0.0, var='y', target="lower")

BCs = [init, upper_x, lower_x, upper_y, lower_y]


def f_model(u_model, x, y, t):
    u = u_model(tf.concat([x, y, t], 1))
    u_x = tf.gradients(u, x)
    u_xx = tf.gradients(u_x, x)
    u_y = tf.gradients(u, y)
    u_yy = tf.gradients(u_y, y)
    u_t = tf.gradients(u, t)
    f_u = u_t + u * (u_x + u_y) - (0.01 / tf.constant(math.pi)) * (u_xx+u_yy)
    return f_u


layer_sizes = [3, 20, 20, 20, 20, 20, 20, 20, 20, 1]

model = CollocationSolverND()
model.compile(layer_sizes, f_model, Domain, BCs)

# to reproduce results from Raissi and the SA-PINNs paper, train for 10k newton and 10k adam
model.fit(tf_iter=10000, newton_iter=10000)

model.save("burger2D_Training_Model")
#model.load("burger2D_Training_Model")

#######################################################
#################### PLOTTING #########################
#######################################################

data = np.load('py-pde_2D_burger_data.npz')

Exact = data['u_output']
Exact_u = np.real(Exact)

x = Domain.domaindict[0]['xlinspace']
y = Domain.domaindict[1]['ylinspace']
t = Domain.domaindict[2]["tlinspace"]

X, Y, T = np.meshgrid(x, y, t)

X_star = np.hstack((X.flatten()[:, None], Y.flatten()[:, None], T.flatten()[:, None]))
u_star = Exact_u.T.flatten()[:, None]

u_pred, f_u_pred = model.predict(X_star)

error_u = tdq.helpers.find_L2_error(u_pred, u_star)
print('Error u: %e' % (error_u))

lb = np.array([-1.0, -1.0, 0.0])
ub = np.array([1.0, 1.0, 1])

tdq.plotting.plot_solution_domain2D(model, [x, y, t], ub=ub, lb=lb, Exact_u=Exact_u.T)

Thank you for the great contribution!

I'm trying to extend the 1D example problems to 2D, but I want to make sure my changes are in the correct place:

1. Dimension variables. I changed them like so:

Domain = DomainND(["x", "y", "t"], time_var='t')

Domain.add("x", [0.0, 5.0], 100) Domain.add("y", [0.0, 5.0], 100) Domain.add("t", [0.0, 5.0], 100)

2. My IC is zero, but for the BCs I'm not sure how to define the left and right borders, please let me know if my implementation is correct:

def func_ic(x,y):
    return 0

init = IC(Domain, [func_ic], var=[['x','y']])
upper_x = dirichletBC(Domain, val=0.0, var='x', target="upper")
lower_x = dirichletBC(Domain, val=0.0, var='x', target="lower")
upper_y = dirichletBC(Domain, val=0.0, var='y', target="upper")
lower_y = dirichletBC(Domain, val=0.0, var='y', target="lower")
        
BCs = [init, upper_x, lower_x, upper_y, lower_y]

All of my BCs and ICs are zero. And my equation has a (forcing) time-dependent source term as such:

def f_model(u_model, x, y, t):
    c = tf.constant(1, dtype = tf.float32)
    Amp = tf.constant(2, dtype = tf.float32)
    freq = tf.constant(1, dtype = tf.float32)
    sigma = tf.constant(0.2, dtype = tf.float32)

    source_x = tf.constant(0.5, dtype = tf.float32)
    source_y = tf.constant(2.5, dtype = tf.float32)

    GP = Amp * tf.exp(-0.5*( ((x-source_x)/sigma)**2 + ((y-source_y)/sigma)**2 ))
    
    S = GP * tf.sin( 2 * tf.constant(math.pi)  * freq * t )
    u = u_model(tf.concat([x,y,t], 1))
    u_x = tf.gradients(u,x)
    u_xx = tf.gradients(u_x, x)
    u_y = tf.gradients(u,y)
    u_yy = tf.gradients(u_y, y)
    u_t = tf.gradients(u,t)
    u_tt = tf.gradients(u_t,t)


    f_u = u_xx + u_yy - (1/c**2) * u_tt + S
    
    return f_u

Please advise.

Looking forward to your reply!

Dear @levimcclenny,

Have you considered in adapt TensorDiffEq to be deterministic? In the way the code is implemented, we can find two sources of randomness:

• The function Domain.generate_collocation_points has a random number generation
• The TensorFlow training procedure (weights initialization and possibility of the use o random batches)

Both sources of randomness can be solved with not much effort. We can define a random state for the first one that can be passed to the function Domain.generate_collocation_points. For the second, we can use the implementation provided on Framework Determinism. I have used the procedures suggested by this code, and the results of TensorFlow are always reproducible (CPU or GPU, serial or distributed).

If you want, I can implement these two features.

Best Regards