Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8,438 changes: 8,438 additions & 0 deletions docs/source/tutorials/tutorial1/tutorial.html

Large diffs are not rendered by default.

8,362 changes: 8,362 additions & 0 deletions docs/source/tutorials/tutorial10/tutorial.html

Large diffs are not rendered by default.

8,938 changes: 8,938 additions & 0 deletions docs/source/tutorials/tutorial11/tutorial.html

Large diffs are not rendered by default.

8,088 changes: 8,088 additions & 0 deletions docs/source/tutorials/tutorial12/tutorial.html

Large diffs are not rendered by default.

8,447 changes: 8,447 additions & 0 deletions docs/source/tutorials/tutorial13/tutorial.html

Large diffs are not rendered by default.

8,311 changes: 8,311 additions & 0 deletions docs/source/tutorials/tutorial14/tutorial.html

Large diffs are not rendered by default.

8,689 changes: 8,689 additions & 0 deletions docs/source/tutorials/tutorial15/tutorial.html

Large diffs are not rendered by default.

8,440 changes: 8,440 additions & 0 deletions docs/source/tutorials/tutorial16/tutorial.html

Large diffs are not rendered by default.

8,762 changes: 8,762 additions & 0 deletions docs/source/tutorials/tutorial17/tutorial.html

Large diffs are not rendered by default.

8,214 changes: 8,214 additions & 0 deletions docs/source/tutorials/tutorial18/tutorial.html

Large diffs are not rendered by default.

8,532 changes: 8,532 additions & 0 deletions docs/source/tutorials/tutorial19/tutorial.html

Large diffs are not rendered by default.

8,761 changes: 8,761 additions & 0 deletions docs/source/tutorials/tutorial2/tutorial.html

Large diffs are not rendered by default.

8,329 changes: 8,329 additions & 0 deletions docs/source/tutorials/tutorial20/tutorial.html

Large diffs are not rendered by default.

8,317 changes: 8,317 additions & 0 deletions docs/source/tutorials/tutorial21/tutorial.html

Large diffs are not rendered by default.

15,401 changes: 15,401 additions & 0 deletions docs/source/tutorials/tutorial22/tutorial.html

Large diffs are not rendered by default.

10,078 changes: 10,078 additions & 0 deletions docs/source/tutorials/tutorial23/tutorial.html

Large diffs are not rendered by default.

8,413 changes: 8,413 additions & 0 deletions docs/source/tutorials/tutorial24/tutorial.html

Large diffs are not rendered by default.

8,516 changes: 8,516 additions & 0 deletions docs/source/tutorials/tutorial3/tutorial.html

Large diffs are not rendered by default.

9,189 changes: 9,189 additions & 0 deletions docs/source/tutorials/tutorial4/tutorial.html

Large diffs are not rendered by default.

8,345 changes: 8,345 additions & 0 deletions docs/source/tutorials/tutorial5/tutorial.html

Large diffs are not rendered by default.

8,587 changes: 8,587 additions & 0 deletions docs/source/tutorials/tutorial6/tutorial.html

Large diffs are not rendered by default.

8,345 changes: 8,345 additions & 0 deletions docs/source/tutorials/tutorial7/tutorial.html

Large diffs are not rendered by default.

8,455 changes: 8,455 additions & 0 deletions docs/source/tutorials/tutorial8/tutorial.html

Large diffs are not rendered by default.

8,271 changes: 8,271 additions & 0 deletions docs/source/tutorials/tutorial9/tutorial.html

Large diffs are not rendered by default.

Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
16 changes: 8 additions & 8 deletions tutorials/tutorial1/tutorial.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
# coding: utf-8

# # Tutorial: Introductory Tutorial: Physics Informed Neural Networks with PINA
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PINA-org/PINA-tutorials/blob/main/tutorial1/tutorial.ipynb)
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PINA-org/PINA-tutorials/blob/main/tutorials/tutorial1/tutorial.ipynb)
#
# > ##### ⚠️ ***Before starting:***
# > We assume you are already familiar with the concepts covered in the [Getting started with PINA](https://mathlab.github.io/PINA/_tutorial.html#getting-started-with-pina) tutorials. If not, we strongly recommend reviewing them before exploring this advanced topic.
# > We assume you are already familiar with the concepts covered in the [Getting started with PINA](https://PINA-org.github.io/PINA-Tutorials/#getting-started-with-pina) tutorials. If not, we strongly recommend reviewing them before exploring this advanced topic.
#

# In this tutorial, we will demonstrate a typical use case of **PINA** for Physics-Informed training. We will cover the basics of training a `PhysicsInformedSingleModelSolver` with PINA, if you want to go further into Physics-Informed solvers look at our dedicated [tutorials](https://mathlab.github.io/PINA/_tutorial.html#physics-informed-neural-networks) on the topic.
# In this tutorial, we will demonstrate a typical use case of **PINA** for Physics-Informed training. We will cover the basics of training a `PhysicsInformedSingleModelSolver` with PINA, if you want to go further into Physics-Informed solvers look at our dedicated [tutorials](https://PINA-org.github.io/PINA-Tutorials/#physics-informed-neural-networks) on the topic.
#
# Let's start by importing the useful modules:

Expand Down Expand Up @@ -150,13 +150,13 @@ def solution(self, pts):
# Once the problem is defined and the data is generated, we can move on to modeling. This process consists of three key steps:
#
# **Choosing a Model**
# - Select a neural network architecture. You can use the model we provide in the `pina.model` module (see [here](https://mathlab.github.io/PINA/_rst/_code.html#models) for a full list), or define a custom PyTorch module (more on this [here](https://pytorch.org/docs/stable/notes/modules.html)).
# - Select a neural network architecture. You can use the model we provide in the `pina.model` module (see [here](https://PINA-org.github.io/PINA/_rst/_code.html#models) for a full list), or define a custom PyTorch module (more on this [here](https://pytorch.org/docs/stable/notes/modules.html)).
#
# **Choosing a Physics-Informed Solver & Defining the Trainer**
# * Use a Physics-Informed solver from `pina.solver` module to solve the problem using the specified model. We have already implemented most State-Of-The-Arte solvers for you, [have a look](https://mathlab.github.io/PINA/_rst/_code.html#solvers) if interested. Today we will use the standard `PhysicsInformedSingleModelSolver` solver.
# * Use a Physics-Informed solver from `pina.solver` module to solve the problem using the specified model. We have already implemented most State-Of-The-Arte solvers for you, [have a look](https://PINA-org.github.io/PINA/_rst/_code.html#solvers) if interested. Today we will use the standard `PhysicsInformedSingleModelSolver` solver.
#
# **Training**
# * Train the model with the [`Trainer`](https://mathlab.github.io/PINA/_rst/trainer.html) class. The Trainer class provides powerful features to enhance model accuracy, optimize training time and memory, and simplify logging and visualization, thanks to PyTorch Lightning's excellent work, see [our dedicated tutorial](https://mathlab.github.io/PINA/tutorial11/tutorial.html) for further details. By default, training metrics (e.g., MSE error) are logged using a lightning logger (CSVLogger). If you prefer manual tracking, use `pina.callback.MetricTracker`.
# * Train the model with the [`Trainer`](https://PINA-org.github.io/PINA/_rst/trainer.html) class. The Trainer class provides powerful features to enhance model accuracy, optimize training time and memory, and simplify logging and visualization, thanks to PyTorch Lightning's excellent work, see [our dedicated tutorial](https://PINA-org.github.io/PINA-Tutorials/tutorial11/tutorial.html) for further details. By default, training metrics (e.g., MSE error) are logged using a lightning logger (CSVLogger). If you prefer manual tracking, use `pina.callback.MetricTracker`.
#
# Let's cover all steps one by one!
#
Expand All @@ -177,7 +177,7 @@ def solution(self, pts):
# Then we build the solver. The `PhysicsInformedSingleModelSolver` class needs to be initialized with a `model` and a specific `problem` to be solved. They also take extra arguments, as the optimizer, scheduler, loss type and weighting for the different conditions which are all set to their default values.
#
# >##### 💡***Bonus tip:***
# > All physics solvers in PINA can handle both forward and inverse problems without requiring any changes to the model or solver structure! See [our tutorial](https://mathlab.github.io/PINA/tutorial7/tutorial.html) of inverse problems for more infos.
# > All physics solvers in PINA can handle both forward and inverse problems without requiring any changes to the model or solver structure! See [our tutorial](https://PINA-org.github.io/PINA-Tutorials/tutorial7/tutorial.html) of inverse problems for more infos.

# In[ ]:

Expand Down Expand Up @@ -270,4 +270,4 @@ def solution(self, pts):
#
# 5. **... and many more!**: The possibilities are vast! Continue experimenting with advanced configurations, solvers, and other features in PINA.
#
# For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/).
# For more resources and tutorials, check out the [PINA Documentation](https://PINA-org.github.io/PINA/).
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
112 changes: 54 additions & 58 deletions tutorials/tutorial10/tutorial.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@
# coding: utf-8

# # Tutorial: Solving the Kuramoto–Sivashinsky Equation with Averaging Neural Operator
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PINA-org/PINA-tutorials/blob/main/tutorial10/tutorial.ipynb)
#
#
# In this tutorial, we will build a Neural Operator using the **`AveragingNeuralOperator`** model and the **`SupervisedSolver`**. By the end of this tutorial, you will be able to train a Neural Operator to learn the operator for time-dependent PDEs.
#
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PINA-org/PINA-tutorials/blob/main/tutorials/tutorial10/tutorial.ipynb)
#
#
# In this tutorial, we will build a Neural Operator using the **`AveragingNeuralOperator`** model and the **`SupervisedSingleModelSolver`**. By the end of this tutorial, you will be able to train a Neural Operator to learn the operator for time-dependent PDEs.
#
# Let's start by importing the necessary modules.

# In[1]:
# In[ ]:


## routine needed to run the notebook on Google Colab
Expand All @@ -24,12 +24,8 @@
get_ipython().system('pip install "pina-mathlab[tutorial]"')
# get the data
get_ipython().system('mkdir "data"')
get_ipython().system(
'wget "https://github.com/PINA-org/PINA-tutorials/raw/refs/heads/main/tutorial10/data/Data_KS.mat" -O "data/Data_KS.mat"'
)
get_ipython().system(
'wget "https://github.com/PINA-org/PINA-tutorials/raw/refs/heads/main/tutorial10/data/Data_KS2.mat" -O "data/Data_KS2.mat"'
)
get_ipython().system('wget "https://github.com/PINA-org/PINA-tutorials/raw/refs/heads/main/tutorials/tutorial10/data/Data_KS.mat" -O "data/Data_KS.mat"')
get_ipython().system('wget "https://github.com/PINA-org/PINA-tutorials/raw/refs/heads/main/tutorials/tutorial10/data/Data_KS2.mat" -O "data/Data_KS2.mat"')

import torch
import matplotlib.pyplot as plt
Expand All @@ -38,47 +34,47 @@
from scipy import io
from pina import Trainer, LabelTensor
from pina.model import AveragingNeuralOperator
from pina.solver import SupervisedSolver
from pina.solver import SupervisedSingleModelSolver
from pina.problem.zoo import SupervisedProblem

warnings.filterwarnings("ignore")


# ## Data Generation
#
#
# In this tutorial, we will focus on solving the **Kuramoto-Sivashinsky (KS)** equation, a fourth-order nonlinear PDE. The equation is given by:
#
#
# $$
# \frac{\partial u}{\partial t}(x,t) = -u(x,t)\frac{\partial u}{\partial x}(x,t) - \frac{\partial^{4}u}{\partial x^{4}}(x,t) - \frac{\partial^{2}u}{\partial x^{2}}(x,t).
# $$
#
#
# In this equation, $x \in \Omega = [0, 64]$ represents a spatial location, and $t \in \mathbb{T} = [0, 50]$ represents time. The function $u(x, t)$ is the value of the function at each point in space and time, with $u(x, t) \in \mathbb{R}$. We denote the solution space as $\mathbb{U}$, where $u \in \mathbb{U}$.
#
#
# We impose Dirichlet boundary conditions on the derivative of $u$ at the boundary of the domain $\partial \Omega$:
#
#
# $$
# \frac{\partial u}{\partial x}(x,t) = 0 \quad \forall (x,t) \in \partial \Omega \times \mathbb{T}.
# $$
#
#
# The initial conditions are sampled from a distribution over truncated Fourier series with random coefficients $\{A_k, \ell_k, \phi_k\}_k$, as follows:
#
#
# $$
# u(x,0) = \sum_{k=1}^N A_k \sin\left(2 \pi \frac{\ell_k x}{L} + \phi_k\right),
# $$
#
#
# where:
# - $A_k \in [-0.4, -0.3]$,
# - $\ell_k = 2$,
# - $\phi_k = 2\pi \quad \forall k=1,\dots,N$.
#
# We have already generated data for different initial conditions. The goal is to build a Neural Operator that, given $u(x,t)$, outputs $u(x,t+\delta)$, where $\delta$ is a fixed time step.
#
#
# We have already generated data for different initial conditions. The goal is to build a Neural Operator that, given $u(x,t)$, outputs $u(x,t+\delta)$, where $\delta$ is a fixed time step.
#
# We will cover the Neural Operator architecture later, but for now, let’s start by importing the data.
#
#
# **Note:**
# The numerical integration is obtained using a pseudospectral method for spatial derivative discretization and implicit Runge-Kutta 5 for temporal dynamics.

# In[2]:
# In[ ]:


# load data
Expand Down Expand Up @@ -106,10 +102,10 @@
# - `B` is the batch size (i.e., how many initial conditions we sample),
# - `N` is the number of points in the mesh (which is the product of the discretization in $x$ times the one in $t$),
# - `D` is the dimension of the problem (in this case, we have three variables: $[u, t, x]$).
#
#
# We are now going to plot some trajectories!

# In[3]:
# In[ ]:


# helper function
Expand Down Expand Up @@ -170,39 +166,39 @@ def plot_trajectory(coords, real, no_sol=None):


# As we can see, as time progresses, the solution becomes chaotic, making it very difficult to learn! We will now focus on building a Neural Operator using the `SupervisedSolver` class to tackle this problem.
#
#
# ## Averaging Neural Operator
#
#
# We will build a neural operator $\texttt{NO}$, which takes the solution at time $t=0$ for any $x\in\Omega$, the time $t$ at which we want to compute the solution, and gives back the solution to the KS equation $u(x, t)$. Mathematically:
#
#
# $$
# \texttt{NO}_\theta : \mathbb{U} \rightarrow \mathbb{U},
# $$
#
#
# such that
#
#
# $$
# \texttt{NO}_\theta[u(t=0)](x, t) \rightarrow u(x, t).
# $$
#
# There are many ways to approximate the following operator, for example, by using a 2D [FNO](https://mathlab.github.io/PINA/_rst/model/fourier_neural_operator.html) (for regular meshes), a [DeepOnet](https://mathlab.github.io/PINA/_rst/model/deeponet.html), [Continuous Convolutional Neural Operator](https://mathlab.github.io/PINA/_rst/model/block/convolution.html), or [MIONet](https://mathlab.github.io/PINA/_rst/model/mionet.html). In this tutorial, we will use the *Averaging Neural Operator* presented in [*The Nonlocal Neural Operator: Universal Approximation*](https://arxiv.org/abs/2304.13221), which is a [Kernel Neural Operator](https://mathlab.github.io/PINA/_rst/model/kernel_neural_operator.html) with an integral kernel:
#
#
# There are many ways to approximate the following operator, for example, by using a 2D [FNO](https://PINA-org.github.io/PINA/_rst/model/fourier_neural_operator.html) (for regular meshes), a [DeepOnet](https://PINA-org.github.io/PINA/_rst/model/deeponet.html), [Continuous Convolutional Neural Operator](https://PINA-org.github.io/PINA/_rst/model/block/convolution.html), or [MIONet](https://PINA-org.github.io/PINA/_rst/model/mionet.html). In this tutorial, we will use the *Averaging Neural Operator* presented in [*The Nonlocal Neural Operator: Universal Approximation*](https://arxiv.org/abs/2304.13221), which is a [Kernel Neural Operator](https://PINA-org.github.io/PINA/_rst/model/kernel_neural_operator.html) with an integral kernel:
#
# $$
# K(v) = \sigma\left(Wv(x) + b + \frac{1}{|\Omega|}\int_\Omega v(y)dy\right)
# $$
#
#
# where:
#
#
# * $v(x) \in \mathbb{R}^{\rm{emb}}$ is the update for a function $v$, with $\mathbb{R}^{\rm{emb}}$ being the embedding (hidden) size.
# * $\sigma$ is a non-linear activation function.
# * $W \in \mathbb{R}^{\rm{emb} \times \rm{emb}}$ is a tunable matrix.
# * $b \in \mathbb{R}^{\rm{emb}}$ is a tunable bias.
#
# In PINA, many Kernel Neural Operators are already implemented. The modular components of the [Kernel Neural Operator](https://mathlab.github.io/PINA/_rst/model/kernel_neural_operator.html) class allow you to create new ones by composing base kernel layers.
#
# **Note:** We will use the already built class `AveragingNeuralOperator`. As a constructive exercise, try to use the [KernelNeuralOperator](https://mathlab.github.io/PINA/_rst/model/kernel_neural_operator.html) class to build a kernel neural operator from scratch. You might employ the different layers that we have in PINA, such as [FeedForward](https://mathlab.github.io/PINA/_rst/model/feed_forward.html) and [AveragingNeuralOperator](https://mathlab.github.io/PINA/_rst/model/average_neural_operator.html) layers.
#
# In PINA, many Kernel Neural Operators are already implemented. The modular components of the [Kernel Neural Operator](https://PINA-org.github.io/PINA/_rst/model/kernel_neural_operator.html) class allow you to create new ones by composing base kernel layers.
#
# **Note:** We will use the already built class `AveragingNeuralOperator`. As a constructive exercise, try to use the [KernelNeuralOperator](https://PINA-org.github.io/PINA/_rst/model/kernel_neural_operator.html) class to build a kernel neural operator from scratch. You might employ the different layers that we have in PINA, such as [FeedForward](https://PINA-org.github.io/PINA/_rst/model/feed_forward.html) and [AveragingNeuralOperator](https://PINA-org.github.io/PINA/_rst/model/average_neural_operator.html) layers.

# In[4]:
# In[ ]:


class SIREN(torch.nn.Module):
Expand All @@ -226,10 +222,10 @@ def forward(self, x):


# Super easy! Notice that we use the `SIREN` activation function, which is discussed in more detail in the paper [Implicit Neural Representations with Periodic Activation Functions](https://arxiv.org/abs/2006.09661).
#
#
# ## Solving the KS problem
#
# We will now focus on solving the KS equation using the `SupervisedSolver` class and the `AveragingNeuralOperator` model. As done in the [FNO tutorial](https://github.com/PINA-org/PINA-tutorials/blob/main/tutorial5/tutorial.ipynb), we now create the Neural Operator problem class with `SupervisedProblem`.
#
# We will now focus on solving the KS equation using the `SupervisedSingleModelSolver` class and the `AveragingNeuralOperator` model. As done in the [FNO tutorial](https://PINA-org.github.io/PINA-Tutorials/tutorial5/tutorial.html), we now create the Neural Operator problem class with `SupervisedProblem`.

# In[ ]:

Expand All @@ -242,7 +238,7 @@ def forward(self, x):
output_variables=sol_train.labels,
)
# initialize solver
solver = SupervisedSolver(problem=problem, model=model)
solver = SupervisedSingleModelSolver(problem=problem, model=model)
# train, only CPU and avoid model summary at beginning of training (optional)
trainer = Trainer(
solver=solver,
Expand All @@ -259,7 +255,7 @@ def forward(self, x):

# We can now visualize some plots for the solutions!

# In[6]:
# In[ ]:


sample_number = 2
Expand All @@ -271,10 +267,10 @@ def forward(self, x):
)


# As we can see, we can obtain nice results considering the small training time and the difficulty of the problem!
# As we can see, we can obtain nice results considering the small training time and the difficulty of the problem!
# Let's take a look at the training and testing error:

# In[7]:
# In[ ]:


from pina.loss import PowerLoss
Expand All @@ -297,13 +293,13 @@ def forward(self, x):
# As we can see, the error is pretty small, which aligns with the observations from the previous plots.

# ## What's Next?
#
#
# You have completed the tutorial on solving time-dependent PDEs using Neural Operators in **PINA**. Great job! Here are some potential next steps you can explore:
#
#
# 1. **Train the network for longer or with different layer sizes**: Experiment with various configurations, such as adjusting the number of layers or hidden dimensions, to further improve accuracy and observe the impact on performance.
#
#
# 2. **Use a more challenging dataset**: Try using the more complex dataset [Data_KS2.mat](dat/Data_KS2.mat) where $A_k \in [-0.5, 0.5]$, $\ell_k \in [1, 2, 3]$, and $\phi_k \in [0, 2\pi]$ for a more difficult task. This dataset may require longer training and testing.
#
# 3. **... and many more...**: Explore other models, such as the [FNO](https://mathlab.github.io/PINA/_rst/models/fno.html), [DeepOnet](https://mathlab.github.io/PINA/_rst/models/deeponet.html), or implement your own operator using the [KernelNeuralOperator](https://mathlab.github.io/PINA/_rst/models/base_no.html) class to compare performance and find the best model for your task.
#
# For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/).
#
# 3. **... and many more...**: Explore other models, such as the [FNO](https://PINA-org.github.io/PINA/_rst/models/fno.html), [DeepOnet](https://PINA-org.github.io/PINA/_rst/models/deeponet.html), or implement your own operator using the [KernelNeuralOperator](https://PINA-org.github.io/PINA/_rst/models/base_no.html) class to compare performance and find the best model for your task.
#
# For more resources and tutorials, check out the [PINA Documentation](https://PINA-org.github.io/PINA/).
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Loading