Conference Agenda

This talk mainly focuses on CNN-FGMRES, a hybrid algorithm that integrates classical numerical linear algebra methods with nonlinear neural operator preconditioning to accelerate the solution of some parametric Partial Differential Equations (PDEs). Specifically, we consider Krylov subspace methods, such as Flexible GMRES (FGMRES), combined with nonlinear preconditioners derived from the trained neural operator models with convolutional neural networks (CNN). The parametric PDEs addressed include the Helmholtz equations, Poisson equations, Darcy flow, and Diffusion-Advection equations, spanning both academic benchmarks and practical datasets. Compared to the classical numerical preconditioners, the trained neural operator preconditioning exhibits significant generalization capabilities in addressing a wide range of numerical and parametric variations. Besides, it owns a matrix-free implementation, indicating it can address some large-scale problems that memory constraints restrict classical numerical preconditioners. In addition, it supports both CPU and GPU implementations. On the other hand, compared to the pure neural network solver, CNN-FGMRES can reach machine precision with much less computational cost than the classical subspace methods. Overall, this work demonstrates the efficiency and flexibility of combining modern neural networks with classical Krylov methods, leveraging the strengths of both to achieve higher attainable accuracy and broader applicability across diverse problem settings. This talk also covers some related follow-ups on generative models for applications to other PDE-based problems.

While traditional high-fidelity methods for solving Partial Differential Equations (PDEs) are highly accurate, their computational cost in many-query scenarios has led to the rise of Scientific Machine Learning (SciML) as a promising paradigm to build data-driven surrogates that incorporate physical and mathematical knowledge. Among recent SciML architectures, Latent Dynamics Networks (LDNets) have demonstrated remarkable performance in predicting the response of spatio-temporal systems, combining Neural ODEs with reduced-order modeling. However, their application has been limited by fixed initial states. In this work, we generalize the LDNet architecture to handle variable initial conditions, broadening their applicability, and enabling the prediction of system evolution from arbitrary measured states. To achieve this, we introduce novel auto-decoding strategies to effectively represent the initial conditions within the lower-dimensional latent space. Crucially, this approach retains the encoder-free nature of the original LDNet architecture, thus preserving resolution independence and the ability to seamlessly process various data formats (e.g., images, graphs). Furthermore, we integrate meta-learning techniques by treating the initial condition as task-specific information, significantly enhancing model generalization and performance. We demonstrate the efficacy of our proposed framework across several challenging benchmarks, ranging from advection-diffusion-reaction equations to computational fluid dynamics.

Deep learning-based hybrid iterative methods (DL-HIMs) promise accelerated convergence by combining the complementary spectral biases of classical numerical solvers and neural operators. However, these methods frequently stall at "false fixed points", where neural updates vanish despite unacceptably large physical residuals, raising significant concerns about their reliability in scientific computing. In this talk, we demonstrate that DL-HIM performance is higly sensitive to training paradigms and update strategies, independent of the underlying neural architecture. Through a detailed analysis of a DeepONet-based solver (HINTS) and an FFT-based Fourier neural solver (FNS), we demonstrate how misaligned training objectives allow physical residuals to persist. Furthermore, we show that classical Anderson acceleration (AA) is poorly suited for nonlinear neural operators. To resolve this, we introduce Physics-Aware Anderson Acceleration (PA-AA), which minimizes the physical residual rather than the fixed-point update. Numerical experiments confirm that PA-AA successfully circumvents false fixed points, restoring reliable convergence in substantially fewer iterations. We conclude that reliability depends not only on architectures, but on physically informed training and iteration design.

Physical laws, such as the conversation of mass and momentum, are fundamental principles in
many physical systems. Neural operators have achieved promising performance in learning the
solutions to those systems, but often fail to ensure conservation. Existing methods typically
enforce strict conservation via hand-crafted post-processing or architectural constraints, leading to
limited model flexibility and adaptability. In this talk I will present a novel plug-and-play adaptive
correction approach to ensure the conservation of fundamental linear and quadratic quantities
for neural operator outputs. The method introduces a lightweight learnable operator to adap-
tively enforce the target conservation law during training. This method allows the model to flex-
ibly and adaptively correct its output to guarantee strict conservation. I will provide a theoretical
result showing that the correction method does not hamper the expression ability of neural operators and can potentially achieve lower recontruction loss than their conservation-constrained
counterparts. The method will be demonstrated across muliple neural operator architectures and represen-
tative PDEs. Extensive experiments will show tha incorporating the correction method into baseline models significantly improves both accuracy and stability. In addition, the experimental results will demonstrate that this approach consistently achieves superior performance over widely used conservation-enforcement techniques on various PDE benchmarks.