UNRAVEL is an NWO XL project with the goal to unravel neural networks through structure preserving computing.
The development of simplified computational models of complex fundamental phenomena in physics, chemistry, astronomy and biology is an ongoing challenge. The purpose of such simplified models is typically to reduce computational cost at a minimal loss of accuracy. At the same time, more importantly, these models can provide fundamental understanding of underlying phenomena.
Recently, the following two concepts have gained importance in computational science:
- machine learning (in particular neural networks)
- structure-preserving (mimetic or invariant-conserving) computing for mathematical models in physics, chemistry, astronomy, biology and more.
While neural networks are very strong as high-dimensional universal function approximators, they require enormous datasets for training and tend to perform poorly outside the range of training data. On the other hand, structure-preserving methods are strong in providing accurate solutions to complex mathematical models from science.
The goal of this research project is to better understand neural networks to enable the design of highly efficient, tailor-made neural networks built on top of and interwoven with structure-preserving properties of the underlying science problems that can serve as the simplified models mentioned above. This is unexplored terrain, and will lead to novel types of machine learning that are much more effective and have a much lower need for abundant sets of data. The proposed research is split into three parts:
- Analysis: Mathematical analysis of neural networks, e.g. variational analysis or modelling in terms of differential equation solvers. Dynamic neural networks will also be considered, and their relations to state space representations.
- Design: Design of novel neural networks with embedded invariance / enforced constraints, e.g. by adapting input features, neurons (activation functions), network architecture, or cost function, as well as developing a theory of model order reduction for neural networks.
- Application: Novel networks for fundamental problems from physics and astronomy, namely turbulence modelling and the challenging gravitational N-body problem.
The resulting deeper understanding of neural networks from mathematical, physical and astronomical point of view is vital for future developments in this rapidly developing area.