Environmental Science Division (EVS)a Division of Argonne National Laboratory
Predictive environmental understanding

Deep Learning Algorithm for Data-Driven Simulation of Noisy Dynamical System

Friday, April 12, 2019

Dr. Kyongmin Yeo
IBM T. J. Watson Research Center
Friday, April 12, 2019
11:00 A.M. – 12:30 P.M.
Argonne National Laboratory
TCS Building 240
Room 4301

In many real physical systems, we do not have an access to the complex spatiotemporal dynamics and often it is challenging to use physics models to understand the state of the system due to the high computational cost as well as the high aleatoric and epistemic uncertainties. Hence, we try to understand the state of the system from limited information acquired through a sensor network. While there are many well established statistical techniques for the modeling of the sensor data, most of the classical methods have limited capability when the underlying dynamical system exhibits complex behaviors, such as nonlinear and multiscale dynamics.

Dr. Yeo will present a deep learning model (DE-LSTM) for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to identify the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. By using the numerical discretization, the function estimation problem can be solved by a multi-label classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the prediction of the probability density function.

Dr. Yeo will show that the time evolution of the probability density function can be computed by a high‑dimensional integration of the transition probability of the LSTM internal states. A Monte Carlo algorithm to approximate the high-dimensional integration is outlined. The behavior of DE-LSTM is thoroughly investigated by using stochastic processes and noisy observations of nonlinear dynamical systems. It is shown that DE-LSTM makes a good prediction of the probability density function without assuming any distributional properties of the underlying stochastic process. For a multiple-step forecast of a chaotic system, the prediction uncertainty, denoted by the 95% confidence interval, first grows, then dynamically adjusts following the evolution of the system, while in the simulation of the forced Van der Pol oscillator, the prediction uncertainty does not grow in time even for a 3,000-step forecast.

Dr. Kyongmin Yeo
Dr. Kyongmin Yeo
portrait of Umakant Mishra