Proper Orthogonal Decomposition Methods for Partial Differential Equations

By Zhendong Luo and Goong Chen

Proper Orthogonal Decomposition Methods for Partial Differential Equations evaluates the potential applications of POD reduced-order numerical methods in increasing computational efficiency, decreasing calculating load and alleviating the accumulation of truncation error in the computational process. Introduces the foundations of finite-differences, finite-elements and finite-volume-elements. Models of time-dependent PDEs are presented, with detailed numerical procedures, implementation and error analysis. Output numerical data are plotted in graphics and compared using standard traditional methods. These models contain parabolic, hyperbolic and nonlinear systems of PDEs, suitable for the user to learn and adapt methods to their own R&D problems.

• Explains ways to reduce order for PDEs by means of the POD method so that reduced-order models have few unknowns
• Helps readers speed up computation and reduce computation load and memory requirements while numerically capturing system characteristics
• Enables readers to apply and adapt the methods to solve similar problems for PDEs of hyperbolic, parabolic and nonlinear types

• Language: English
• Publisher: Academic Press
• Release date: Nov 26, 2018
• ISBN: 9780128167991

Zhendong Luo

Zhendong Luo is Professor of Mathematics at North China Electric Power University, Beijing, China. Luo is heavily involved in the areas of Optimizing Numerical Methods of PDEs; Finite Element Methods; Finite Difference Scheme; Finite Volume Element Methods; Spectral-Finite Methods; and Computational Fluid Dynamics. For the last 12 years, Luo has worked mainly on Reduced Order Numerical Methods based on Proper Orthogonal Decomposition Technique for Time Dependent Partial Differential Equations.

Book preview

VTeX

Foreword and Introduction

Zhendong Luo; Goong Chen     Beijing, China

College Station, TX, USA

We are living in an era of internet, WiFi, mobile apps, Facebook, Twitters, Instagram, selfies, clouds, …– surely we have not named them all, but one thing certain is that these all deal with digital and computer-generated data. There is a trendy name for the virtual space of all these things together: big data. The size of the big data space is ever-growing with an exponential rate. Major issues such as analytics, effective processing, storage, mining, prediction, visualization, compression, and encryption/decryption of big data have become problems of major interest in contemporary technology.

This book aims at treating numerical methods for partial differential equations (PDEs) in science and engineering. Applied mathematicians, scientists, and engineers are always dealing with big data, all the time. Where do their big data originate? They come, mostly, from problems and solutions of equations of physical and technological systems. A large number of the modeling equations are PDEs. Therefore, effective methods and algorithms for processing and resolving such data are much in demand. This is not a treatise on general big data. However, the main objective is to develop a methodology that can effectively help resolve the challenges of dealing with large data sets and with speedup in treating time-dependent PDEs.

Our approach here is not the way in which standard textbooks on numerical PDEs are written. The central theme of this book is actually on the technical treatment for effective methods that can generate numerical solutions for time-dependent PDEs involving only a small set of data, but yield decent solutions that are accurate and suitable for applications. It reduces data storage, CPU time, and, especially, computational complexity – several orders of magnitude. The key idea and methodology is proper orthogonal decomposition (POD), from properties of eigensolutions to a problem involving a large data set. (Indeed, POD has been known as an effective method for big data even before the term big data was coined.) In the process, we have developed the necessary mathematical methods and techniques adapting POD to fundamental numerical PDE methods of finite differences, finite elements, and finite volume in connection with various numerical schemes for a wide class of time-dependent PDEs as showcases.

PDEs and Their Numerical Solutions

Physical, biological, and engineering processes are commonly described by PDEs. Such processes are naturally dynamic, meaning that their time evolution constitutes the main features, properties, and significance of the system under investigation or observation. The spatial domains of definition of the PDEs are usually multidimensional and irregular in shape. Thus, there are fundamental difficulties involving geometry and dimensionality. The PDEs themselves can also take rather complex forms, involving a large variety of nonlinearities, system couplings, and source terms. These are inherent difficulties of the PDEs that compound those due to geometry and dimensionality. In general, exact (analytic) nontrivial solutions to PDEs are rarely available. Numerical methods and algorithms must be developed and then implemented on computers to render approximate numerical solutions. Therefore, computations become the only way to treat PDE problems quantitatively. The study of numerical solutions for PDEs is now a major field in computational science that includes computational mathematics, physics, engineering, chemistry, biology, atmospheric, geophysical and ocean sciences, etc.

Computational PDEs represent an active, prosperous field. New methods and developments are constantly emerging. However, three canonical schemes stand out: finite difference (FD), finite element (FE), and finite volume element (FVE) methods. These methods all require the division of the computational domain into meshes. Thus, they involve many degrees of freedom, i.e., unknowns, which are related to the number of nodes of mesh partition of the computational domains. For a routine, real-world problem in engineering, the number of unknowns can easily reach hundreds of thousands or even millions. Thus, the amount of computational load and complexity is extremely high. The accuracy of numerical solutions is also affected as truncation errors tend to accumulate. For a large-scale problem, the CPU time on a supercomputer may require days, weeks, or even months. It is possible, for example, if we use these canonical methods of FD, FE, and FVE to simulate the weather forecast in atmospheric science, after a protracted period of computer calculations, that the output numerical results have already lost their significance as the days of interest are bygone.

There are two ways of thinking for the resolution of these difficulties. First, one can think of computer speedup by building the best supercomputers with continuous refinement. As of June 2016, the world's fastest supercomputer on the TOP500 (http://top500.org) supercomputer list is the Sunway TaihuLight in China, with a LINPACK benchmark score of 93 PFLOPS (Peta, or 10¹⁵, FLOPS), exceeding the previous record holder, Tianhe-2, by around 59 PFLOPS. Tianhe-2 had its peak electric power consumption at 17.8 MW, and its annual electricity bill is more than $14 million or 100 million Chinese Yuan. Thus, most tier-1 universities cannot afford to pay such a high expense. The second option is to instead develop highly effective computational methods that can reduce the degrees of freedom for the canonical FD, FE, and FVE schemes, lighten the computational load, and reduce the running CPU time and the accumulation of truncation errors in the computational processes. This approach is based on cost-optimal, rational, and mathematical thinking and will be the one taken by us here.

The focal topic of the book, the POD method (see [56,60]), is one of the most effective methods that aims exactly at helping computational PDEs.

The Advantages and Benefits of POD

Reduce the degrees of freedom of numerical computational models for time-dependent PDEs, alleviate the calculation load, reduce the accumulation of truncated errors in the computational process, and save CPU computing time and resources for large-scale scientific computing.

POD in a Nutshell

The POD method essentially provides an orthogonal basis for representing a given set of data in a certain least-squares optimal sense, i.e., it offers ways to find optimal lower-dimensional approximations for the given data set.

A Brief Prior History of the Development of POD

The POD method has a long history. The predecessor of the POD method was an eigenvector analysis method, which was initially presented by K. Pearson in 1901 and was used to extract targeted, main ingredients of huge amounts of data (see [132]). (The trendy name of such data is big data.) Pearson's data mining, sample analysis, and data processing techniques are relevant even today.

However, the method of snapshots for POD was first presented by Sirovich in 1987 (see [150]). The POD method has been widely and successfully applied to numerous fields, including signal analysis and pattern recognition (see [43]), statistics (see [60]), geophysical fluid dynamics or meteorology (see also [60], [60] or [78]), and biomedical engineering (see [48]). For a long time since 1987, the POD method was mainly used to perform the principal component analysis in statistical computations and to search for certain major behavior of dynamic systems (see reduced-order Galerkin methods for PDEs, proposed in the excellent work in 2001 by Kunisch and Volkweind [62,63]). From that moment forth, the model reduction or reduced basis of the numerical computational methods based on POD for PDEs underwent some rapid development, providing improved efficiency for finding numerical solutions to PDEs (see [2,11,15,19,45,48,54,57,58,135,137,138,141,166,167,170,171,184–186,199]). At first, Kunisch–Volkweind's POD-based reduced-order Galerkin methods were applied to reduced-order models of numerical solutions for PDEs with error estimates presented in [62,63]. Those error estimates consist of some uncertain matrix norms. In particular, they took all . This produces some repetitive computation but not much extra gain. One begins to ponder how to improve this and, furthermore, how to generalize the methodology initiated by Kunisch–Volkweind's work beyond the Galerkin FE method to other FE methods and also to FD and FVE schemes. This book aims exactly at answering these questions.

Development of POD for Time-Dependent PDEs

The first author, Zhendong Luo, was attracted to the study of reduced-order numerical methods based on POD for PDEs at the beginning of 2003. At that time, few or no comprehensive accounts existed and only fragmentary introductions about POD were available. He spent three years (2003–2005) studying the underlying optimization methods, statistical principles, and numerical solutions for POD. Then, in 2006, he and his collaborators published their first two papers for POD methods (see [26,27]). These dealt with oceanic models and data assimilation.

Afterwards, Luo and his coauthors have established some POD-based reduced-order FD schemes (see [5,38,40,91,113,118,122,155]) as well as FE formulations (see [37,39,70,88–90,92,93,100,101,103,109,112,119,123,124,164]). They deduced the error estimates for POD-based reduced-order solutions for PDEs of various types since 2007 in a series of papers. They also proposed some POD-based reduced-order formulations and relevant error estimates for POD-based reduced-order FVE solutions (see [71,104,106,108,120]) for PDEs in another series of papers beginning in 2011. These POD-based reduced-order methods were specific to the classical FD schemes, FE methods, and FVE methods for the construction of the reduced-order models, in which they extracted one from every ten classical numerical solutions as snapshots. Therefore, these POD-based reduced-order methods constitute improvements, generalizations, and extensions for Kunisch–Volkweind's methods in [62,63]. The reduced-order methods in the above cited work need only repeat part .

Since 2012, Luo and his collaborators have established the following three main methods:

i.  PODROEFD: POD-based reduced-order extrapolation FD schemes (see [6,7,79,81,94–96,102,110,111,117,121,127,154,158]);

ii.  PODROEFE: POD-based reduced-order extrapolation FE methods (see [69,75,82,83,97,116,159–161,165,179]);

iii.  PODROEFVE: POD-based reduced-order extrapolation FVE methods (see [84,85,87,98,99,114,115,162,163]).

These POD-based reduced-order extrapolation methods ) of, respectively, the classical FD, FE, and FVE schemes as snapshots in order to formulate the POD basis. Therefore, they have significantly improved the previous, existing version of the reduced-order models. They do not have to repeat wholesale computations. The physical significance is that one can use existing data to forecast the future evolution of nature. Furthermore, our PODROEFD, PODROEFE, and PODROEFVE methods can be treated in a similar way as the classical FD, FE, and FVE methods, leading to error estimates with concrete orders of convergence. The application of these POD-based extrapolation methods will provide anyone with the advantages and benefits of POD mentioned earlier.

The second author, Goong Chen, has strong interests in the computation of numerical solutions of PDEs arising from real-world applications. He has constantly been faced with the challenges to deal with the needs for large data storage, process speedup, effective reduction of order, and the extraction of prominent physical features from the supercomputer numerical solution of PDEs. When he noticed that Zhendong Luo had already done significant work on the POD methods for time-dependent PDEs fitting many of his needs, he got very excited and proposed that a book project be prepared to publish and promote this very important topic. This started the collaboration of the authors, with this book as the outcome. Our collaboration is ongoing, hoping more research papers will be produced in the near future demonstrating the advantages of POD-based methods. However, G. Chen happily acknowledges that all technical contributions in this book are to be credited to the first author alone. He has learned tremendously from the collaboration – this by itself makes the book project worthwhile and satisfying as far as the second author is concerned.

Organization of the Book

In this book, we aim to provide the technical details of the construction, theoretical analysis, and implementations of algorithms and examples for PODROEFD, PODROEFE, and PODROEFVE methods for a broad class of dynamic PDEs. It is organized into the following four chapters.

Chapter 1 includes four sections. In the first section, we review the basic theory of classical FD schemes. It is intended to ensure the self-containedness of the book. Then, in the subsequent three sections, we introduce the construction, the theoretical analysis, and the implementations of algorithms for the PODROEFD schemes for the following two-dimensional (2D) PDEs: the parabolic equation, the nonstationary Stokes equation, and the shallow water equation with sediment concentration, respectively. Examples and discussions are also given for each equation.

Chapter 2 is similarly structured as Chapter 1, with four sections. There we begin by reviewing the basic theory of Sobolev spaces and elliptic theory, the classical FE method, and the mixed FE (MFE) method. Then we describe the construction, theoretical analysis, and implementations of algorithms for the PODROEFE methods for the following 2D PDEs: the viscoelastic wave equation, the Burgers equation, and the nonstationary parabolized Navier–Stokes equation (for which the stabilized Crank–Nicolson extrapolation scheme is used), respectively. Numerical examples and graphics are again illustrated.

Chapter 3 contains three sections, aiming at the treatment of PODROEFVE. We first introduce the basics of FVE. Then three sections for the construction, theoretical error analysis, and the implementations of algorithms for the PODROEFVE methods for the following three 2D dynamic PDEs are studied: the hyperbolic equation, Sobolev equation, and incompressible Boussinesq equation, respectively, are developed, with concrete examples and illustrations.

Numerical results on these model equations as presented in the book have demonstrated the effectiveness and accuracy of our POD methods.

Finally, Chapter 4 is a short epilogue and outlook, consisting of concluding remarks and forward-looking statements.

The book is written to be as self-contained as possible. Readers and students need only to have an undergraduate level in applied and numerical mathematics for the understanding of this book. Many parts can be used as a standard graduate-level textbook on numerical PDEs. The theory, methods, and computational algorithms will be valuable to students and practitioners in science, engineering, and technology.

Acknowledgments

The authors thank all collaborators, colleagues, and institutions that have generously supported our work. In particular, the authors are delighted to acknowledge the partial financial support over the years by the National Natural Science Foundation of China (under grant #11671106), the Qatar National Research Fund (under grant #NPRP 8-028-1-001), the North China Electric Power University, and the Texas A&M University.

Bibliography

[2] D. Ahlman, F. Södelundon, J. Jackson, A. Kurdila, W. Shyy, Proper orthogonal decomposition for time-dependent lid-driven cavity flows, Numerical Heat Transfer. Part B, Fundamentals 2002;42:285–306.

[5] J. An, Z.D. Luo, A reduced finite difference scheme based on POD basis and posteriori error estimate for the three dimensional parabolic equation, Acta Mathematica Scientia 2011;31A(3):769–775.

[6] J. An, Z.D. Luo, H. Li, P. Sun, A reduced spectral-finite difference scheme based on POD method and posteriori error estimate for the three-dimensional parabolic equation, Frontiers of Mathematics in China 2015;10(5):1025–1040.

[7] J. An, Z.D. Luo, P. Sun, A reduced finite difference scheme based on POD and posteriori error estimate for the two dimensional generalized nonlinear sine-Gordon equation, Acta Mathematicae Applicatae Sinica 2015;31(4):1–12.

[11] J. Baiges, R. Codina, S. Idelsohn, Explicit reduced-order models for the stabilized finite element approximation of the incompressible Navier–Stokes equations, International Journal for Numerical Methods in Fluids 2013;72(12):1219–1243.

[15] P. Benner, A. Cohen, M. Ohlberger, A.K. Willcox, Model Reduction and Approximation: Theory and Algorithm. Computational Science & Engineering. SIAM; 2017.

[19] J. Borggaard, T. Iliescu, Z. Wang, Artificial viscosity proper orthogonal decomposition, Mathematical and Computer Modelling 2011;53(1–2):269–279.

[26] Y.H. Cao, J. Zhu, Z.D. Luo, I.M. Navon, Reduced order modeling of the upper tropical Pacific Ocean model using proper orthogonal decomposition, Computers & Mathematics with Applications 2006;52:1373–1386.

[27] Y.H. Cao, J. Zhu, I.M. Navon, Z.D. Luo, A reduced order approach to four-dimensional variational data assimilation using proper orthogonal decomposition, International Journal for Numerical Methods in Fluids 2007;53:1571–1583.

[37] Z.H. Di, Z.D. Luo, Z.H. Xie, A.W. Wang, A reduced finite element scheme based on POD two dimensional unsaturated soil water flow equation, Journal of Beijing Jiaotong University 2011;35(3):142–149.

[38] Z.H. Di, Z.D. Luo, Z.H. Xie, A.W. Wang, I.M. Navon, An optimizing implicit difference scheme based on proper orthogonal decomposition for the two-dimensional unsaturated soil water flow equation, International Journal for Numerical Methods in Fluids 2012;68:1324–1340.

[39] J. Du, F. Fang, Z.D. Luo, J. Zhu, Reduced order modeling based on POD for finite element scheme of 3-D Boussinesq equations, Journal of Beijing Jiaotong University 2011;35(3):150–155.

[40] J. Du, J. Zhu, Z.D. Luo, I.M. Navon, An optimizing finite difference scheme based on proper orthogonal decomposition for CVD equations, International Journal for Numerical Methods in Biomedical Engineering 2011;27(1):78–94.

[43] K. Fukunaga, Introduction to Statistical Recognition. New York: Academic Press; 1990.

[45] R. Ghosh, Y. Joshi, Error estimate in POD-based dynamic reduced-order thermal modeling of data centers, International Journal of Heat and Mass Transfer