Higher Order Dynamic Mode Decomposition and Its Applications

By Jose Manuel Vega and Soledad Le Clainche

Higher Order Dynamic Mode Decomposition and Its Applications provides detailed background theory, as well as several fully explained applications from a range of industrial contexts to help readers understand and use this innovative algorithm. Data-driven modelling of complex systems is a rapidly evolving field, which has applications in domains including engineering, medical, biological, and physical sciences, where it is providing ground-breaking insights into complex systems that exhibit rich multi-scale phenomena in both time and space.

Starting with an introductory summary of established order reduction techniques like POD, DEIM, Koopman, and DMD, this book proceeds to provide a detailed explanation of higher order DMD, and to explain its advantages over other methods. Technical details of how the HODMD can be applied to a range of industrial problems will help the reader decide how to use the method in the most appropriate way, along with example MATLAB codes and advice on how to analyse and present results.

• Includes instructions for the implementation of the HODMD, MATLAB codes, and extended discussions of the algorithm
• Includes descriptions of other order reduction techniques, and compares their strengths and weaknesses
• Provides examples of applications involving complex flow fields, in contexts including aerospace engineering, geophysical flows, and wind turbine design

• Language: English
• Publisher: Academic Press
• Release date: Sep 22, 2020
• ISBN: 9780128227664

Jose Manuel Vega

Professor Vega currently holds a Professorship in Applied Mathematics at the School of Aerospace Engineering of the Universidad Politécnica de Madrid (UPM). He received a Master and a PhD, both in Aeronautical Engineering at UPM, and a Master in Mathematics at the Universidad Complutense de Madrid. Along the years, his research has focused on applied mathematics at large, including applications to physics, chemistry, and aerospace and mechanical engineering. The main topics were connected to the analysis of partial differential equations, nonlinear dynamical systems, pattern formation, water waves, reaction–diffusion problems, interfacial phenomena, and, more recently, reduced order models and data processing tools. The latter two topics are related, precisely, to the content of this book. Specifically, he developed (with Dr. Le Clainche as collaborator) the higher order dynamic mode decomposition method, and also several extensions, including the spatio-temporal Koopman decomposition method. His research activity resulted in the publication of more than one hundred and twenty research papers in first class referred journals, as well as around forty publications resulting from scientific meetings and conferences.

Book preview

techniques.

Preface

José M. Vega; Soledad Le Clainche     

The numerical simulation of many problems of scientific or industrial interest, modeled by partial differential equations, may involve a large number of numerical degrees of freedom. This is because, strictly speaking, the phase space associated with partial differential equations is infinite dimensional, meaning that it involves infinitely many degrees of freedom, which can lead to a very large number of numerical degrees of freedom upon discretization (large scale systems). However, for many such systems, the underlying physical laws introduce data redundancies, which allow for decreasing the number of numerical degrees of freedom to a smaller number of physically meaningful degrees of freedom (dimension reduction) within a good approximation. The experimental counterpart of these large scale systems involves similar difficulties and opportunities. In addition, experimental data may exhibit non-negligible noise, which must be filtered to get rid of the associated noisy unphysical artifacts.

The discretized outcomes of the problems mentioned above can give rise to very large databases. This can happen because of the involved high dimensionality (one or more space dimensions, time, and perhaps various involved parameters) and/or a large number of grid points along some dimensions. Efficient manipulation and extracting knowledge (i.e., identifying the relevant patterns that are present) from the very large amount of involved data is of high interest. This can be performed using appropriate (purely data-driven) post-processing tools, which are the main object of the present book.

Post-processing tools can be used to construct data driven reduced order models, which allow for the very fast online simulation of the system. These reduced order models are constructed from just experimental or numerical (obtained via black-box numerical solvers) data, without using the governing equations at all. Other reduced order models, known as projection-based reduced order models, whose construction requires using the governing equations, are outside the scope of the present book and will only be briefly addressed for completeness. On the other hand, projection-based reduced order models can be used to efficiently obtain the data needed to construct data-driven reduced order models. In other words, these two types of reduced order models are complementary of each other.

Against this background, this book focuses on two recent post-processing tools developed by the authors. The main tool is the higher order dynamic mode decomposition [101], which is an improvement of the well-known standard dynamic mode decomposition [154] (see also [146,155,179]). Because of the improvement, the new tool gives good results in cases in which standard dynamic mode decomposition fails, as will be explained and illustrated in several toy models and in some, less academic applications along the book. Both standard dynamic mode decomposition and higher order dynamic mode decomposition apply to dynamics that exhibit exponential/oscillatory behavior in the temporal coordinate. The second tool is the spatio-temporal Koopman mode decomposition [104], which is a spatio-temporal extension of higher order dynamic mode decomposition that simultaneously treats time and some distinguished spatial coordinates, which will be called the longitudinal coordinates. This tool applies to dynamics that exhibit exponential/oscillatory behavior in both time and the longitudinal coordinates. In particular, this method is very efficient in identifying pure or modulated traveling and standing waves, as will be seen along the book. Developing these methods requires using more classical postprocessing tools that exhibit their own interest and will also be considered.

Preliminary versions of a part of the material presented in the book have been already published in research papers (which will be referred to) and presented in scientific meetings and conferences. In preparing the book, the authors have benefited from several discussions on the methods with, e.g., Marta Net and Joan Sánchez (who provided us with some computer programs that will be included in an annex to Chapter 2 and also with some numerical data that will be used in Chapter 5), from the Universitat Politècnica de Catalunya, Barcelona, Spain, and also from suggestions on the applicability of the methods to various problems from colleagues. These include, in particular, Julio Soria, from Monash University, Melbourne, Australia, Xueri Mao, from Nottinghan University, Nottinghan, U.K., Rubén Moreno-Ramos, from Altram Co., Madrid, Spain, and Paul Taylor, from Gulfstream Co., Savanah, Georgia, USA.

The book is foremost intended for researchers, including young researchers, and graduate students with knowledge of MATLAB® programming, numerics, advanced linear algebra, and fluid dynamics. Thus, the book could be used as an advanced graduate textbook. In order to make the material user friendly, the implementation of the various tools in the MATLAB environment is indicated by addressing to the appropriate MATLAB commands, when possible, and giving the relevant MATLAB functions (in annexes to the various chapters) otherwise. The methods will be illustrated using some toy models. In addition, some practice problems are given in the annexes to the various chapters.

Already existing related books include both descriptions of dynamic mode decomposition, some of its previous extensions [90], and applications to dynamic programming [81] and fluid mechanics [30]. The dynamic mode decomposition improvements in the present book are both robust and flexible. Also, a variety of applications of scientific and industrial interest are addressed, including pattern forming systems such as the Ginzburg–Landau equation and thermal convection problems, various fluid dynamics systems (e.g., the cylinder wake, the zero-net-mass-flux jet), and some more industrially oriented applications, such as magnetic resonance, aircraft flight tests, and various wind turbine flows.

Madrid, March 2020

References

[30] Y.A. Cengel, J.M. Cimbala, eds. Fluid Mechanics: Fundamentals and Applications. McGraw-Hill; 2013.

[81] Y. Jiang, Z.-P. Jiang, Robust Adaptive Dynamic Programming. Wiley; 2017.

[90] J.N. Kutz, J. Nathan, S.L. Brunton, B.W. Brunton, J.L. Proctor, Dynamic mode decomposition: data-driven modeling of complex systems, Soc. Ind. Appl. Math. 2016.

[101] S. Le Clainche, J.M. Vega, Higher order dynamic mode decomposition, SIAM J. Appl. Dyn. Syst. 2017;16:882–925.

[104] S. Le Clainche, J.M. Vega, Spatio-temporal Koopman decomposition, J. Nonlinear Sci. 2018;28:1793–1842.

[146] C.W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, D.S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech. 2009;641:115–127.

[154] P.J. Schmid, J.L. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bull. Am. Phys. Soc., 61st APS meeting. San Antonio. 2008:208.

[155] P.J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech. 2010;656:5–28.

[179] J.H. Tu, C. Rowley, D.M. Luchtemburg, S.L. Brunton, J.N. Kutz, On dynamic mode decomposition: theory and applications, J. Comput. Dyn. 2014;1:391–421.

Chapter 1: General introduction and scope of the book

Abstract

This chapter addresses some preliminary data processing tools that are needed in the remainder of the book. These tools include singular value decomposition in its various versions (economy, compact, and truncated singular value decomposition), proper orthogonal decomposition, and higher order singular value decomposition in its various versions (economy, compact, and truncated higher order singular value decomposition). These tools are illustrated with some toy models. An introduction to reduced order models, both data-driven and projection-based, is also included. The organization of the book is summarized at the end of the chapter. Some selected practice problems are proposed in Annex 1.1. Some MATLAB functions are also given in Annex 1.2 that allow for computing the various versions of singular value decomposition and higher order singular value decomposition.

Keywords

Multidimensional databases; Post-processing tools; Singular value decomposition; Proper orthogonal decomposition; Higher order singular value decomposition; Data-driven reduced order models; Projection-based reduced order models

1.1 Introduction to post-processing tools

Database post-processing tools for matrices (namely, two-dimensional databases) and higher than two order tensors (namely, higher-dimensional databases) are used for various purposes, including filtering out errors in noisy databases, decreasing the database size (compression), and extracting the underlying relevant patterns. These tools can also be used for constructing purely data-driven reduced order models (ROMs), which allow for the fast online simulation of the physical systems from which the databases have been extracted.

There are many data-processing tools [182,189], also referred to as, e.g., data analytics or data mining tools, depending on the context. Here, we concentrate in those that are most closely related to the scope of the book and, in fact, will be needed along the book. These tools apply to both real and complex data.

1.1.1 Singular value decomposition

Let us begin with the singular value decomposition (SVD) method, which was invented [168] independently by Beltrami and Jordan (in 1873-74) for square matrices and further developed by Eckart and Young [54] for general rectangular matrices.

matrix A of rank r is given by [71]

(1.1)

where the superscript H denotes hereinafter the Hermitian adjoint, namely the transpose conjugate, i.e.,

(1.2)

with the overbar and the superscript ⊤ denoting the complex conjugate and the transpose, respectively. Note that the Hermitian adjoint is computed with the MATLAB® command ‘prime’ and that it reduces to the transpose for real matrices. U and V are ) matrices, whose columns, known as the left and right modes, respectively, of the decomposition, are mutually orthonormal with the Hermitian inner product. Namely, they are such that

(1.3)

. The matrix S is diagonal and its elements, known as the singular values , are real, non-negative, and usually sorted in decreasing order, namely

(1.4)

Note that, because of round-off errors, the left and right sides of (1.1) are not equal in finite precision computations, but exhibit a difference comparable to zero-machine.

There are several versions of SVD, which differ among each other in the sizes of the matrices U, S, and V. For instance, in the version economy SVD, computed with the MATLAB command ‘svd’, option ‘econ’, the sizes of the matrices U, S, and V are . In this case, Eq. (1.1) can also be written in terms of the elements of the various matrices as

(1.5)

whose right hand side can be seen as a linear combination of linearly independent rank-one matrices. In fact, the rank of A can be defined as the minimum number of involved rank-one matrices such that the decomposition , the definition of the rank as the minimum number of rank-one matrices such that the expansion (1.5) is possible implies that only the first r ones are zero and can be ignored. Namely, Eq. (1.5) can be substituted by

(1.6)

This is consistent in Eq. (1.1) with retaining only the first r singular values in the matrix S and the first r columns in both U and V. The resulting version of SVD is known as compact SVD. For compact SVD, Eqs. (1.3) and (1.4) read

(1.7)

and

(1.8)

where, as above, r is the rank of the matrix A. For convenience (to eliminate round-off errors) we do not consider the exact rank here, but the approximate rank, defined as the maximum value of r such that

(1.9)

Using this, compact SVD is implemented in the MATLAB function ‘CompactSVD.m’, given in Annex 1.2, at the end of this chapter.

Figure 1.1 Illustration of the columns (left) and rows (right) of a matrix.

As in economy SVD, defined in Eq. (1.5), because of round-off errors and truncation, the approximation provided by compact SVD in terms of the relative root mean square (RMS) error (see Eq. (1.16) below) is slightly larger than zero-machine.

Note that, generically, the columns of A . In other words, generically, the rank of A and the columns of A , namely

(1.10)

then the rank of A . In this case, the singular values computed by the economy SVD method are (truncated to five significant digits)

(1.11)

Using compact SVD, instead, only the first two nonzero singular values are retained, namely the singular values are

(1.12)

It turns out that the Frobenius norm of the matrix A, defined as the square root of the sum of the squares of the absolute values of the elements of A, is given by [71]

(1.13)

Truncation in the expansion singular values, yields an approximation of A by a smaller rank (namely, a smaller effective size) matrix, within an error [71]

(1.14)

denotes the truncated SVD reconstruction of the matrix A singular values. Equations (1.13) and (1.14) mean that

(1.15)

where, for an approximation to a vector, matrix, or tensor A, the relative RMS error, RRMSE, is defined hereinafter as

(1.16)

denoting the Euclidean norm for vectors and the Frobenius norm for matrices and tensors. Note that, in all cases, this norm is the square root of the sum of the squares of the absolute values of the elements of the vector, matrix or tensor. When the elements of A are strongly correlated, the singular values decay fast and the rank reduction is very large keeping the relative RMS error of the approximation, RRMSE (as defined in Eq. (1.16)), very small. On the other hand, appropriate truncation of SVD is a very good means for cleaning noisy databases [167,181] provided that noise is somewhat uncorrelated with the physically meaningful data.

According to , the following must be imposed:

(1.17)

(no strict truncation). The resulting algorithm, retaining only these singular values, is called truncated SVD, and implemented in the MATLAB function ‘TruncSVD.m’, given in Annex 1.2 at the end of this chapter. Note that truncated SVD can also be computed replacing compact SVD by economy SVD because the additional singular values introduced by the latter method are zero (or extremely small) and thus they do not contribute to the left hand side of (1.17).

Equation is sometimes selected as the smallest index such that

(1.18)

, which obviously does not coincide with its counterpart in Eq. ), condition (1.18) is either comparable or even more strict than condition (1.17).

1.1.2 A toy model to illustrate SVD

Let us now consider a toy model illustrating the various versions of SVD given above using the function (plotted in Fig. 1.2)

(1.19)

factor in the last term, it is not analytic in the considered domain.

Figure 1.2 The function f defined in Eq. (1.19).

Using the function matrix

(1.20)

. Applying compact SVD (implemented in the MATLAB function ‘CompactSVD.m’ given in Annex . This error is almost identical to that obtained via the economy SVD (implemented in the MATLAB command ‘svd’, option ‘econ’). Thus, the truncation associated with the approximate rank does not increase the reconstruction error significantly.

The singular values retained by compact SVD are plotted in .

Figure 1.3 Singular values σ n (as computed using compact SVD) vs. n for the matrix defined in Eq. (1.20) with I = J = 100.

On the other hand, using in both cases.

It is interesting to compare the singular values provided by compact SVD, plotted in vs. n (precisely the approximate rank of A as computed by compact SVD). The first 18 singular values almost exactly coincide with their counterparts computed by compact SVD, while the remaining ones are extremely small and affected by round-off errors.

Figure 1.4 Counterpart of Fig. 1.3 using economy SVD.

Let us now see the effect of controlled noise. To this end, we add to the matrix A a random noise with zero mean constructed with the MATLAB command ‘rand’, which gives uniformly distributed noise varying between 0 and 1. Namely

(1.21)

is defined as

(1.22)

Note that this noise exhibits zero mean, while its RMS value is seen to be

(1.23)

, the singular values are as plotted in vs. n , exhibits a RMS error (compared to the clean