Differential Calculus and Its Applications

By Michael J. Field

This text offers a synthesis of theory and application related to modern techniques of differentiation. Based on undergraduate courses in advanced calculus, the treatment covers a wide range of topics, from soft functional analysis and finite-dimensional linear algebra to differential equations on submanifolds of Euclidean space. Suitable for advanced undergraduate courses in pure and applied mathematics, it forms the basis for graduate-level courses in advanced calculus and differential manifolds.
Starting with a brief resume of prerequisites, including elementary linear algebra and point set topology, the self-contained approach examines liner algebra and normed vector spaces, differentiation and calculus on vector spaces, and the inverse- and implicit-function theorems. A final chapter is dedicated to a consolidation of the theory as stated in previous chapters, in addition to an introduction to differential manifolds and differential equations.

• Language: English
• Publisher: Dover Publications
• Release date: Apr 10, 2013
• ISBN: 9780486298849

Book preview

INDEX

Preface

My primary concern in writing this book about several-variable differential calculus has been to achieve a synthesis of theory and application in the framework of the modern approach to differentiation as expounded in Dieudonné’s treatise on the Foundations of Modern Analysis. By ‘application’ I mean here not only a working knowledge and familiarity with the techniques of differentiation but also applications of the theory to applied mathematics and other areas of pure mathematics, notably differential topology and differential equations.

The text is based on undergraduate courses on advanced calculus that I have taught at the Universities of Minnesota and Warwick and covers a fairly wide range-from soft functional analysis and finite-dimensional linear algebra to differential equations on submanifolds of Euclidean space. I hope that the book will not only be suitable for the final two years of an undergraduate degree course in pure or applied mathematics but will also form a suitable basis for graduate level courses on advanced calculus and differential manifolds.

Chapter 0 contains a brief resumé of prerequisites. These essentially consist in a knowledge of elementary linear algebra (linear maps, matrices, bases, etc.) together with a little point set topology. For the latter, it is sufficient for most of the text to know only the rudiments of metric space theory (that is, the definitions of a metric space, metric space topology, continuous map and completeness). Apart from the material referred to in Chapter 0, the book is relatively self-contained.

Chapter 1 naturally splits into two parts. The first half is mainly soft functional analysis and includes the elementary theory of normed vector spaces, inner product and Hilbert spaces, spaces of continuous linear and multilinear maps and the Riesz representation and Hahn-Banach theorems. The second half is devoted to finite-dimensional theory and includes the spectral theorem for self-adjoint and orthogonal maps (though not the Jordan canonical form theorem) and the elementary theory of quadratic forms.

In Chapter 2 we give a systematic exposition of the elementary theory of differentiation on normed vector spaces. Stress is laid on examples of differentiable functions defined on open subsets of Euclidean space. There are sections on extrema, curves in Rn, various forms of Taylor’s theorem and higher derivatives.

More advanced topics in the theory of differentiation are covered in Chapter 3. Among these are the inverse-function theorem, implicit-function theorem and the rank theorem. In the final section of this chapter, coordinate transformations are discussed together with applications to the solution of partial differential equations.

The final chapter is intended both as a consolidation of the theory given in the preceding chapters and as an introduction to differential manifolds and differential equations. The key results of Chapters 2 and 3 are repeatedly used in this chapter. Critical point theory, differential structures, submanifolds of Euclidean space, orientation of hypersurfaces, vector fields, tangent spaces and bundles are among the topics discussed.

Finally, one or two remarks about notation. The derivative of a map f: E F at x will be denoted by Dfx (not Df(x)) and will be a linear map from E to F. The value of Dfx at e E will be denoted by Dfx(e) (not Df(x)(e)). Arbitrary functions will be denoted by f, g, h, … but linear maps will usually be denoted by capitals S, T, U, … We write the composition of the functions g and f by gf but the composition of the linear maps S and T by S .T. This ‘dot’ notation is expanded considerably in the text; suffice it to say that its use helps simplify many computations and formulae. For typographical convenience and clarity of exposition I have adopted the notation ‘f;j(x)’ to mean the jth partial derivative of f at x. These, and other conventions, are explained more fully in the text.

September, 1975

M. J. FIELD

CHAPTER 0

Preliminaries

0.1

Vector spaces and linear maps

In this section we shall make a brief survey of that part of the elementary theory of vector spaces and linear maps that will be assumed in the rest of the text. For more detailed expositions we refer the reader to any of the many basic texts on linear algebra and finite-dimensional vector spaces (Halmos [9] is particularly recommended).

The symbol K will always be used to denote a field which, in practice, will either be R (the ‘real numbers’) or C (the ‘complex numbers’). The identity element of K will be denoted by 1 and the zero by 0.

Definition 0.1.1

A vector space over the field K consists of a set E, with distinguished element 0E, together with operations

such that the following conditions are satisfied.

1. The set E forms an Abelian group under + with zero element 0E

(a) e + f = f + e for all e, f ∈ E.

(b) e + (f + g) = (e + f) + g for all e, f, g ∈ E.

(c) e + 0E = 0E + e for all e ∈ E.

(d) For every e ∈ E, there exists a unique element −e ∈ E such that e + (−e) = 0E.

2. k(e + f) = ke + kf for all e, f ∈ E and k ∈ K.

3. (k + l)e = ke + le for all k, l ∈ K and e ∈ E.

4. k(le) = (kl)e, for all k, l ∈ K and e ∈ E.

5. le = e for all e ∈ E.

We shall denote a vector space by the symbol used for its underlying set. Thus, in the definition, the vector space would be denoted by E. In the sequel we shall usually denote the zero element of E by 0, as opposed to 0E. Thus we shall write 0e = 0 rather than 0e = 0E. We shall also often omit reference to the underlying field of scalars as it will generally remain fixed during the course of discussion.

We recall the definition of a vector subspace.

Definition 0.1.2

A subset F of the vector space E is said to be a vector subspace of E if the restriction of the vector space operations of E to F induces on F the structure of a vector space.

Remark. In order that F be a vector subspace of E it is clearly necessary and sufficient that

(a) F is closed under addition: e + f ∈ F for all e, f ∈ F.

(b) F is closed under scalar multiplication: ke ∈ F for all k ∈ K and e ∈ F.

The basic objects of study in the theory of vector spaces are linear maps, and we recall their definition.

Definition 0.1.3

Let E and F be vector spaces over the field K and let T : E F be a function from E to F. We call T a linear map or linear homomorphism (strictly, a K-linear map or K-linear homomorphism) if the following condition is satisfied for all e, f ∈ E and k ∈ K:

T(ke + f) = kT(e) + T(f)

Associated with a linear map T : E F we have the following vector spaces:

The kernel of T, Ker T, defined by Ker T = {e ∈ E: T(e) = 0}.

The image of T, Im T, defined by Im T = {f ∈ F: ∃ e ∈ E with T(e) = f}.

The cokernel of T, Coker T, defined by Coker T = F/Im T.

Ker T is a vector subspace of E and Im T is a vector subspace of F.

T is said to be an injection (or monomorphism, or 1:1) if Ker T = {0}.

T is said to be a surjection (or epimorphism or onto) if Im T = F.

T is said to be an isomorphism if T is injective and surjective.

If T : E E we say that T is an endomorphism of E.

Definition 0.1.4

Let E = (ei)i ∈ I be a collection of elements of E indexed by the set Ia (Hammel) basis for E if

(a) Every e ∈ E :

    where kj ∈ K,     ej j m.

.

A standard theorem of linear algebra asserts that every vector space has a basis.

We are particularly interested in the case when I is finite. (If I is infinite, Definition 0.1.4 does not give a very useful definition of basis.) We have the following fundamental theorem.

Theorem 0.1.5

Let E be a vector space over the field K = {e1, …, em} be a basis for E containing m elements. Then every other basis for E has exactly m elements, m is called the dimension of E, dimKE. Moreover, every element x ∈ E can be written uniquely in the form

where xi ∈ Ki m.

Proof

Remarks

1. The m-tuple (x1, …, xm) ∈ Km given by Theorem 0.1.5 is said to define the coordinates of x of E.

2. It can be shown that if E has an infinite basis, then the cardinal of the basis is an invariant of the space. The coordinate representation given by the basis is, however, no longer useful.

Examples

= {e1, …, em} be a basis for the vector space E. Then e1 has coordinates (1, 0, 0, …, 0). Similarly for ej, j > 1.

2. Km has a standard or canonical basis defined by taking the set of m-tuples

{(1, 0, 0, …, 0), (0, 1, 0, …, 0), …, (0, …, 0, 1)}.

We have an obvious identification between the coordinates of a basis vector and the vector (compare the previous example). We always refer to this basis on Km as the standard basis of Km.

0.1.6 THE MATRIX OF A LINEAR MAP WITH RESPECT TO A BASIS

For computational purposes it is often useful to have a numerical description of a linear map in terms of elements of the underlying field K.

Let A : E F be a K-linear map between the vector spaces E and F. Suppose that E and F are finite dimensional with respective bases

Define elements aij ∈ K by the rule

That is

aij = ith coordinate of A(ej)

The m × n array of elements of K,

is called the matrix of A .

We usually denote the matrix of A in abbreviated form by

[A]     or     [aij].

If it is necessary to emphasize the dependence of the matrix of A , we may also write

[A    or     [aij.

for E and F respectively and an m × n matrix [aij] of elements of K. Using the fact that a linear map is uniquely defined by specifying its values on the basis elements of E, we may define a linear map A : E F, associated with the matrix [aij], by the rule

Notice that, if x ∈ E has coordinates (x1, …, xmof E, then A(x) is given, relative to the coordinate system on F, by

In other words, the coordinates of A(x) are given by the usual rules of matrix multiplication.

Example= {e1, …, em} be a basis of E. We may define a linear map A : E Km by

The matrix of A is the identity matrix

Notice that A is not the identity map unless E = Km is the standard basis of Km!

Finally, we note that the above description of matrices easily gives us the correct rule for matrix multiplication. More specifically, suppose that A : E F and B : F G are linear maps. We shall denote the composite of A and B by B . A : E G. The reader may easily verify that B . A be bases for E, F and G respectively. We set

[A] = [aij],     B = [bij],     B . A = [cij];

[A], [B] and [B : A] are p × q, q × r and p × r matrices respectively. Then,

Definition 0.1.7

Let F and G be vector subspaces of E. Suppose that F ∩ G = {0}. The direct sum of F and G is the vector subspace F ⊕ G of E defined by

F ⊕ G = {f + g: f ∈ F, g ∈ G}.

If F ⊕ G = E then F and G are said to be complementary subspaces of E.

Closely related to the notion of direct sum we have the definition of a product of vector spaces.

Definition 0.1.8

Let E and F be two vector spaces over the same field, The product of E and F is the vector space E × F defined by

E × F = {(e, f): e ∈ E, f ∈ F},

with addition and scalar multiplication defined coordinatewise in the obvious way.

Example. The relation between direct sum and product is shown by the following: Let F and G be complementary subspaces of E; then F ⊕ G is canonically isomorphic to F × G.

Remark. The term ‘canonical’ used in the above example means that the isomorphism is defined independently of any choice of bases for E and F. Sometimes the more suggestive term ‘natural’ is used instead, but we shall regard these terms as interchangeable in the sequel.

Exercises

1. Let E and F be vector spaces and suppose that T : E F is linear. Show that the graph of T is a vector subspace of E × F.

2. Let

be two bases for E and

be two bases for F.

′ is the linear map A : E E defined by

The change of coordinates map B : F F ′ is defined similarly. Show that if [T] is the matrix of a linear map T : E F, then

What is the corresponding result if we wish to find how the linear map varies if we fix the matrix and vary the bases?

4. Let E and F and dimensions m and n. Let M(m, n; k) denote the set of all m × n matrices with coefficients in K and L(E, F) denote the set of all linear maps from E to F. Prove that

(a) M(m, n: k) is a K-vector space of dimension mn under the usual operations of matrix addition and scalar multiplication.

(b) The map L(E, FM(m, n; k); T [T], is bijective. Find operations of addition and scalar multiplication which make L(E, F) a K-vector space and the map of (b) a vector-space isomorphism. What is the inverse of the map given in (b)?

0.2

Topological and metric spaces

In this section we briefly list some of the basic definitions and properties of topological and metric spaces. For more details we refer the reader to [11], [15]. The material presented here on metric spaces will be of the greater relevance in the sequel.

First let us recall the definition of a topological space.

Definition 0.2.1

Let X a collection of subsets of X. We say the pair (X) is a topological space if

1. X denotes the ‘empty set’).

are called open subsets of the topological space (X).

In the sequel we shall often abbreviate (Xto X, and refer instead to the topological space X.

Definition 0.2.2

Let (X) be a topological space. A subset V of X is said to be closed if X\V .

A large source of examples of topological spaces is given by metric spaces.

Definition 0.2.3

Let X be a set and d: X × X R be a function. Then the pair (X, d) is said to define a metric space if

1. d(x, y0 for all pairs (x, y) ∈ X × X.

2. d(x, y) = 0 if and only if x = y.

3. d(x, y) + d(y, zd(x, z) for all x, y, z ∈ X (‘triangle inequality’).

4. d(x, y) = d(y, x) for all (x, y) ∈ X × X.

d is called the distance function or metric on X.

In the sequel we shall often refer to (X, d) as the metric space X.

If (X, d) is a metric space then we can construct a topology on X, ‘the metric topology of X’, as follows:

First, for all r > 0 and x ∈ X, we set

Br(x) is called the open disc centre x and radius ris called the closed disc centre x and radius r.

An open set for the metric topology of X is then defined to be any subset of X which can be expressed as a union of open discs (technically, the open discs form a basis for the metric topology of X. We have the following well-known result.

Theorem 0.2.4

1. (X) is a topological space.

2. U if and only if for every x ∈ U, there exists r > 0 such that Br(x) ⊂ U.

Examples

Metric spaces form in many ways the most intuitive collection of topological spaces. For example, the surface of the earth or of a ball is a 2-sphere, denoted by S², and a metric may be defined by taking the shortest distance between two points measured along the surface of the sphere. Intuitively, we measure distance between two points X and Y on the earth’s surface as the length of a piece of string stretched taut between the two points X and Y. That is, distance is measured along great circles.

In this context the triangle equality merely states that the distance from London to New York is less than or equal to the distance from London to New York via, say, Madrid.

Another example of a metric space is given by the Cartesian product of two circles, S¹ × S¹. This space is called a torus (Fig. 0.1). In this case a metric can be defined by drawing a string taut between points on the surface subject to the constraint that the string is forced to lie on the surface.

FIG. 0.1

Other examples of metric spaces are given by Rn, n 1, with the Euclidean distance defined by

See also Section 1.1.

In practice many of the functions studied in, say, mathematical physics are defined on metric spaces like S² or S¹ × S¹ rather than on open subsets of Rn.

Next we wish to recall the definition and some of the elementary properties of continuous functions.

Definition 0.2.5

Let (X) and (Y) be topological spaces and f : X Y. f is said to be continuous if

f    for all V .

This definition may be given in many equivalent forms. In particular, f is continuous if and only if for every x ∈ X and every open subset V of Y containing f(x) there exists an open neighbourhood U of x such that f(U) ⊂ V.

If (X, d) and (Y, ρ) are metric spaces, then continuity may be characterized by any of the following equivalent conditions

1. f−1 (Br(y)) is open for every open disc Br(y) ⊂ Y.

2. f⊂ Y.

3. Given any open disc of the form Br(f(x)) contained in Y, there exists a disc B (x) ⊂ X such that f(B (x)) ⊂ Br(f(x)).

4. Let x ∈ X. Then for every r > 0 such that d(f(x), f(x′)) < r for all x′ ∈ X satisfying d(x, x.

Let x, …, xn, … be a sequence of points of Xor, more simply, just as (xn).

Definition 0.2.6

Let (xn) be a sequence of points of the metric space (X, d). The sequence (xn) is said to converge to the point x ∈ X if d(xn, x0 as n ∞.

Definition 0.2.7

The sequence (xn) of points of the metric space (X, d) is said to be a Cauchy sequence if d(xn, xm0 as n, m ∞.

Remark. In general a Cauchy sequence need not converge. See the exercises at the end of this section.

Definition 0.2.8

The metric space (X, d) is said to be complete if every Cauchy sequence converges.

Returning to the study of topological spaces we have the following important definition.

Definition 0.2.9

X is said to be a compact topological space if, for every collection {Ui}i ∈ I of open subsets that cover X (Ui ∈ I Ui = X), it is possible to find a finite subcollection {Ui}i ∈ I′ (I′ ⊂ I and I′ finite) which still covers X.

Examples

1. Every closed and bounded subset of Rn is compact. For example, S² and S¹ × S¹ are compact. Conversely every compact subset of Rn is closed and bounded.

2. Rn is not compact.

Definition 0.2.10

A topological space is said to be locally compact if every point has a neighbourhood with compact closure.

Examples

1. Every compact topological space is locally compact.

2. Rn is locally compact.

Exercises

1. Show that an open subset of R provides an example of an incomplete metric space.

2. Let (X, d) be a metric space. Define an equivalence relation on the set of all Cauchy sequences of points of X as follows: Say the two Cauchy sequences (xn) and (yn) are equivalent if d(xn, yn0 as n ∞. Show

(a) If (X, d) is complete, then (xn) and (yn) are equivalent if and only if they are convergent to the same point of X.

(b) The above defined relation on Cauchy sequences is-an equivalence relation.

naturally contains X and the metric d on X) is called the completion of (X, d).

) is complete.

3. Show that a closed subset of a compact topological space is compact. Show that a product of n closed bounded intervals is a compact subset of Rn (Assume the Bolzano-Weierstrass theorem). Hence show that every closed and bounded subset of Rn is compact.

CHAPTER 1

Linear algebra and normed vector spaces

1.1

Normed vector spaces

In this chapter K will always denote either the field of real numbers, R, or the field of complex numbers, C. |   | will denote the absolute value or modulus of numbers in K.

Let E be a vector space over K of not necessarily finite dimension. As it stands, E is too general to admit of interesting study. Thus, without any topology on E, questions like ‘Is vector space addition a continuous operation?’ or ‘Are linear maps continuous?’ are meaningless. We therefore wish to start by studying vector spaces with some additional structure such as a topology or metric. One such structure that the reader will have already encountered in the study of the vector spaces R, R², R³ and C is the important notation of the length of a vector. The generalization of length to an arbitrary vector space is given by the following definition.

Definition 1.1.1

Let E be a vector space over the field K. A function

is called a norm on E if

x = 0 if and only if x = 0

for all x, y ∈ E (‘triangle inequality’)

3. for all x ∈ E and k ∈ K.

The pair (E) is then called a normed vector space.

A norm, then, is our generalization of length. The following proposition shows that the norm of a vector is, like length, always positive.

Proposition 1.1.2

Let (Ex 0 for all x ∈ E.

Proof.

Associated with a norm on a vector space E we may define a translation-invariant metric on E.

Proposition 1.1.3

Let (E) be a normed vector space. Define d(x, yx − y , x, y ∈ E. Then d is a metric on E’, and d satisfies the following additional properties:

1. d(x + z, y + z) = d(x, y) for all x, y, z ∈ E (d is ‘translation invariant’).

2. d(kx, ky) = | k | d(x y) for all x, y ∈ E and k ∈ K.

Proof.

The translation invariance of d is shown by x − y .

FIG. 1.1

Examples

1. K is a normed vector space with norm defined by taking absolute value. Thus, if k ∈ Kk = | k |. In the sequel we shall always take this norm on K.

2. Let Rn = {(x1, …, xn): x1, …, xn ∈ R} (real n-space) and Cn = {(z1, …, zn): z1, …, zn ∈ C} (complex n-space).

We can define many different norms on Rn and Cn.

(c) More generally, for p 1, we may define the p-norm on Rn by

All the above norms may be defined on Cn is known as Minkowski’s inequality. We leave the proof of this important inequality to the exercises at the end of this section. We have, therefore, constructed infinitely many different metrics on Rn (at least for n 2). However, we shall show later that all these metrics have the same associated topology.

3. Let X denote an arbitrary set and B(X) denote the set of all bounded real valued functions on X. That is, f ∈ B(X) if and only if ∃ M 0 such that | f(xM for all x ∈ X. We can easily give B(X) the structure of a real vector space if we define vector space addition and scalar multiplication respectively by

. Observe that

satisfies the triangle inequality. The other axioms for a norm are easily verified and it follows that (B(Xis called the sup norm on B(X) and we shall have occasion to refer to it in the sequel. We may similarly define (BC(X), the (complex) normed vector space of bounded complex valued functions on X.

4. Injecting some topology into Example 2, let us suppose that X is a compact topological space. We let C(X) denote the set of continuous real-valued functions on X. Then, by a standard result of point-set topology, C(X) is a subset of B(X). Standard properties of continuous functions show that C(X) is a vector subspace of B(X). Restricting the sup norm defined in Example 2 to C(X) it follows that (C(X) is a normed vector subspace of (B(X).

The metric associated with the sup norm on C(X) is given by

We call the topology associated with this metric on C(X), the topology of uniform convergence on C(X). The reader should check, in the special case when X is the compact interval [a, b], that if we are given a sequence of continuous functions (fn) on X, then fn f uniformly (classical definition) if and only if fn f in the topology of uniform convergence on C(X). See also the exercises at the end of this section. We may similarly define (CC(X), the (complex) normed vector space of continuous complex-valued functions on X.

5. Many examples of normed vector spaces are given by sequence spaces. In what follows, N+ denotes the set of strictly positive integers.

(a) Let m if and only if ∃ M 0 such that | xn M for all n ∈ N+. We may give m the structure of a real vector space by defining addition and scalar multiplication coordinatewise. For example, if (xi), (yi) ∈ m, then (xi) + (yi) is the sequence

We define a norm on m by setting

We leave it as an exercise for the reader to verify that (m) is a real normed vector space. We may similarly define the space of bounded complex sequences which again is a normed vector space with obvious norm.

Let us look at some subspaces of m.

(b) Let c denote the set of convergent sequences of real numbers and c0 denote the set of sequences convergent to 0. We leave it to the reader to verify that c and c0 are normed vector subspaces of m.

(c) Let s denote the subset of m if and only if ∃ p ∈ N+, such that xn = 0, n p. It is easily seen that s is a normed vector subspace of m. It is also a subspace of c0.

(d) We may define many other vector subspaces of m with, however, different norms from that induced by m. For example, let lif and only if

Since (1, 1, …, 1, …) ∉ l¹, l¹ ≠ m. It is not difficult to show that l¹ is a vector subspace of m defines a norm on l.

More generally, let lp satisfying the condition

where p is greater than or equal to 1.

However, now we need Minkowski’s inequality even to verify that lp is a vector space. We refer the reader to the exercises for this verification and also the fact that (lp) is a normed vector space. Unlike the case of Rnare all different, and indeed, regarded as vector subspaces of m, lp = lp′ if and only if p = p′.

Exercises

1. Prove Proposition 1.1.3.

on R², interpret geometrically the triangle inequality.

3. Find all norms on the real line R and the complex line C (regarded as a complex vector space).

4. In this exercise we indicate the main steps of proof of Minkowski’s inequality.

(a) Consider the function f(x, y) = αx y − xαy where 0 < α, < 1, α + = 1 and x and y are assumed positive. By finding the minimum value of f for a fixed value of y, show that

αx y xαy    for all x, y 0.

(b) Let (an) and (bn) be real (or complex) sequences consisting of N terms. Let r, r′ > 1 and satisfy the condition

m N

Thus we have

From the inequality proved in step (a), it follows that

and hence by summation over m that

Hence deduce Hölder’s inequality:

(c) With the assumptions of step (b) show that

and apply Hölder’s inequality to the sums on the right to deduce Minkowski’s inequality:

defined on Rn in this section are indeed norms.

5. By extending Minkowski’s inequality to infinite sequences, prove that (lp) is a normed vector space.

for p less than 1?

7. Consider the set of all continuous functions on the closed unit interval [0, 1]. Define

Prove that (C) is a normed vector space. How might one define (C), p > 1?

8. Consider the vector space C[0, 1] together with the sup norm and let d denote the associated metric. Suppose that (fn) is a Cauchy sequence in C[0, 1]. Prove (using any standard results about uniformly convergent sequences of functions you may need) that (fn) converges to an element