Analysis and Probability

By Aurel Spataru

Probability theory is a rapidly expanding field and is used in many areas of science and technology. Beginning from a basis of abstract analysis, this mathematics book develops the knowledge needed for advanced students to develop a complex understanding of probability. The first part of the book systematically presents concepts and results from analysis before embarking on the study of probability theory. The initial section will also be useful for those interested in topology, measure theory, real analysis and functional analysis. The second part of the book presents the concepts, methodology and fundamental results of probability theory. Exercises are included throughout the text, not just at the end, to teach each concept fully as it is explained, including presentations of interesting extensions of the theory. The complete and detailed nature of the book makes it ideal as a reference book or for self-study in probability and related fields.

• Covers a wide range of subjects including f-expansions, Fuk-Nagaev inequalities and Markov triples.
• Provides multiple clearly worked exercises with complete proofs.
• Guides readers through examples so they can understand and write research papers independently.

• Language: English
• Publisher: Elsevier Science
• Release date: Jan 12, 2013
• ISBN: 9780124017276

Book preview

Analysis and Probability

taru

Romanian Academy

Bucharest

Romania

Table of Contents

Cover image

Title page

Copyright

Preface

PART 1: ANALYSIS

Chapter 1. Elements of Set Theory

1 Sets and Operations on Sets

2 Functions and Cartesian Products

3 Equivalent Relations and Partial Orderings

References

Chapter 2. Topological Preliminaries

4 Construction of Some Topological Spaces

5 General Properties of Topological Spaces

6 Metric Spaces

Chapter 3. Measure Spaces

7 Measurable Spaces

8 Measurable Functions

9 Definitions and Properties of the Measure

10 Extending Certain Measures

Chapter 4. The Integral

11 Definitions and Properties of the Integral

12 Radon-Nikodým Theorem and the Lebesgue Decomposition

13 The Spaces

14 Convergence for Sequences of Measurable Functions

Chapter 5. Measures on Product σ-Algebras

15 The Product of a Finite Number of Measures

16 The Product of Infinitely Many Measures

PART 2: PROBABILITY

Chapter 6. Elementary Notions in Probability Theory

17 Events and Random Variables

18 Conditioning and Independence

Chapter 7. Distribution Functions and Characteristic Functions

19 Distribution Functions

20 Characteristic Functions

References

Chapter 8. Probabilities on Metric Spaces

21 Probabilities in a Metric Space

22 Topology in the Space of Probabilities

Chapter 9. Central Limit Problem

23 Infinitely Divisible Distribution/Characteristic Functions

24 Convergence to an Infinitely Divisible Distribution/Characteristic Function

Reference

Chapter 10. Sums of Independent Random Variables

25 Weak Laws of Large Numbers

26 Series of Independent Random Variables

27 Strong Laws of Large Numbers

28 Laws of the Iterated Logarithm

Chapter 11. Conditioning

29 Conditional Expectations, Conditional Probabilities and Conditional Independence

30 Stopping Times and Semimartingales

Chapter 12. Ergodicity, Mixing, and Stationarity

31 Ergodicity and Mixing

32 Stationary Sequences

List of Symbols

Copyright

Elsevier

32, Jamestown Road, London NW1 7BY, UK

225 Wyman Street, Waltham, MA 02451, USA

First Edition 2013

Copyright © 2013 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods or professional practices, may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information or methods described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

Application submitted

ISBN: 978-0-12-401665-1

For information on all Elsevier publications visit our website at store.elsevier.com

This book has been manufactured using Print On Demand technology. Each copy is produced to order and is limited to black ink. The online version of this book will show color figures where appropriate.

Preface

First and foremost, this book is designed as a probability theory text for graduate students and as a reference work in this field. The reader will require a solid background in abstract analysis in order to tackle the development of probability theory from a measure-theoretic perspective, which is the main focus of the book. For this reason, Part One of the book – Chapters 1 to 5 – systematically presents concepts and results from analysis with which every reader should be familiar before embarking on the study of probability theory; this part should also be useful for those interested in topology, measure theory, real analysis and functional analysis. Part Two – Chapters 6 to 12 – then presents the concepts, methodology and fundamental results that a graduate-level student of probability theory would be expected to know. Probability theory is a rapidly expanding field and so the author has necessarily made some subjective judgments about which topics will be included and which ones omitted. For instance, the important notion of Markov dependence is not discussed here, though the subject is touched upon during the presentation of the concepts of the Markov triple and the Markov shift. Similarly, neither stochastic processes in continuous time nor multi parameter processes are considered in this book; indeed, each of these topics merits an entire monograph of its own. Nonetheless, it is worth noting that the book does contain material about subjects, such as f-expansions, Fuk-Nagaev inequalities and Markov triples, that typically do not appear in other graduate-level probability texts.

Extra care has been taken to ensure that the exposition is kept as self-contained as possible. Except for elementary analysis, there are no other prerequisites for reading the book. Consequently, the book should lend itself well to individual study. To facilitate understanding and to offer interesting extensions of the theory, every section ends with a large collection of exercises. Since some of the exercises will be needed for subsequent definitions and proofs in the main text, the reader is encouraged to work carefully through the exercises.

A brief description of each of the twelve chapters may be useful at this point. Chapter 1 contains a review of some essential concepts and results from set theory upon which the development of probability theory in later chapters will be based. Section 1 deals mainly with operations on sets, Section 2 presents several types of functions and the concept of a Cartesian product, and Section 3 introduces the concepts of equivalence relation and partial ordering. Special attention is paid to the classification of sets according to the number of elements they contain. Some equivalent forms of the axiom of choice are presented at the end of the chapter. It should be noted that the treatment of set-theoretic notions in Chapter 1 is not axiomatic; rather the presentation relies on intuition and elementary logic, and does so without introducing any sets that could lead to paradoxes.

Chapter 2 is motivated by the prominent role that topology plays in measure theory – and, hence, in probability theory which, to some extent, may be viewed as a branch of measure theory. In this chapter we introduce and discuss most of the topological concepts that will be used in the book. In particular, Section 4 deals with various methods of constructing topological spaces, while Section 5 and Section 6 present important properties of topological spaces that will be used in subsequent chapters. In particular, Section 6 includes some deep results, such as Urysohn’s embedding theorem, the Stone-Weierstrass theorem and the Arzelà-Ascoli theorem.

Chapter 3 is devoted to the study of three fundamental notions: measurable space, measurable function and measure. While the definitions of measurable space and measurable function do not inherently depend on the existence of a measure, measurable spaces and measurable functions, in practice, are almost always associated with a measure; this justifies the term measurable in their names. In Section 7 we presenta detailed study of algebras and σ-algebras and introduce a special family of sets called a Dynkin system. In most situations, a Dynkin system can replace the notion of a monotone class (which is defined in various books on probability and measure theory) and proves to be a highly efficient tool. In Section 8 and Section 9 we discuss measurable functions and measures and then, in Section 10, we develop the technique of extending σ-finite measures, culminating in Carathéodory’s extension theorem and the construction of Lebesgue-Stieltjes measures.

The material in .

Chapter 5 is devoted to the construction of measures on product σ-algebras. In Section 15 we construct the product of a finite number of σ-finite measures. This is extended to infinite products of probability measures in Section 16, which also presents the Daniell-Kolmogorov extension theorem.

In Chapter 6 we introduce the first notions of probability theory. In Section 17 we present the primary probabilistic concepts of events, random variables, random vectors, random elements, moments and variances. Section 18 begins with a presentation of elementary properties of conditional probabilities, and proceeds to consider the concept of independence for σ-algebras and for random variables and elements, and to establish such basic results as the Kolmogorov zero-one law, the Borel-Cantelli lemma and the Hewitt-Savage zero-one law.

Chapter 7 is devoted to two important tools in the study of random variables, namely, the distribution function and the characteristic function. Section 19 deals with distribution functions and sequences of distribution functions for both random variables and random vectors. The convolution of two distribution functions, the concept of weak convergence of a sequence of distribution functions and the related concept of complete convergence are studied in this section, which also includes a discussion of various types of distribution functions and the presentation of the important Helly-Bray theorem. The main purpose of Section 20 is an exposition of characteristic functions and their properties and how they may be used to establish results in probability theory. One such result is the continuity theorem, due to P. Lévy, which replaces the convergence of a sequence of distribution functions by the convergence of the corresponding sequence of characteristic functions, the latter generally being much more tractable than the former.

Chapter 8 puts a focus on the theory of probability on abstract metric spaces. The connection between probability theory and topology is illustrated in the first part of Section 21 and culminates in Lusin’s theorem on the approximation of measurable functions by continuous functions. This section also presents an examination of the relationship between nonnegative linear functionals and probabilities, including a version of the famous Riesz representation theorem. In Section 22 we topologize the space of all probabilities on the σ-algebra of Borel subsets of a metric space and then investigate convergence properties of sequences of probabilities. Further, we proceed to analyze the question of metrization of this topology and derive a well-known result of Yu. V. Prohorov characterizing the compact subsets of probabilities.

By making use of powerful tools and techniques, such as characteristic functions and symmetrization, we present in Chapter 9 the crowning achievement of classical probability theory: the central limit theorem. Given a triangular array of row-wise independent random variables, the central limit problem seeks conditions under which the distribution functions of the row sums converge weakly to some distribution function. It turns out that the totality of such limit distribution functions forms a special set called the class of infinitely divisible distribution functions. Section 23 is devoted to a detailed study of this class and introduces the notions of self-decomposable and stable distribution functions. In Section 24 we present necessary and sufficient conditions for the weak convergence of the row sums to a given infinitely divisible distribution function, and we apply this convergence criterion to three important limit distribution functions: the degenerate, the Poisson and the normal distribution function.

The behaviour of sums of independent random variables is the main concern of Chapter 10. Making use of results from the preceding chapter, we focus in Section 25 on the weak convergence of row sums to zero, which leads to the weak law of large numbers. In Section 26 we investigate the almost sure convergence of series of independent random variables, culminating in the Kolmogorov three-series theorem and the two-series theorem. Section 27 is devoted to obtaining various strong laws of largenumbers, i.e theorems establishing conditions under which stabilized sums of independent random variables converge almost surely to zero. Further, in Section 28, we investigate the order of magnitude of the fluctuations of sums of independent random variables by establishing two famous results of classical probability theory, namely, the Kolmogorov law of the iterated logarithm and the Hartman-Wintner law of the iterated logarithm.

While Chapter 10 is devoted to the limiting behaviour of sums of independent random variables, Chapter 11 considers the more general setting in which a sequence of random variables is conditioned with respect to a sequence of σ-algebras. Such sequences are important in Markov theory, semimartingale theory and ergodic theory. The first part of Section 29 introduces the concepts of conditional expectation and conditional probability with respect to a given σ-algebra and develops their most important properties. This section concludes with discussion about the notions of conditional independence and Markov triple. In Section 30 we apply results from the preceding section to the theory of martingales and semimartingales in discrete time. After an exposition of the basic theory of these processes, starting with the concept of stopping time, we proceed to establish a number of powerful results, including Doob’s optional sampling theorem, supremum inequalities, the submartingale convergence theorem, an extension of the Borel-Cantelli lemma to conditional probabilities, and Wald’s equations.

Classical ergodic theorems for measure-preserving transformation and their role in the theory of stationary sequences are the focus of Chapter 12. Section 31 formulates and proves the fundamental ergodic theorems due to G. D. Birkhoff and J. von Neumann. To this end, we first introduce and study such basic concepts as measure-preserving transformations, invariant σ-algebras, invariant random variables and ergodic trans-formations. This section concludes with some results concerning the related notions of weak-mixing and strong-mixing. In Section 32 we present some useful results for general stationary sequences of random elements, which are sequences that encompass a dependence structure of great importance in probability theory.

u for many fruitful discussions and helpful suggestions. I thank the IMSAM for providing such a friendly and stimulating environment and for giving me the freedom and opportunity to complete this work.

Jim Tomkins kindly improved the presentation of an essential part of the text, and I warmly thank him for his help in this matter. Ido wish to thank the following reviewers: George Roussas, University of California at Davis; Mark Pinsky, Northwestern University; and Peter Spreij, University of Amsterdam. Finally, I am grateful to the editors for accepting this project in the Elsevier Insights, and to the Elsevier production team for their kind and efficient cooperation in publishing.

Romanian Academy, Bucharest, Romania

March 2012

taru

PART 1

ANALYSIS

Chapter 1 Elements of Set Theory

Chapter 2 Topological Preliminaries

Chapter 3 Measure Spaces

Chapter 4 The Integral

Chapter 5 Measures on Product σ-Algebras

Chapter 1

Elements of Set Theory

This chapter contains a review of some essential concepts and results of set theory on which lean the following chapters. Section 1 deals mainly with operations on sets. In Section 2 we define several types of functions, and discuss the notion of a Cartesian product. In Section 3 we study the concepts of equivalent relation and partial ordering. A special attention is paid to the classification of sets according to the number of their elements. At the end of this section we state some equivalent forms of the axiom of choice.

The treatment of the set-theoretic notions is not axiomatic, but relies on intuition and elementary logic. However, this will not introduce sets which lead to paradoxes.

1 Sets and Operations on Sets

The notion of a set and some notations 1.1

.

. We assume a knowledge on the part of the reader of the ordering structure and the algebraic structure of these sets. The reader is also assumed to know elementary analysis concerning these sets. Another important set will be now introduced.

Definition 1.2

is called the set of extended real numbersis defined by

.

Definitions 1.3

is a subset of are equal sets.

.

is called the power set of .

Definitions 1.4

be sets. The union of and

The intersection of and are said to be disjoint. The difference . The symmetric difference of and .

Definition 1.5

is called the complement of .

Theorem 1.6

Let be sets. Then we have:

;

(ii) 

;

(iii) 

.

Proof

.

Exercise 1.7

sets.

Exercise 1.8

prove:

;

;

;

;

;

;

;

(h) 

;

;

(j) 

;

;

;

.

2 Functions and Cartesian Products

In the first part of this section we introduce the concept of a function, and we define several types of functions. Many other classes of functions will be studied subsequently. By making use of the axiom of choice, in the last part of the present section we discuss the notion of a Cartesian product.

Definitions 2.1

be sets. A function [mapping, transformationis called the image of under [value of at is called the domain of is called the range of .

are equal functionsin this domain.

Definitions 2.2

.

. If

.

is called the constant function with value wherever necessary.

Definition 2.3

is called the restriction of (to Xis called an extension of .

Definition 2.4

is called the composition of and .

Definitions 2.5

is called the image of under is called the inverse image of under .

be a family of sets. The inverse image of under .

Theorem 2.6

Let and be functions, and let . Then .

Proof

.

Definitions 2.7

is surjective onto is said to be injectiveis both surjective and injective, then it is said to be bijective, is called the inverse of .

Example 2.8

is called the inclusion function is called the identity function.

which is called the family on induced by . In several occasions we will encounter induced families in this text (see (4.16) and (7.12)).

Definition 2.9

given by

is called the indicator function of . This fact allows replacing operations on subsets by equivalent operations on indicator functions, and so makes the indicator function an extremely useful tool in measure theory.

The concept of an indexed set we introduce now permits, among other things, to extend the notions of a union and of an intersection of two sets defined in (1.4).

Definition 2.10

is called an indexed set .

Example 2.11

is called a sequence.

, is called a subsequence .

Notation 2.12

."

Examples 2.13

.)

, and the indexed set to which ()

, and the indexed set to which

Definitions 2.14

is called the union of all sets of is called the intersection of all sets of , respectively.

is called the union of all sets of is called the intersection of all sets of .

Definition 2.15

A partition of such that:

;

.¹

Definitions 2.16

is called the inferior limit of is called the superior limit of .

De Morgan’s laws 2.17

, then:

;

.

The proof of these identities is easy and is left to the reader.

Remark 2.18

. To be in accordance with .

Theorem 2.19

Let be a function, let and . Then we have:

;

;

.

The proof of this theorem is easy and is left to the reader.

In what follows we want to introduce the notion of a Cartesian product of a nonempty family of sets. To do this, we will first enunciate an important axiom of set theory which has many applications in mathematics.

Axiom of choice 2.20

, called a choice function. Thus this axiom asserts that one can choose simultaneously an element from each nonempty set of a nonempty family of sets. The axiom of choice is consistent with the other axioms of set theory. In this form, or in other equivalent forms (see (3.29)–(3.33), we will use it throughout whenever convenient.

Definitions 2.21

be an indexed family of sets. The Cartesian product of the sets of .

is called the i-th coordinate of f.

Example 2.22

, and

.

Definition 2.23

is called the projection from onto .

Theorem 2.24

Let be a Cartesian product of sets. Then implies .

Proof

, we have

Definitions 2.25

, be functions. The Cartesian product of the functions of , are called the coordinate of .

Example 2.26

.

Theorem 2.27

Let be a Cartesian product of sets, and let be a partition of such that . Then the function is bijective.

Proof

is surjective and injective.

Remarks 2.28

(a) . Comments on the associativity of Cartesian products of sets can be found in Measure Theory (Halmos, 1974).

(b) Notation is as in (.

Definitions 2.29

, the set

is called the section of at .

be a function. The section of at .

Examples 2.30

(a) Let

, we have

.

be as in , we have

Exercise 2.31

.

Exercise 2.32

.

Exercise 2.33

be sets. Prove:

;

;

.

Exercise 2.34

.

Exercise 2.35

Prove:

;

;

;

;

;

.

Exercise 2.36

.

Exercise 2.37

.

Exercise 2.38

.

Exercise 2.39

.

Exercise 2.40

Prove:

;

(b) 

.

Exercise 2.41

be a family of sets. Prove:

;

;

;

;

;

;

;

.

Exercise 2.42

.

Exercise 2.43

.

Exercise 2.44

.

Exercise 2.45

be as in . Prove:

;

;

;

;

;

.

Exercise 2.46

.

Exercise 2.47

. Prove the following.

.

.

.

Exercise 2.48

.

.

, and that this inclusion may be strict.

3 Equivalent Relations and Partial Orderings

are sets, then in agreement with .

Definitions 3.1

is called a relationis called the inverse of .

Example 3.2

is called the graph of . Many authors identify a function with its graph.

Definitions 3.3

is said to be reflexive is said to be symmetricis said to be antisymmetric is said to be transitive.

Examples 3.4

has a single element.

be the diagonal .