Stochastic Processes

By Emanuel Parzen

Well-written and accessible, this classic introduction to stochastic processes and related mathematics is appropriate for advanced undergraduate students of mathematics with a knowledge of calculus and continuous probability theory. The treatment offers examples of the wide variety of empirical phenomena for which stochastic processes provide mathematical models, and it develops the methods of probability model-building.
Chapter 1 presents precise definitions of the notions of a random variable and a stochastic process and introduces the Wiener and Poisson processes. Subsequent chapters examine conditional probability and conditional expectation, normal processes and covariance stationary processes, and counting processes and Poisson processes. The text concludes with explorations of renewal counting processes, Markov chains, random walks, and birth and death processes, including examples of the wide variety of phenomena to which these stochastic processes may be applied. Numerous examples and exercises complement every section.

• Language: English
• Publisher: Dover Publications
• Release date: May 5, 2015
• ISBN: 9780486804712

Book preview

INDEX

INTRODUCTION

Role of the theory of stochastic processes

THE SCIENTIST making measurements in his laboratory, the meteorologist attempting to forecast weather, the control systems engineer designing a servomechanism (such as an aircraft autopilot or a thermostatic control), the electrical engineer designing a communications system (such as the radio link between entertainer and audience or the apparatus and cables that transmit messages from one point to another), the economist studying price fluctuations and business cycles, and the neurosurgeon studying brain-wave records—all are encountering problems to which the theory of stochastic processes may be relevant. Before beginning our study of the theory of stochastic processes, let us show why this theory is an essential part of such diverse fields as statistical physics, the theory of population growth, communication and control theory, management science (operations research), and time series analysis. For a survey of other applications of the theory of stochastic processes, especially in astronomy, biology, industry, and medicine, see Bartlett (1962)† and Neyman and Scott (1959).

STATISTICAL PHYSICS

Many parts of the theory of stochastic processes were first developed in connection with the study of fluctuations and noise in physical systems (Einstein [1905], Smoluchowski [1906], Schottky [1918]). Consequently, the theory of stochastic processes can be regarded as the mathematical foundation of statistical physics.

Stochastic processes provide models for physical phenomena such as thermal noise in electric circuits and the Brownian motion of a particle immersed in a liquid or gas.

Brownian motion. When a particle of microscopic size is immersed in a fluid, it is subjected to a great number of random independent impulses owing to collisions with molecules. The resulting vector function (X(t), Y(t), Z(t)) representing the position of the particle as a function of time is known as Brownian motion.

Thermal noise. Consider a resistor in an electric network. Because of the random motions of the conduction electrons in the resistor, there will occur small random fluctuations in the voltage X(t) across the ends of the resistor. The fluctuating voltage X(t) is called thermal noise (its probability law may be shown to depend only on the resistance R and the absolute temperature T of the resistor).

Shot noise. Consider a vacuum diode connected to a resistor. Because the emission of electrons from the heated cathode is not steady, a current X(t) is generated across the resistance which consists of a series of short pulses, each pulse corresponding to the passage of an electron from cathode to anode. The fluctuating current X(t) is called shot noise.

STOCHASTIC MODELS FOR POPULATION GROWTH

The size and composition of a population (whether it consists of living organisms, atoms undergoing fission, or substances decaying radioactively) is constantly fluctuating. Stochastic processes provide a means of describing the mechanisms of such fluctuations (see Bailey [1957], Bartlett [1960], Bharucha-Reid [1960], Harris [1964]).

Some of the biological phenomena for which stochastic processes provide models are (i) the extinction of family surnames, (ii) the consequences of gene mutations and gene recombinations in the theory of evolution, (iii) the spatial distribution of plant and animal communities, (iv) the struggle for existence between two populations that interact or compete, (v) the spread of epidemics, and (vi) the phenomenon of carcinogenesis.

COMMUNICATION AND CONTROL

A wide variety of problems involving communication and/or control (including such diverse problems as the automatic tracking of moving objects, the reception of radio signals in the presence of natural and artificial disturbances, the reproduction of sound and images, the design of guidance systems, the design of control systems for industrial processes, forecasting, the analysis of economic fluctuations, and the analysis of any kind of record representing observation over time) may be regarded as special cases of the following general problem:

Let T denote a set of points on a time axis such that at each point t in T an observation has been made of a random variable X(t). Given the observations {X(t), t T}, and a quantity Z related to the observation in a manner to be specified, one desires to form in an optimum manner estimates of, and tests of hypotheses about, Z and various functions h(Z). This imprecisely formulated problem provides the general context in which to pose the following usual problems of communication and control.

Prediction (or extrapolation). Having observed the stochastic process X(t) over the interval s − L ≤ t ≤ s, one wishes to predict X(s + α) for any α > 0. The interval L of observation may be finite or infinite.

Smoothing. Suppose that the observations

may be regarded as the sum, X(t) = S(t) + N(t), of two stochastic processes (or time series) S( · ) and N( · ), representing signal and noise, respectively. One desires to estimate the value S(t) of the signal at any time t in s − L ≤ t ≤ s. The terminology smoothing derives from the fact that often the noise N( · ) consists of very high frequency components compared with the signal S( · ); estimating or extracting the signal S( · ) can then be regarded as an attempt to pass a smooth curve through a very wiggly record. The problem of predicting S(s + α) for any α > 0 is called the problem of smoothing and prediction.

Parameter estimation (signal extraction and detection). Suppose that the observations {X(t), 0 ≤ t ≤ T} may be regarded as a sum, X(t) = S(t) + N(t), where S( · ) represents the trajectory (given by S(t) = x0 + υt + (a/2)t², say) of a moving object, and N( · ) represents errors of measurement. One desires to estimate the velocity υ and acceleration a of the object. More generally, one desires to estimate such quantities as S(t) and (d/dt) S(t) at any time t in 0 ≤ t ≤ T, under the assumption that the signal S( · ) belongs to a known class of functions.

It is clear that the solutions one gives to the foregoing problems depend on the assumptions one makes about the signals and noise received, and also on the criterion one adopts for an optimum solution. It is well established (see Lanning and Battin [1956], Helstrom [1960], Middleton [I960]) that the design of optimum communications and control systems requires recognition of the fact that the signals and noises arising in such systems are best regarded as stochastic processes. Consequently, the first step in the study of modern communication and control systems theory is the study of the theory of stochastic processes. The purpose of such a study is:

(i) to provide a language in which one may state communications and control problems, since the evaluation of communications and control systems must necessarily be in terms of their average behavior over a range of circumstances described probabilistically;

(ii) to provide an insight into the most realistic assumptions to be made concerning the stochastic processes representing the signals and/or noises.

MANAGEMENT SCIENCE

Stochastic processes provide a method of quantitatively studying and managing business operations and, consequently, play an important role in the modern disciplines of management science and operations research. The two fields in which the theory of stochastic processes has found greatest application are inventory control and waiting-line analysis (see Arrow, Karlin, and Scarf [1958], Syski [I960]).

Inventory control. Two problems of considerable importance to such diverse organizations as retail shops, wholesale distributors, manufacturers, and consumers holding stocks of spare parts are (i) deciding when to place an order for replenishment of their stock of items, and (ii) deciding how large an order to place. Two kinds of uncertainty must be taken into account in making these decisions: (i) uncertainty concerning the number of items that will be demanded during a given time period; (ii) uncertainty concerning the time-of-delivery lag that will elapse between the date on which an order is placed and the date on which the items ordered will actually arrive. If it were not for these factors, one could perhaps order new stock in such a manner that it would arrive at the precise instant when it is needed. One would then not have to maintain inventory, which is often very expensive to do. Inventory control is concerned with minimizing the cost of maintaining inventories, while at the same time keeping a sufficient stock on hand to meet all contingencies arising from random demand and random time-of-delivery lag of new stock ordered. One approach to the problem of optimal inventory control is to consider inventory policies actually used and to describe the effects of these policies. Given a specific inventory policy, the resultant fluctuating inventory level is a stochastic process.

Queues. A queue (or waiting line) is generated when customers (or servers) arriving at some point to receive (or render) service there must wait to receive (or render) service. The group waiting to receive (or render) service, perhaps including those receiving (or rendering) service, is called the queue.

There are many examples of queues. Persons waiting in a railroad station or airport terminal to buy tickets constitute a queue. Planes landing at an airport make up a queue. Ships arriving at a port to load or unload cargo form a queue. Taxis waiting at a taxi stand for passengers are in a queue. Messages transmitted by cable constitute a queue. Mechanics in a plant form a queue at the counters of tool cribs where tools are stored. Machines on a production line which break down and are repaired are in queue for service by mechanics.

In the mathematical theory of queues, waiting lines are classified according to four aspects: (i) the input distribution (the probability law of the times between successive arrivals of customers) ; (ii) the service time distribution (the probability law of the time it takes to serve a customer) ; (iii) the number of service channels ; (iv) the queue discipline (the manner in which customers are selected to be served; possible policies are first come, first served, random selection for service, and service according to order of priority). Queueing theory is concerned with the effect that each of these four aspects has on various quantities of interest, such as the length of the queue and the waiting time of a customer for service.

TIME SERIES ANALYSIS

A set of observations arranged chronologically is called a time series. Time series are observed in connection with quite diverse phenomena and by a wide variety of researchers. Examples are (i) the economist observing yearly wheat prices, (ii) the geneticist observing daily egg production of a certain new breed of hen, (iii) the meteorologist studying daily rainfall in a given city, (iv) the physicist studying the ambient noise level at a given point in the ocean, (v) the aerodynamicist studying atmospheric turbulence gust velocities, (vi) the electronics engineer studying the internal noise of a radio receiver.

To represent a time series, one proceeds as follows. The set of time points at which measurements are made is called T. In many applications, T is a set of discrete equidistant time points (in which case one writes T = {1, 2, · · ·, N}, where N is the number of observations) or T is an interval of the real time axis (in which case one writes T = {0 ≤ t ≤ L}, where L is the length of the interval). The observation made at time t is denoted by X(t). The set of observations {X(t), t T} is called a time series.

In the development of the theory of stochastic processes, an important role has been played by the study of economic time series. Consider the prices of a commodity or corporate stocks traded on an exchange. The record of prices over time may be represented as a fluctuating function X(t). The analysis of such economic time series has been a problem of great interest to economic theorists desiring to explain the dynamics of economic systems and to speculators desiring to forecast prices.

The basic idea (see Wold [1938], Bartlett [1955], Grenander and Rosenblatt [1957], Hannan [I960]) of the statistical theory of analysis of a time series {X(t), t T} is to regard the time series as an observation made on a family of random variables {X(t), t T} ; that is, for each t in T, the observation X(t) is an observed value of a random variable. A family of random variables {X(t), t T} is called a stochastic process. Having made the assumption that the observed time series {X(t), t T} is an observation (or, in a different terminology, a realization) of a stochastic process {X(t), t T}, the statistical theory of time series analysis attempts to infer from the observed time series the probability law of the stochastic process. The method by which it treats this problem is similar in spirit (although it requires a more complicated analytic technique) to the method by which classical statistical theory treats the problem of inferring the probability law of a random variable X for which one has a finite number of independent observations X1, X2, · · · , Xn.

In order to analyze a time series {X(t), t T}, one must first assume a model for {X(t), t T}, which is completely specified except for the values of certain parameters which one proceeds to estimate on the basis of an observed sample. Consequently, the first step in the study of time series analysis is the study of the theory of stochastic processes. The purpose of such a study is

(i) to provide a language in which assumptions may be stated about observed time series ;

(ii) to provide an insight into the most realistic and/or mathematically tractable assumptions to be made concerning the stochastic processes that are adopted as models for time series.

---

†References are listed in detail at the rear of the book.

CHAPTER

1

Random variables and stochastic processes

PROBABILITY THEORY is regarded in this book as the study of mathematical models of random phenomena. A random phenomenon is defined as an empirical phenomenon that obeys probabilistic, rather than deterministic, laws.

A random phenomenon that arises through a process (for example, the motion of a particle in Brownian motion, the growth of a population such as a bacterial colony, the fluctuating current in an electric circuit owing to thermal noise or shot noise, or the fluctuating output of gasoline in successive runs of an oil-refining mechanism) which is developing in time in a manner controlled by probabilistic laws is called a stochastic process.†

For reasons indicated in the Introduction, from the point of view of the mathematical theory of probability a stochastic process is best defined as a collection {X(t), t T} of random variables. (The Greek letter e is read belongs to or varying in.) The set T is called the index set of the process. No restriction is placed on the nature of T. However, two important cases are when T = {0, ± 1, ± 2, · · · } or T = {0, 1, 2, · · · }, in which case the stochastic process is said to be a discrete parameter process, or when T = {t: − ∞ < t < ∞} or T = {t: ≥ 0}, in which case the stochastic process is said to be a continuous parameter process.

This chapter discusses the precise definition of random variables and stochastic processes that will be employed in this book. It also introduces two stochastic processes, the Wiener process and the Poisson process, that play a central role in the theory of stochastic processes.

1-1RANDOM VARIABLES AND PROBABILITY LAWS

Intuitively, a random variable X is a real-valued quantity which has the property that for every set B of real numbers there exists a probability, denoted by P[X is in B], that X is a member of B. Thus X is a variable whose values are taken randomly (that is, in accord with a probability distribution). In the theory of probability, a random variable is defined as a function on a sample description space. By employing such a definition, one is able to develop a calculus of random variables studying the characteristics of random variables generated, by means of various analytic operations, from other random variables.†

In order to give a formal definition of the notion of a random variable we must first introduce the notions of

(i) a sample description space,

(ii) an event,

(iii) a probability function.

The sample description space S of a random phenomenon is the space of descriptions of all possible outcomes of the phenomenon.

An event is a set of sample descriptions. An event E is said to occur if and only if the observed outcome of the random phenomenon has a sample description in E.

It should be noted that, for technical reasons, one does not usually permit all subsets of S of subsets of S which has the following properties:

(i) S .

belongs the complement Ec of any set E .

of any sequence of sets E1, E.

Note that the family of all subsets of S possesses properties (i)–(iii). However, this family is often inconveniently large (in the sense that for certain sample description spaces S it is impossible to define on the family of all subsets of S a probability function P[ · ] satisfying axiom 3 below). For the development of the mathematical theory of probability, it often suffices to take as the family of events the smallest family of subsets of S possessing properties (i)–(iii) and also containing as members all the sets in which we expect to be interested. Thus, for example, in the case in which the sample description space S is defined as the smallest family of sets of real numbers that possesses properties (i)–(iii) and, in addition, contains as members all intervals. (An interval is a set of real numbers of the form {x: a < x < b}, {x: a < x ≤ b}, {x: a ≤ x ≤ b}, {x: a ≤ x < b}, in which a and b may be finite or infinite numbers.)

To complete the mathematical description of a random phenomenon, one next specifies a probability function Pof random events; more precisely, one defines for each event E a number, denoted by P[E] and called the probability of E (or the probability that E will occur). Intuitively, P[E] represents the probability that (or relative frequency with which) an observed outcome of the random phenomenon is a member of E.

Regarded as a function on events, P[ · ] is assumed to satisfy three axioms:

Axiom 1. P[E] ≥ 0 for every event E.

Axiom 2. P[S] = 1 for the certain event S.

Axiom 3. For any sequence of events E1, E2, · · ·, En, · · · which are mutually exclusive (that is, events satisfying the condition that, for any two distinct indices j and kdenotes the impossible event or empty set),

In applied probability theory, sample description spaces are not explicitly employed. Rather, most problems are treated in terms of random variables.

An object X is said to be a random variable if (i) it is a real finite valued function defined on a sample description space S of events a probability function P[ · ] has been defined, and (ii) for every Borel set B of real numbers the set {s: X(s) is in B.

The probability function of a random variable X, denoted by ΡX[ · ], is a function defined for every Borel set B of real numbers by

In words, ΡX[Β] is the probability that an observed value of X will be in B.

Two random variables X and Y are said to be identically distributed if their probability functions are equal; that is, if PX[B] = PY[B] for all Borel sets B.

The probability law of a random variable X is defined as a probability function P[ · ] that coincides with the probability function PX[ · ] of the random variable X. By definition, probability theory is concerned with the statements that can be made about a random variable, when only its probability law is known. Consequently, to describe a random variable, one need only state its probability law.

The probability law of a random variable can always be specified by stating the distribution function FX( · ) of the random variable X, defined for any real number x by

A random variable X is called discrete if there exists a function, called the probability mass function of X and denoted by pX( · ), in terms of which the probability function PX[ · ] may be expressed as a sum; for any Borel set B,

It then follows that, for any real number x,

A random variable X is called continuous if there exists a function called the probability density function of X and denoted by fX( · ), in terms of which ΡX[ · ] may be expressed as an integral†; for any Borel set B,

Since

it follows that

at all real numbers x at which the derivative exists.

The expectation, or mean, of a random variable X, denoted by E[X], is defined (when it exists) by

depending on whether X is specified by its distribution function,† its probability density function, or its probability mass function. The expectation of X is said to exist if the improper integral or infinite series given in Eq. 1.8 converges absolutely (see Mod Prob, pp. 203 and 250); in symbols, E[X] exists if and only if E[| X |] < ∞.

The variance of X is defined by

The standard deviation of X is defined by

The moment-generating function ψX( · ) of X is defined, for any real number t, by

The characteristic function φX( · ) of X is defined, for any real number u, by

A random variable may not possess a finite mean, variance, or moment-generating function. However, it always possesses a characteristic function. Indeed, there is a one-to-one correspondence between distribution functions and characteristic functions. Consequently, to specify the probability law of a random variable, it suffices to state its characteristic function.

Various inversion formulas, giving distribution functions, probability density functions, and probability mass functions explicitly in terms of characteristic functions, may be stated (see Mod Prob, Chapter 9). Here we state without proof two inversion formulas:

(i) for any random variable X

(ii) if the characteristic function φX( · ) is absolutely integrable in the sense that

then X is continuous, and its probability density function is given by

One expresses Eq. 1.15 in words by saying that fX( · ) is the Fourier transform of φX( · ). Conversely, if X is continuous, then its characteristic function φX( · ) is the Fourier transform of its probability density function :

Notice that Eqs. 1.15 and 1.16 are not quite symmetrical formulas; they differ by the factor (l/2π) and by a minus sign in the exponent.

Tables 1.1 and 1.2 give some probability laws, their characteristic functions, means, and variances. Table 1.3 gives some examples of random variables that obey these probability laws.

Jointly distributed random variables. Several random variables X1, X2, · · ·, Xn are said to be jointly distributed if they are defined as functions on the same sample description space. Their joint distribution

TABLE 1.1. Some frequently encountered discrete probability laws

function FX1,X2,..., Xn (· ,·,···, ·) is defined for all real numbers x1, x2, · · ·, xn by

Their joint characteristic function φX1,X2,..., Xn (·,·,···,·) is defined for all real numbers u1, u2, · · · , un by

Independent random variables. Jointly distributed random variables X1, X2, · · ·, Xn are said to be independent if and only if any of the following equivalent statements is true.

(i) Criterion in terms of probability functions: for all sets B1, B2, · · ·, Bn of real numbers

(ii) Criterion in terms of distribution functions: for all real numbers x1, x2, · · · , xn

TABLE 1.2. Some frequently encountered continuous probability laws

TABLE 1.3. Examples of random variables that obey the probability laws given in Tables 1.1 and 1.2

Reviews for Stochastic Processes

[hcubic · Rating: 4 out of 5 stars]

This was the text for one of the most difficult courses I tackled as a chemistry graduate student. it wasn't the book's fault!