Cover -- Preface to the International Edition -- Contents -- Chapter 1 Random Variables and Stochastic Processes -- 1.1 Generating Functions -- 1.1.1 Introduction -- 1.1.2 Probability Generating Function: Mean and Variance -- 1.1.3 Sum of (a Fixed Number of) Random Variables -- 1.1.4 Sum of a Random Number of Discrete Random Variables (Stochastic Sum) -- 1.1.5 Generating Function of Bivariate Distribution -- 1.2 Laplace Transform -- 1.2.1 Introduction -- 1.2.2 Some Important Properties of Laplace Transforms: see Appendix A1 -- 1.2.3 Inverse Laplace Transform -- 1.3 Laplace (Stieltjes) Transform of a Probability Distribution or of a Random Variable -- 1.3.1 Definition -- 1.3.2 The Laplace Transform of the Distribution Function in Terms of that of the Density Function -- 1.3.3 Mean and Variance in Terms of (Derivatives of) L.T. -- 1.3.4 Some Important Distributions -- 1.3.5 Three Important Theorems -- 1.3.6 Geometric and Exponential Distributions -- 1.3.7 Sum of a Random Number of Continuous Random Variables Stochastic δm -- 1.3.8 Randomization and Mixtures -- 1.4 Classification of Distributions -- 1.4.1 Hazard (or Failure) Rate Function -- 1.4.2 Mean Residual Life (MRL) -- 1.4.3 Further Properties -- 1.5 Stochastic Processes: An Introduction -- 1.5.1 Specification of Stochastic Processes -- Exercises -- References -- Chapter 2 Markov Chains -- 2.1 Definition and Examples -- 2.1.1 Transition Matrix (or Matrix of Transition Probabilities -- 2.1.2 Order of a Markov Chain -- 2.1.3 Markov Chains as Graphs -- 2.2 Higher Transition Probabilities -- 2.3 Generalisation of Independent Bernoulli Trials: Sequence of Chain-Dependent Trials -- 2.3.1 Markov-Bernoulli Chain -- 2.3.2 Correlated Random Walk -- 2.4 Classification of States and Chains -- 2.4.1 Communication Relations -- 2.4.2 Class Property -- 2.4.3 Classification of Chains.

2.4.4 Classification of States: Transient and Persistent (Recurrent) States -- 2.5 Determination of Higher Transition Probabilities -- 2.5.1 Aperiodic Chain: Limiting Behaviour -- 2.6 Stability of a Markov System -- 2.6.1 Computation of the Equilibrium Probabilities -- 2.7 Graph Theoretic Approach -- 2.8 Markov Chain With Denumerable Number of States (Or Countable State Space) -- 2.9 Reducible Chains -- 2.9.1 Finite Reducible Chains with a Single Closed Class -- 2.9.2 Chain with One Single Class of Persistent Non-null Aperiodic States -- 2.9.3 Absorbing Markov Chains -- 2.9.4 Extension: Reducible Chain with one Closed Class of Persistent Aperiodic States -- 2.9.5 Further Extension: Reducible Chains with more than one Closed Class -- 2.10 Statistical Inference for Markov Chains -- 2.10.1 M.L. Estimation and Hypothesis Testing -- 2.10.2 Determination of the Order of a Markov Chain by MAICE -- 2.11 Markov Chains With Continuous State Space -- 2.12 Non-Homogeneous Chains -- 2.12.1 Matrix Approach for Finite Non-homogeneous Chain -- Exercises -- References -- Chapter 3 Markov Processes with Discrete State Space: Poisson Process and its Extensions -- 3.1 Poisson Process -- 3.1.1 Introduction -- 3.1.2 Postulates for Poisson Process -- 3.1.3 Properties of Poisson Process -- 3.2 Poisson Process and Related Distributions -- 3.2.1 Interarrival Time -- 3.2.2 Further Interesting Properties of Poisson Process -- 3.3 Generalisations of Poisson Process -- 3.3.1 Poisson Process in Higher Dimensions -- 3.3.2 Poisson Cluster Process (Compound or Cumulative Poisson Process) -- 3.3.3 Pure Birth Process: Yule-Furry Process -- 3.3.4 Birth-Immigration Process -- 3.3.5 Time-dependent Poisson Processes (Non-homogeneous Poisson process) -- 3.3.6 Random Variation of the Parameter -- 3.3.7 Renewal Process -- 3.4 Birth and Death Process -- 3.4.1 Particular Cases.

3.5 Markov Processes with Discrete State Space (Continuous time Markov Chains) -- 3.5.1 Introduction -- 3.5.2 Chapman-Kolmogorov Equations -- 3.5.3 Limiting Distribution (Ergodicity of Homogeneous Markov Process) -- 3.5.4 Graph Theoretic Approach for Determining V -- 3.6 Randomization (Uniformization): Derived Markov Chains -- 3.7 Erlang Process -- 3.7.1 Introduction -- 3.7.2 Erlangian Distribution -- Exercises -- References -- Chapter 4 Markov Processes with Continuous State Space -- 4.1 Introduction: Brownian Motion -- 4.2 Wiener Process -- 4.3 Differential Equations for a Wiener Process -- 4.4 Kolmogorov Equations -- 4.5 First Passage time Distribution for Wiener Process -- 4.5.1 Distribution of the Maximum of a Wiener Process -- 4.5.2 Distribution of the First Passage Time to a Fixed Point -- 4.6 Ornstein-Uhlenbeck Process -- Exercises -- References -- Chapter 5 Martingales -- 5.1 Introduction -- 5.2 Definitions and Examples -- 5.3 Properties of Martingales -- 5.3.1 Maximal Inequality for Nonnegative Martingales -- 5.3.2 Martingale Convergence Theorem -- 5.3.3 Optional Stopping Theorem -- 5.4 Continuous Parameter Martingales -- Exercises -- References -- Chapter 6 Renewal Processes and Theory -- 6.1 Renewal Process -- 6.1.1 Renewal Process in Discrete Time -- 6.1.2 Relation Between F(s) and P(s) -- 6.1.3 Renewal Interval -- 6.1.4 Generalised Form: Delayed Recurrent Event -- 6.1.5 Renewal Theory in Discrete Time -- 6.2 Renewal Processes in Continuous Time -- 6.2.1 Renewal Function and Renewal Density -- 6.3 Renewal Equation -- 6.4 Stopping Time: Wald's Equation -- 6.4.1 Stopping Time -- 6.4.2 Wald's Equation -- 6.5 Renewal Theorems -- 6.5.1 Elementary Renewal Theorem -- 6.5.2 Applications -- 6.5.3 Some Definitions -- 6.5.4 Renewal Theorems (Blackwell's and Smith's) -- 6.5.5 Central Limit Theorem for Renewals.

6.6 Delayed and Equilibrium Renewal Processes -- 6.6.1 Delayed (modified) Renewal Process -- 6.6.2 Equilibrium (or Stationary) Renewal Process -- 6.6.3 Probability Generating Function (p.g.f.) of Renewal Processes -- 6.7 Residual and Excess Lifetimes -- 6.7.1 Poisson Process as a Renewal Process -- 6.7.2 Distribution of Y (t) and Z (t) -- 6.7.3 Moments of the Asymptotic Distributions -- 6.8 Renewal Reward (Cumulative Renewal)Process -- 6.9 Alternating (Or Two-Stage) Renewal Process -- 6.10 Regenerative Stochastic Processes: Existence of Limits -- 6.11 Regenerative Inventory System -- 6.11.1 Introduction -- 6.11.2 Inventory Position -- 6.12 Generalisation of the Classical Renewal Theory -- Exercises -- References -- Chapter 7 Markov Renewal and Semi-Markov Processes -- 7.1 Introduction -- 7.2 Definitions and Preliminary Results -- 7.2.1 Waiting Times -- 7.3 Markov Renewal Equation -- 7.3.1 Interval Transition Probability Matrix (i.t.p.m.) -- 7.4 Limiting Behaviour -- 7.4.1 Limiting Distribution of s-M.P. and 'Recurrence Times' -- 7.5 First Passage TIme -- Exercises -- References -- Chapter 8 Stationary Processes and Time Series -- 8.1 Stationary Processes -- 8.1.1 Second-Order Processes -- 8.1.2 Stationarity -- 8.1.3 Gaussian Processes -- 8.2 Time Series: Introduction -- 8.2.1 Purely Random Process (or White Noise Process -- 8.2.2 First Order Markov Process -- 8.2.3 Moving Average (MA) Process -- 8.2.4 Autoregressive Process (AR Process) -- 8.2.5 Autoregressive Process of Order Two (Yule Process) -- 8.2.6 Autoregressive Moving Average Process (ARMA Process) -- 8.3 Time And Frequency Domain: Power Spectrum -- 8.3.1 Properties of Covariance and Correlation Functions -- 8.3.2 Continuous Parameter Processes -- 8.4 Statistical Analysis of Time Series: Some Observations -- Exercises -- References -- Chapter 9 Branching Processes -- 9.1 Introduction.

9.2 Properties of Generating Functions of Branching Processes -- 9.2.1 Moments of Xn -- 9.3 Probability of Ultimate Extinction -- 9.3.1 Asymptotic Distribution of Xn -- 9.3.2 Examples -- 9.4 Distribution of the Total Number of Progeny -- 9.5 Conditional Limit Laws -- 9.5.1 Critical Processes -- 9.5.2 Subcritical Processes -- 9.6 Generalisations of the Classical Galton-Watson Process -- 9.6.1 Branching Processes with Immigration -- 9.6.2 Processes in Varying and Random Environments -- 9.6.3 Multitype Galton-Watson Process -- 9.6.4 Controlled Galton-Watson Branching Process -- 9.7 Continuous-Time Markov Branching Process -- 9.8 Age Dependent Branching Process: Bellman-Harris Process -- 9.8.1 Generating Function -- 9.9 General Branching Processes -- 9.9.1 Crump-Mode-Jagers Process -- 9.9.2 Some Extensions and Generalisations of Continuous Time Processes -- 9.10 Some Observations -- 9.10.1 Inference Problems -- 9.10.2 Concluding Remarks -- Exercises -- References -- Chapter 10 Applications in Stochastic Models -- 10.1 Queueing Systems and Models -- 10.1.1 Queueing Processes -- 10.1.2 Notation -- 10.1.3 Steady State Distribution -- 10.1.4 Little's Formula -- 10.2 Birth and Death Processes in Queueing Theory: Markovian Models -- 10.2.1 Birth and Death Processes -- 10.2.2 The Model M/M/s -- 10.2.3 Model M/M/s/s: Erlang Loss Model -- 10.3 Non-Markovian Queueing Models -- 10.3.1 Queues with Poisson Input: Model M/G/1 -- 10.3.2 Pollaczek-Khinchine Formula -- 10.3.3 Busy Period -- 10.4 The Model GI/M/1 -- 10.4.1 Steady State Distribution -- 10.5 Reliability Models -- 10.5.1 Introduction -- 10.5.2 System Reliability -- 10.5.3 Markovian Models in Reliability Theory -- 10.5.4 Shock Models and Wear Processes -- 10.5.5 Some Surveys -- 10.6 Other Models -- Exercises -- References -- Chapter 11 Simulation -- 11.1 Introduction.

11.1.1 Generation of Pseudorandom Numbers.