Statistics 650 Applied Stochastic Processes
Information about Mid-term Test
Information about Final Examination
Sample Final-Exam Problems Instructor: Eric Slud, Statistics Program, Math.
Dept. Office: Mth 2314, x5-5469,
email evs@math.umd.edu Office hours: M3, W4 or by
appointment. Course Text: P. Bremaud, Markov-Chains:
Gibbs Fields, Monte Carlo Simulation and Queues,
This book is a well-written mathematical treatment. The first two items
in the subtitle
Spring
2007
MW 5-6:15pm , Mth 0106
1999 Springer.
indicate that the applications emphasized are slightly
different from the usual ones,
respectively statistical-mechanics
type models and statistical simulation and computation.
2nd ed. 1975,
Academic Press.
This is an excellent standard book, with thorough treatment of all
important topics
and a different set of applications from the
Bremaud text. However, its problems
are (much) too hard.
R. Durrett, Essentials of Stochastic processes, 1999,
Springer-Verlag.
Slightly less formal problem-oriented text,
which explains things well,
and has lots of simple examples and
problems.
Overview: This course is about
Stochastic Processes, or time-evolving
collections of random variables, primarily about the discrete-state
continuous time Markov chains which arise in applications in a
variety of disciplines. For the first part of the course, both the
random variables and the time index-set are discrete: in this setting,
our object of study is discrete-time discrete-state Markov chains.
Examples of "states" which arise in applications include the size of a
population or a waiting-line, or the state ("in control" versus "out
of control") of a manufacturing process, or other indicators such as
"married" or "employed" etc. for an individual. "Markov chains" are
time-evolving systems whose future trajectory does not depend on their
past history, given their current state. But the most interesting
applications involve the generalization of the same ideas to
continuous time.
Probability theory material needed throughout this course includes
joint probability laws, probability mass functions and densities,
conditional expectations, moment generating functions, and an
understanding of the various kinds of probabilistic convergence,
including the Law of Large Numbers.
Various technical tools developed in the course and used in an
essential way include: Laplace transforms and moment generating
functions, methods of solving recursions and systems of difference
equations, ergodic and renewal theorems for Markov chains, and
(discrete-time) martingales.
Prerequisite: Stat 410, or Math 410 plus one semester of probability theory.
Course requirements and Grading: there will
be 7 or 8 graded homework sets
(one every 1½ to 2 weeks) which
together will count 50% of the course grade.
There will also be an
in-class test and a final examination, which will respectively
count
20% and 30% toward the overall course grade.
The seventh HW assignment is now
posted
here , with due date of Friday, 5/11/07 .
See the posted Homework problem set at the indicated link for some
re-wording
and correction
of the originally posted problems. Note also that problem #2 and
the part of Durrett #8.25(a) asking you to justify uniqueness of
forward and
backward equation solutions have both been made
optional.
Recall that there will be a review session on Friday, May 11,
3-5pm, for the Final Exam,
and you may hand in the last HW at
that time (or in my mailbox or under my office
door at any time that afternoon).
The Final Exam is 4-6pm on Wednesday 5/16 in the regular classroom.
For a handout on the non-Markovian example I discussed in class
as being constructed by
lumping 2 of the states in a 3-state
Homogeneous Markov Chain, click here.
For a handout on existence of invariant vectors for
countable-state irreducible transient HMC's,
click
here.
For a handout on the uniqueness of solutions of the Kolmogoroff
forward and backward equations
when the rates q_i of leaving
states are uniformly bounded, click here.
Mid-Term Test
A mid-term test will be given in-class on Monday, April 2,
2007. It will be
easier than the HW problems, consisting of
short-answer questions emphasizing
definitions and the application
of results of Theorems. On the test, you will
not be permitted to use your books, but you are allowed to bring up to
two
notebook sheets of notes for reference.
Sample problems and information about
specific coverage of the test can be Sample Final Exam
You can find sample final-exam problems here. By all means try them and The Final Examination will be held from 4 to 6pm on Wednesday,
May 16, 2007. Also, you will be allowed 2 2-sided notebook sheets of
reference material for The UMCP
Math Department home page. The University of
Maryland home page. My home
page. © Eric V Slud,
May 10, 2007.
found here.
ask
questions about them, either in class or in the Review Session to
be held Friday, May 11, in our classroom from 3--5pm.
Final Examination
It will have 5 or 6 problems. The coverage is
cumulative of all material
from the Bremaud book assigned or
covered in class throughout the semester.
In particular, the
Queueing material from the Durrett book will not be
specifically covered in the exam.
the exam.
But the exam will otherwise be closed-book.
SYLLABUS for Stat 650
We will cover Chapters 2-4 and 8 of Bremaud's thoroughly,
plus parts of chapters 5-6,
especially Sections 5.3, 6.1-6.2,
and some of the Chapter 7 material on Simulation.
OUTLINE
0. Probability Review.
( Chapter 1.) 1 Lecture
Probability spaces, countable additivity.
Conditional expectation and transforms of r.v.'s
Spaces of trajectories (infinite sequences of states).
2. Discrete-time Discrete-State Markov
Chains. ( Chapter 2.)
5 Lectures
(a) Markov property. Examples of Markov & non-Markov random
sequences.
(b) Multistep transition probabilities. Chapman-Kolmogorov equation.
(c) "First-step analysis"
(d) Classification of states.
(e) Notions of limiting behavior. Reducibility. Recurrence. Steady state.
(f) Time reversibility & regeneration.
3. Recurrence and Ergodicity.
( Chapter 3.)
4 Lectures
(a) Criteria for recurrence and positive recurrence.
Random-walk & birth-death
examples.
(b) Empirical averages. Ergodic Theorem (Law of Large Numbers).
Renewal reward theorem.
4. Asymptotic Behavior. Equilibrium & Renewal theory.
( Chapter 4.)
4 Lectures
(a) Coupling proof of equilibrium.
(b) Renewal theory. Regenerative processes.
(c) Analysis of Absorption (in transient examples).
5. Martingales & Applications.
( Section 5.3. )
3 Lectures
(a) Definitions and Optional Sampling Theorem.
(b) Expectation calculations in examples.
(c) Random-walk, branching-process and other examples.
6. Poisson Processes & Continuous-time Chains.
( Chapter 8 )
6 Lectures
(a) Poisson process def'ns and characterizations
(b) Relation to Discrete-time chains. Embedding.
(c) Transition probabilities. Recurrence. Limiting behavior.
(d) Birth-death process examples.
(e) Queueing examples.
(f) Reversibility. Applications in Queueing
& Markov Chain Monte Carlo.
7. Random fields and Applications to
Statistical Computation.
( Chapter 7,
Sections 1-2, 6-8. )
4 Lectures
Important Dates
IN-CLASS FINAL EXAM: NOTE 4pm instead of 5pm.