Statistics 650 Applied Stochastic Processes
Spring 2020A directory containing old final exams and sample
exam problems, and also a list of topics we emphasized in the Spring 2018 term's course, can be
found here.
Instructor: Eric Slud, Statistics Program, Math. Dept.
Office: MTH (Kirwan Hall) 2314, x5-5469, email slud@umd.edu
Office hours: M11am-12noon, W1-2pm or by appointment.
Required Course Text:
This book is somewhat more formal about theorems and proofs than Durrett, and uses some real-analysis language and the notation of "sigma-algebras", but is presented at the level of a slightly strengthened version of undergraduate Probability (STAT 410) and does not require any knowledge of measure theory.
Recommended Texts:
This is an excellent standard book, with thorough treatment of all important topics and
lots of applications. However, its problems are (much) too hard.
Additional Recommended Course Texts (used in Spring 2016 and 2018):
This is a good, readable book, with intuitive explanations and many interesting problems. The beta 2nd edition is available free online and also here.
This book is definitely easier than the others; it is radable at a more basic level and is a source for easier problems and more straightforward applications.
The Serfozo and Lefebvre books can be freely downloaded as e-books (through the University of Maryland libraries) by currently registered University of Maryland College Park students, who can also purchase hard-copies for $25.00. (Inquire at the EPSL (STEM) Library for details.)
Current HW Assignment Updated HW Solutions Info about Mid-term Test Info about Final Examination Sample Final-Exam Problems
Course Coverage: The core material of the course consists of Chapters 1-4 in the Serfozo book, with considerable skipping of more advanced topics, and Chapters 3, 5, and 6 in the Lefebvre book: the primary topics are Discrete Time Markov Chains, Poisson Processes, Continuous Time Markov Chains, and Renewal Processes. See detailed Course Outline below. The primary applications are population and queueing processes, but many other special applied examples occur in both books.
Overview: This course is about Stochastic Processes, or
time-evolving collections of random variables, primarily about the discrete-state
continuous time Markov chains which arise in applications in a variety of disciplines.
For the first part of the course, both the random variables and the time index-set are discrete:
in this setting, our object of study is discrete-time discrete-state Markov chains.
Examples of "states" which arise in applications include the size of a population or a
waiting-line, or the state ("in control" versus "out of control") of a manufacturing process,
or other indicators such as "married" or "employed" etc. for an individual. "Markov chains" are
time-evolving systems whose future trajectory does not depend on their past history,
given their current state. But many of the most interesting applications involve the generalization
of the same ideas to continuous time.
Probability theory material needed throughout this course includes joint probability laws,
probability mass functions and densities, conditional expectations, moment generating functions,
and an understanding of the various kinds of probabilistic convergence, including the Law of
Large Numbers.
Various technical tools developed in the course and used in an essential way include: Laplace
transforms and moment generating functions, methods of solving recursions and systems of
difference equations, ergodic and renewal theorems for Markov chains, and (discrete-time)
martingales.
Prerequisite: Stat 410 plus additional background in mathematical analysis and proofs, or Math 410 plus one semester of probability theory.
Course requirements and Grading: there will be 6 or 7 graded homework sets (one every 1½ to 2 weeks) which together will count 50% of the course grade.
In all homework problems, your solution will be graded correct only if you
fully justify your reasoning. There will also be an in-class test and a final examination,
which will respectively count 20% and 30% toward the overall course grade.
NOTE As of March 30: There will be 6 HW's in all. The Midterm will be an online 60 minute Quiz administered through ELMS. The Final will be Take-Home. The weights used in computing the final grade will be changed to: HW 60%, Midterm 15%, Final 25%. Details of these course changes, the revised Syllabus, and the mode of delivery of Lectures, Office Hours and Discussions can be found in the ELMS Pages and Files and here.
HW Assignment 1 (due in class Monday, Feb. 10):
(i) Consider the outcome of the first roll of the dice, to conclude that x = 3/36 + (27/36) x and solve.
(ii) Let A_{k} be the event that the sum is k and justify why x = P(A_{4}) / P( A_{4} ∪ A_{7}).
(iii) Calculate directly and then sum up over k the probabilities that 4 occurs before 7 and that this happens on the k'th trial.
HW Assignment 2 (due in class Monday, Feb. 24):
HW Assignment 3 (due in class Wednesday, March 11):
(1). Handout on a non-Markovian example constructed
by lumping 2 of the states in a 3-state Homogeneous Markov Chain.
(2). Handout of 5 pages from Durrett's "Essentials of Stochastic Processes" book
containing proofs for equivalent conditions to positive-recurrence, for aperiodic irreducible HMC's.
(3). Handout clarifying two loose ends from recent classes: (1) the condition for existence of a stationary probability distribution for recurrent Birth-Death chains, and (2) the notational verification as in Durrett that the "cycle trick" provides a stationary distribution for an irreducible positive-recurrent HMC.
(4). Handout on existence of invariant vectors for countable-state irreducible transient Homogeneous Markov Chains.
(5). Handout on the uniqueness of solutions of the Kolmogorov forward and backward equations when the rates q_i of leaving states are uniformly bounded.
(6). A Sample Test containing short-answer problems like the ones to appear on the March 30
in-class test can be found here. Note that this sample test does
not have Poisson process problems, although they are in scope for the mid-term.
(7). Handout on the main limit theorems of renewal theory, taken from 2 pages of the Grimmett and Stirzaker book, "Probability and Random Processes".
(8). Handout on Introduction to Markov Chain Monte Carlo. This is a statistical computing topic based on a strong theoretical foundation involving Markov Chains in discrete-time on discrete or continuous state spaces. This is unit 7 on my STAT 705 web-page, for which you can find lecture notes on Metropolis-Hastings and the Gibbs Sampler here. In addition, you can find lectures from a mini-course pair of lectures I gave on the topic in this folder.
Mid-Term Test
In light of the campus closure and online course format from March 30 through April 10, a mid-term test will be given on-line to be taken through ELMS between April 8 and 10, 2020. It will be easier than the HW problems, consisting of short-answer questions emphasizing definitions and the application of results of Theorems, plus a true-false component. Sample problems can be found here.
Sample problems and information about specific coverage of the test can be
found here.
Sample Final Exam
You can find sample final-exam problems here. By all
means try them and ask questions about them.
The Final Examination will be held from 4 to 6pm on Monday, May 18, 2020 in the
regular classroom if campus re-opens by then. If the campus does
not re-open by then, the final will be a set of take-home problems to be solved over 3-4
days. If the final is in-class, then it will have 5 or 6 problems. The coverage of the
Final is cumulative of all material from the Course Outline assigned and covered in class
throughout the semester. For an in-class final, you will be allowed 2 2-sided notebook
sheets of reference material for the exam; but the exam will otherwise be closed-book.
If the Final Exam is to be in-class, there will be a review session for the Final on
Friday, May 15, 4:30-6pm, in the regular classroom.
The UMCP Math Department home page.
The University of Maryland home page.
My home page.
© Eric V Slud, May 11, 2020.