Statistics 818R  Estimating Equations and Applications

Fall 2026 MWF 9-9:50am,    PHYS 2124

Link to Homework Assignments

Instructor: Eric Slud, Statistics program, Math. Dept.

Office:  Mth 2314, x5-5469, email slud@umd.edu, Office Hours: M1, W10, or by appointment

Overview: Estimating equations play an important role in many different statistical applications. The topic arises in analyzing data via Generalized Linear Models, Biostatistics including Survival Analysis, Survey Inference, Missing Data methodology, Stochastic Regression, Longitudinal Models, Spatial Statistics, Robust Statistics or Quantile Regression models. The theory of Estimating Equations is primarily a frequentist large-sample subject area, connecting problems where an exact maximum likelihood analysis is infeasible or inconvenient either because one does not want to model all distributions in detail in order to define a likelihood or because the full likelihood is analytically intractable. That is, there can be modeling reasons or technical reasons to estimate parameters by solving an equation (usually defined by summing a function across observations) depending on both observations and parameters. The basic idea is that, in data (Xi, Yi) available for indviduals i=1,...,n and independent identically distributed across individuals, an assumed relationship between a vector Xi of explanatory variables and a scalar or vector output Yi may be expressed in the form    E(Q(Xi, Yi, θ)) = 0    in terms of an unknown parameter for which the true value   θ0   is the unique solution, so that (subject to regularity conditions) the estimator   θ* = θ   solving   (1/n)   ∑i=1n   Q(Xi, Yi, θ*) = 0   is consistent and asymptotically normal with mean   θ0   and variance approximately proportional to   1/n.

The course will be run at Masters' level, like the Categorical Data class (STAT 770) that has often been taught in the past from the Agresti recommended text; that course material is more or less included in this course. The value of this class is in unifying the theoretical idea behind many practical applied models, including
---   Generalized Linear Models for Categorical Data (logistic regression, Poisson regression, etc.)
---   Longitudinal and Repeated-Measures Models
---   Quantile Regression Models
---   Weighted Models to compensate for Missing Data in Biostatistics and Surveys
The course will cover the models and basic theory and R computation and data examples.

Prerequisite: Stat 700, plus some computing familiarity. The level of the course will be similar to Stat 701.

Course requirements and Grading: there will be 6 graded homework sets (one every 2--2.5 weeks, split between theoretical and computational problems) which together will count 40% of the course grade, and a final project in the form of a 10-12 page paper presented orally on which there may be questions, that will serve as a final exam and will count 60% of the grade.

The COURSE SYLLABUS summarizes the grading and course policies.


Homework 1 Assignment linked here (hand in through ELMS)

There will be no Required Course Text, but book and journal references include the following. General Math Stat references are: at the level of STAT 700-701, P. Bickel and K. Doksum, Mathematical Statistics, 2nd ed., vol. I, CRC Press. And at a more advanced level, covering several theoretical topics that will come up in the course, A. van der Vaart, Advanced Statistcs, Cambridge Univ. Press.

Recommended References:   some of which can be obtained directly online from the UMD Libraries

Quasi-likelihood and GLM
Agresti, A. (2013) Categorical Data Analysis, 3rd ed. 2013.      Errata here.

Generalized Estimating Equations, Correlated Data
Hardin, J. and Hilbe, J. Generalized Estimating Equations, 2nd ed. 2013, CRC Press ISBN 978-1-4398-8113-2
Ziegler, A. Generalized Estimating Equations, 2011, Springer ISBN 978-1-4614-0499-6
Song, Peter X.-K. (2007), Correlated Data Analysis, Sprnger ISBN 978-0-387-71392-2

Biostatistics and Semiparametrics
Tsiatis, A. (2007), Semiparametric Theory and Missing Data, Springer.

Partial likelihood and Stochastic Regression
Slud, E. and Kedem, B. (1994), Partial likelihood analysis of logistic regression and autoregression, Statistica Sinica 4, 89-106
Kedem, B. and Fokianos, K.(2002), Regression Models for Time Series Analysis, Wiley-Interscience.

Missing-Data and Survey Topics
J.-K. Kim and J. Shao, (2013) Statistical Methods for Handling Incomplete Data, CRC.
R. Little and D. Rubin, Statistical Analysis with Missing Data (2002), 2nd ed., Wiley.


One part of the course -- roughly half -- will be devoted to the modeling aspects of Estimating Equation (EE) estimators, showing the models where they are used, the parameterizations used to formulate them, and the computational and software aspects of applying them. The other half of the course will be devoted to theoretical topics:
(i) the alternative likelihood concepts that have been used to motivate and analyze EE estimators, including quasilikelihood for GLMs, partial likelihood in Biostatistics, pseudolikelihood (same word but different meaning) in Survey statistics and Spatial statistics, and composite likelihood;
(ii) the general theory establishing consistency and asymptotic normality for EE parameter estimators, and
(iii) a little semiparametric theory (mostly in a biostatistical context) to explain the relative efficiency of EE methods of estimating finite-dimensional parameters in the presence of infinite-dimensional nuisance parameters.

NOTE ON USE OF THEORETICAL MATERIAL.  In homeworks and lectures, frequent use will be made of theoretical material at the level of probability theory needed to apply the law of large numbers and central limit theorem, along with the `delta method' (Taylor linearization) and other manipulations at advanced-calculus level.

NOTE ON COMPUTING.  Both in the homework-sets and the course project, you will be required to do computations on real datasets well beyond the scope of hand calculation or spreadsheet programs. Any of several statistical-computing platforms can be used to accomplish these: R, or Python or SAS, Matlab, or others. If you are learning one of these packages for the first time, I recommend   R which is free and open-source and is the most flexible and useful for research statisticians. I will provide links to free online R tutorials and will provide examples and scripts and will offer some R help.

Getting Started in R. Lots of R introductory materials can be found on the STAT 705 website.

FINAL PROJECT ASSIGNMENT, due Saturday, Dec 19, 2026. As a final course project, you are to write a paper including some 8--10 pages of narrative, plus relevant code and graphical or tabular exhibits, on a statistical journal article related to the course or else a data analysis or case-study based on a dataset of your choosing. The guideline is that the paper should be 10--12 pages if it is primarily expository based on an article, but could have somewhat fewer pages of narrative if based on a data-analytic case study. However, for the latter kind of paper, all numerical outputs should be accompanied by an Appendix of code used to generate them, plus discussion and interpretation of software outputs and graphical exhibits. For a data-analysis or case study, the paper should present a coherent and reasoned data analysis with supporting evidence for the model you choose to fit, the method and approach to handling missing data, and an assessment of the results.

Possible topics for the paper will be discussed and distributed during the course. They include:
(i) implementation and analysis/interpretation of one or more missing-data or imputation methods on a real dataset (e.g., survey public-use data from American Community Survey) using methods and software discussed in the course; exposition of a journal paper on missing data methods in a subject-matter application, such as educational statistics; exposition of some other missing-data topic, such as double- or interval-censored data, from a paper or book-chapter; or some other topic you propose.
(ii) Discussion and implementation of a method like Generalized Estimating Equations (for longitudinal data) or Propensity Weighting or Quantile Regression introduced in the course.
(iii) Other topics closely related to those covered in the course, like Causal Inference (related to inverse probability weighting in survey inference).

Handouts

(a) A list of miscellaneous references on Estimating Equations, in addition to the recommended books listed above, can be found here.

(b) The Estimating Equations part of the theoretical Chapter 6 in Bickel and Doksum's Mathematical Statistics text (vol.I) will be covered early in the semester to show how Consistency and Asymptotic Normality holds generally for Estimating Equation estimators (EEEs) in a parametric setting, but efficiency only for the EEEs associated with the true model likelihood.

(c) The optimality (efficiency) result mentioned in (b) above is generally credited to Godambe (1960) and a followup paper of V. Godambe and M. Thompson (1974), Ann. Stat. 2, pp.568-571. These are good places for general reading on Estimating Functions. Another general account of "Inference Functions" (same as Estimating Functions") with emphasis on likelihood-based methods of constructing them for correlated data, see Chapter 3 of the newly added Song recommended book listed above, which can be obtained free from the UMD Libraries.

(d) A very famous paper Econometrica by Halbert White (1982) on Estimating Equation analysis (via MLE) of misspecified models was discussed in class 9/14.

(e) In class, we discussed an estimating equation approach to the famous Fay-Herriot small area estimation model. We motivated this by the generalized method of moments, but it is a good example of why non-identical estimating functions for distinct observations are useful. Some additional highly cited papers presenting methods for the FH model that are essentially estimating equations can be found here and here.

(f) For an account of Estimating Equations based on Composite Likelihood, and of EM algorithms for solving them in settings with missing data, see this paper.

Additional Computing Resources.  There are many publicly available datasets for practice data-analyses. Many of them are taken from journal articles and/or textbooks and documented or interpreted. A good place to start is data included in R packages on the R-CRAN website. Datasets needed in the course will be either be posted to the course web-page, or indicated by links which will be provided here.

A good set of links to data sources from various organizations including Federal and international statistical agencies is at Washington Statistical Society links.

Important Dates

The UMCP Math Department home page.
The University of Maryland home page.
My home page.
©   Eric V Slud, September 17, 2026.