Numerical Analysis I
AMSC/CMSC 666, Fall 2021
Course Information
(*) Additional notes on online class format are found here.
Course Description

Approximation Theory
 General overview. Leastsquares vs. the uniform norm
 Orthogonal polynomials
 Least squares I. Fourier expansions.
• Assignment
[ #1 ]
... with [ answers]
 Least squares II  the finitedimensional case. QR, SVD, PCA (LS rank approximations)
• Lecture notes:
[ SVD ]
(with additional notes on QR factorization )
• Additional reading:
► Gilbert Strang
[on SVD]
(and also here)
► Lijie Cao
[SVD applied to digital image processing]
• Assignment [ #2 ]
 Gauss quadrature

Numerical Solution of ODEs: InitialValue Problems

Preliminaries. Stability of systems of ODEs.
• Assignment
[ #3 ]

Examples of basic numerical methods: Euler's method, LeapFrog, Milne.
• Assignment

Consistency and stability imply convergence
• Assignment
 PredictorCorrector methods: AdamsBashforthMoulton schemes
• Assignment
 RungeKutta methods
 Local time stepping and error estimates. RK4 and RKF5.
 Stability and convergence of RungeKutta methods
• Assignment

Stiff systems and absolute stability

StrongStability Preserving (SSP) methods:
Iterative Methods for Solving Systems of Linear Equations
 Stationary methods: Jacobi, GaussSeidel, SOR, ...
• Assignment
 Energy functionals and gradient methods
 Steepest descent. Conjugate gradient. GMRES*
• Assignment
 Acceleration method. preconditioners
 ADI and dimensional splitting; Multigrid methods*
Numerical Optimization
 General overview. Fixed point iterations, loworder and Newton's methods
 Finding roots of nonlinear equations: scalar and systems. Homotopy
• Assignment
 Computation of minimizers. Numerical Optimization
 Gradient flows
 Line search methods. Trust regions and Wolfe conditions
 Newton and quasiNewton: SR1 and BFGS methods
 Nonlinear Leastsquares. GaussNewton.
References
GENERAL TEXTBOOKS
W. Gautschi, NUMERICAL ANALYSIS, Birkhauser, 2012
K. Atkinson, An INTRODUCTION to NUMERICAL ANALYSIS, Wiley, 1987
S. Conte & C. deBoor, ELEMENTARY NUMERICAL ANALYSIS, McGrawHill
User friendly; Shows how 'it' works; Proofs, exercises and notes
G. Dahlquist & A. Bjorck, NUMERICAL METHODS, PrenticeHall,
User friendly; Shows how 'it' works; Exercises
A. Ralston & P. Rabinowitz, FIRST COURSE in
NUMERICAL ANALYSIS, 2nd ed., McGrawhill,
Detailed; Scholarly written; Comprehensive; Proofs exercises and notes
J. Stoer & R. Bulrisch, INTRODUCTION TO NUMERICAL ANALYSIS, 2nd ed., Springer
detailed account on approximation, linear solvers & eigensolvers,
ODE solvers,..
APPROXIMATION THEORY
E. W. Cheney, INTRODUCTION TO APPROXIMATION THEORY
Classical
P. Davis, INTERPOLATION & APPROXIMATION, Dover
Very readable
T. Rivlin, AN INTRODUCTION to the APPROXIMATION of FUNCTIONS
Classical
R. DeVore & G. Lorentz, CONSTRUCTIVE APPROXIMATION, Springer
A detailed account from classical theory to the modern theory; everything; Proofs exercises and notes
NUMERICAL INTEGRATION
F. Davis & P. Rabinowitz, NUMERICAL INTEGRATION,
Everything...
NUMERICAL SOLUTION Of INITIALVALUE PROBLEMS
E. Hairer, S.P. Norsett and G. Wanner, SOLVING ODEs I: NONSTIFF PROBLEMS,
SpringerVerlag, Berlin. 1991, (2nd ed)
Everything  the modern version
A. Iserles, A FIRST COURSE in the NUMERICAL ANALYSIS of DEs,
Cambridge Texts
W. Gear, NUMERICAAL INITIAL VALUE PROBLEMS in ODEs, 1971
The classical reference on theory and applications
Lambert, COMPUTATIONAL METHODS for ODEs, 1991
Detailed discussion of ideas and practical implementation
Shampine and Gordon, COMPUTER SOLUTION of ODES, 1975
Adams methods and practial implementation of ODE "black box" solvers
Butcher, NUMERICAL ANALYSIS of ODEs, 1987
Comprehensive discussion on RungeKutta methods
(mainly) ITERATIVE SOLUTION OF LINEAR SYSTEMS
A. Householder, THE THEORY OF MATRICES IN NUMERICAL ANALYSIS
The theoretical part by one of
the grand masters; Outdated in some aspects
G. H. Golub & Van Loan, MATRIX COMPUTATIONS,
The basic modern reference
Y. Saad, ITERATIVE METHODS for SPARSE LINEAR SYSTEMS,
PWS Publishing, 1996. (Available on line at
http://wwwusers.cs.umn.edu/~saad/books.html)
R. Varga, MATRIX ITERATIVE ANALYSIS,
Classical reference for the theory of iterations
James Demmel, APPLIED NUMERICAL LINEAR ALGEBRA,
SIAM, 1997
Kelley, C. T., ITERATIVE METHODS for LINEAR and NONLINEAR EQUATIONS,
SIAM 1995
NUMERICAL OPTMIZATION
J. Nocedal, S. Wright, NUMERICAL OPTIMIZATION
Springer, 1999
T. Kelly, ITERATIVE METHODS for OPTIMIZATION
SIAM
Eitan Tadmor

