  # Numerical Analysis I

## AMSC 666, Fall 2017

### Course Information

 Lecture TuTh 2-3:15pm Room 4122 CSIC Bldg. #406 Note special place: on Tue 10/10 Math Bldg. Rm. 3201 Note special place: on Thu 10/12 Math Bldg. Rm. 3201 Addt'l Lectures on Wed 11/15, Wed 11/29 and Wed 12/6 10-11:15am Room 4122 CSIC Bldg. #406 Instructor Professor Eitan Tadmor Contact tel.: x5-0648   e-mail: Eitan Tadmor Office Hours By appointment (e-mail: Eitan Tadmor ) CSCAMM 4122 CSIC Bldg. #406 Teaching Assistant Kilian Cooley e-mail: Kilian Cooley Final Thursday 12/14 3-5pm 4122 CSIC Bldg. #406 NOTE: use of a calculator and your notes is allowed Grading 40% Homework; 60% Final

### Course Description

1. #### Approximation Theory

1. General overview
• On the choice of norm: least-squares vs. the uniform norm
• Weierstrass' density theorem; Bernstein polynomials

2. Least squares approximations I
• Least squares: ill-conditioning of Gramm matrix
• Orthogonal polynomials: Legendre, Chebyshev, Sturm's sequence
• Generalized Fourier expansions; Chebyshev expansions

##### Assignment #1: [ pdf file ] ... with answers [ pdf file]
3. Min-Max approximations: Chebyshev interpolant; economization

4. Error estimates in interpolation*
• Polynomial interpolation: Runge effect, region of analyticity
• Trigonometric interpolation: Sobolev least-squares vs uniform error bounds

##### • Lecture notes: Jackson's theorem [ pdf file]
• M. J. D. Powell, On the maximum errors of polynomial approximations defined by interpolation and by least squares criteria, Computer J., 1967, vol. 9, pp. 404-407, [pdf file]

The sin(x) subroutine in MATLAB [pdf file]

##### • Additional lecture notes on FFT:
J. Colley, P. Lewis, J. Welch, The fast Fourier Transform and its applications, IEEE on Education, 1969, vol. 12 (1), [pdf file]

C. DeBoor, FFT as nested multiplication with a twist, SISC 1980 vol. 1(1) [pdf file]

5. Least squares II -- the finite-dimensional case
• QR and and LS solution of over-detremined systems
• SVD, PCA and LS rank approximations
• From discrete least-squares to interpolation
##### Assignment #3 [pending]
6. Least squares III -- Gauss quadrature
On Newton-Cotes quadrature, Analysis of Numerical Methods, Issacson & Keller

Lecture notes: On Numerical integration and Euler-Macluarin formula
2. #### Numerical Solution of Initial-Value Problems

1. Systems of ODEs. Existence, uniqueness & stability.

2. Euler's method. Multistep methods -- stability and instability of Milne methods

• Predictor-Corrector methods: the class of Adams-Bashforth-Moulton schemes
##### Assignment #4 [ pdf file ]
3. Consistency, stability and convergence
• Dahlquist theory. The root condition
• Convergence of multi-step methods

##### Assignment #5 [ pdf file ]
4. Runge-Kutta methods
• Local time stepping and error estimates
• Examples of RK4 and RKF5
• Stability and Convergence of Runge-Kutta methods
##### • Lecture notes: Numerical methods for approximate solution of ODEs [ pdf file]
5. Strong-Stability Preserving (SSP) schemes. Methods for stiff equations
• On strong-stability preserving memthods: for [linear problems] and [nonlinear problems]

3. #### Iterative Methods for Linear Systems of Equations

1. Introduction and motivation. Stiff systems

2. Classical methods: Jacobi, Gauss-Siedel, SOR, ...

3. ##### • Lecture notes: Jacobi, Gauss-Seidel and SOR iterations [ pdf file]
• The original work of [ D. Young Thesis (1950)]
and a Fourier look at the converegence analysis [Garabedian (1956)]
##### Assignment #7 [pdf file]
4. Energy functionals and gradinet methods
• Steepest descent
• ADI and dimensional splitting methods

5. Non-stationary methods
• Polynomial equations: local vs. global methods

6. Preconditioners

7. Multigrid mthods*; GMRES*

##### Assignment #8 [pdf file]
• "Iterative methods for linear systems" Demmel's lecture notes [pdf file]
• Recent review of Krylov spaces [pdf file]
4. #### Numerical Optimization

1. Introduction: fixed point iterations, low-order and and Newton's methods

2. • Iterative solution of nonlinear eqs -- lecture notes by P. von Petersdorff [pdf file]

3. Steepest descent

4. Line search methods - Wolfe conditions

5. Newton's and quasi-Newton methods: Rank one/two modifications

6. Rates of converegence

### References

GENERAL TEXTBOOKS

K. Atkinson, An INTRODUCTION to NUMERICAL ANALYSIS, Wiley, 1987

S. Conte & C. deBoor, ELEMENTARY NUMERICAL ANALYSIS, McGraw-Hill
User friendly; Shows how 'it' works; Proofs, exercises and notes

G. Dahlquist & A. Bjorck, NUMERICAL METHODS, Prentice-Hall,
User friendly; Shows how 'it' works; Exercises

A. Ralston & P. Rabinowitz, FIRST COURSE in NUMERICAL ANALYSIS, 2nd ed., McGraw-hill,
Detailed; Scholarly written; Comprehensive; Proofs exercises and notes

J. Stoer & R. Bulrisch, INTRODUCTION TO NUMERICAL ANALYSIS, 2nd ed., Springer
detailed account on approximation, linear solvers & eigen-solvers, ODE solvers,..

APPROXIMATION THEORY

E. W. Cheney, INTRODUCTION TO APPROXIMATION THEORY
Classical

P. Davis, INTERPOLATION & APPROXIMATION, Dover

T. Rivlin, AN INTRODUCTION to the APPROXIMATION of FUNCTIONS
Classical

R. DeVore & G. Lorentz, CONSTRUCTIVE APPROXIMATION, Springer
A detailed account from classical theory to the modern theory; everything; Proofs exercises and notes

NUMERICAL INTEGRATION

F. Davis & P. Rabinowitz, NUMERICAL INTEGRATION,
Everything...

NUMERICAL SOLUTION Of INITIAL-VALUE PROBLEMS

E. Hairer, S.P. Norsett and G. Wanner, SOLVING ODEs I: NONSTIFF PROBLEMS, Springer-Verlag, Berlin. 1991, (2nd ed)
Everything - the modern version

A. Iserles, A FIRST COURSE in the NUMERICAL ANALYSIS of DEs, Cambridge Texts

W. Gear, NUMERICAAL INITIAL VALUE PROBLEMS in ODEs, 1971
The classical reference on theory and applications

Lambert, COMPUTATIONAL METHODS for ODEs, 1991
Detailed discussion of ideas and practical implementation

Shampine and Gordon, COMPUTER SOLUTION of ODES, 1975
Adams methods and practial implementation of ODE "black box" solvers

Butcher, NUMERICAL ANALYSIS of ODEs, 1987
Comprehensive discussion on Runge-Kutta methods

(mainly) ITERATIVE SOLUTION OF LINEAR SYSTEMS

A. Householder, THE THEORY OF MATRICES IN NUMERICAL ANALYSIS
The theoretical part by one of the grand masters; Outdated in some aspects

G. H. Golub & Van Loan, MATRIX COMPUTATIONS,
The basic modern reference

Y. Saad, ITERATIVE METHODS for SPARSE LINEAR SYSTEMS,
PWS Publishing, 1996. (Available on line at http://www-users.cs.umn.edu/~saad/books.html)

R. Varga, MATRIX ITERATIVE ANALYSIS,
Classical reference for the theory of iterations

James Demmel, APPLIED NUMERICAL LINEAR ALGEBRA,
SIAM, 1997

Kelley, C. T., ITERATIVE METHODS for LINEAR and NONLINEAR EQUATIONS,
SIAM 1995

NUMERICAL OPTMIZATION

J. Nocedal, S. Wright, NUMERICAL OPTIMIZATION
Springer, 1999

T. Kelly, ITERATIVE METHODS for OPTIMIZATION
SIAM