Cover -- Contents -- Preface -- Symbols and Acronyms -- 1 The Linear Data Fitting Problem -- 1.1 Parameter estimation, data approximation -- 1.2 Formulation of the data fitting problem -- 1.3 Maximum likelihood estimation -- 1.4 The residuals and their properties -- 1.5 Robust regression -- 2 The Linear Least Squares Problem -- 2.1 Linear least squares problem formulation -- 2.2 The QR factorization and its role -- 2.3 Permuted QR factorization -- 3 Analysis of Least Squares Problems -- 3.1 The pseudoinverse -- 3.2 The singular value decomposition -- 3.3 Generalized singular value decomposition -- 3.4 Condition number and column scaling -- 3.5 Perturbation analysis -- 4 Direct Methods for Full-Rank Problems -- 4.1 Normal equations -- 4.2 LU factorization -- 4.3 QR factorization -- 4.4 Modifying least squares problems -- 4.5 Iterative refinement -- 4.6 Stability and condition number estimation -- 4.7 Comparison of the methods -- 5 Direct Methods for Rank-Deficient Problems -- 5.1 Numerical rank -- 5.2 Peters-Wilkinson LU factorization -- 5.3 QR factorization with column permutations -- 5.4 UTV and VSV decompositions -- 5.5 Bidiagonalization -- 5.6 SVD computations -- 6 Methods for Large-Scale Problems -- 6.1 Iterative versus direct methods -- 6.2 Classical stationary methods -- 6.3 Non-stationary methods, Krylov methods -- 6.4 Practicalities: preconditioning and stopping criteria -- 6.5 Block methods -- 7 Additional Topics in Least Squares -- 7.1 Constrained linear least squares problems -- 7.2 Missing data problems -- 7.3 Total least squares (TLS) -- 7.4 Convex optimization -- 7.5 Compressed sensing -- 8 Nonlinear Least Squares Problems -- 8.1 Introduction -- 8.2 Unconstrained problems -- 8.3 Optimality conditions for constrained problems -- 8.4 Separable nonlinear least squares problems -- 8.5 Multiobjective optimization.

9 Algorithms for Solving Nonlinear LSQ Problems -- 9.1 Newton's method -- 9.2 The Gauss-Newton method -- 9.3 The Levenberg-Marquardt method -- 9.4 Additional considerations and software -- 9.5 Iteratively reweighted LSQ algorithms for robust data fitting problems -- 9.6 Variable projection algorithm -- 9.7 Block methods for large-scale problems -- 10 Ill-Conditioned Problems -- 10.1 Characterization -- 10.2 Regularization methods -- 10.3 Parameter selection techniques -- 10.4 Extensions of Tikhonov regularization -- 10.5 Ill-conditioned NLLSQ problems -- 11 Linear Least Squares Applications -- 11.1 Splines in approximation -- 11.2 Global temperatures data fitting -- 11.3 Geological surface modeling -- 12 Nonlinear Least Squares Applications -- 12.1 Neural networks training -- 12.2 Response surfaces, surrogates or proxies -- 12.3 Optimal design of a supersonic aircraft -- 12.4 NMR spectroscopy -- 12.5 Piezoelectric crystal identification -- 12.6 Travel time inversion of seismic data -- Appendix A: Sensitivity Analysis -- A.1 Floating-point arithmetic -- A.2 Stability, conditioning and accuracy -- Appendix B: Linear Algebra Background -- B.1 Norms -- B.2 Condition number -- B.3 Orthogonality -- B.4 Some additional matrix properties -- Appendix C: Advanced Calculus Background -- C.1 Convergence rates -- C.2 Multivariable calculus -- Appendix D: Statistics -- D.1 Definitions -- D.2 Hypothesis testing -- References -- Index -- A -- B -- C -- D -- E -- F -- G -- H -- I -- J -- K -- L -- M -- N -- O -- P -- Q -- R -- S -- T -- U -- V -- W.