Contents -- Series Foreword -- Preface -- Chapter 1. Introduction: Optimization and Machine Learning -- 1.1 Support Vector Machines -- 1.2 Regularized Optimization -- 1.3 Summary of the Chapters -- 1.4 References -- Chapter 2. Convex Optimization with Sparsity-Inducing Norms -- 2.1 Introduction -- 2.2 Generic Methods -- 2.3 Proximal Methods -- 2.4 (Block) Coordinate Descent Algorithms -- 2.5 Reweighted- 2 Algorithms -- 2.6 Working-Set Methods -- 2.7 Quantitative Evaluation -- 2.8 Extensions -- 2.9 Conclusion -- 2.10 References -- Chapter 3. Interior-Point Methods for Large-Scale Cone Programming -- 3.1 Introduction -- 3.2 Primal-Dual Interior-Point Methods -- 3.3 Linear and Quadratic Programming -- 3.4 Second-Order Cone Programming -- 3.5 Semidefinite Programming -- 3.6 Conclusion -- 3.7 References -- Chapter 4. Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey -- 4.1 Introduction -- 4.2 Incremental Subgradient-Proximal Methods -- 4.3 Convergence for Methods with Cyclic Order -- 4.4 Convergence for Methods with Randomized Order -- 4.5 Some Applications -- 4.6 Conclusions -- 4.7 References -- Chapter 5. First-Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose Methods -- 5.1 Introduction -- 5.2 Mirror Descent Algorithm: Minimizing over a Simple Set -- 5.3 Problems with Functional Constraints -- 5.4 Minimizing Strongly Convex Functions -- 5.5 Mirror Descent Stochastic Approximation -- 5.6 Mirror Descent for Convex-Concave Saddle-Point Problems -- 5.7 Setting up a Mirror Descent Method -- 5.8 Notes and Remarks -- 5.9 References -- Chapter 6. First-Order Methods for Nonsmooth Convex Large-Scale Optimization, II: Utilizing Problem's Structure -- 6.1 Introduction -- 6.2 Saddle-Point Reformulations of Convex Minimization Problems -- 6.3 Mirror-Prox Algorithm.

6.4 Accelerating the Mirror-Prox Algorithm -- 6.5 Accelerating First-Order Methods by Randomization -- 6.6 Notes and Remarks -- 6.7 References -- Chapter 7. Cutting-Plane Methods in Machine Learning -- 7.1 Introduction to Cutting-plane Methods -- 7.2 Regularized Risk Minimization -- 7.3 Multiple Kernel Learning -- 7.4 MAP Inference in Graphical Models -- 7.5 References -- Chapter 8. Introduction to Dual Decomposition for Inference -- 8.1 Introduction -- 8.2 Motivating Applications -- 8.3 Dual Decomposition and Lagrangian Relaxation -- 8.4 Subgradient Algorithms -- 8.6 Relations to Linear Programming Relaxations -- 8.7 Decoding: Finding the MAP Assignment -- 8.8 Discussion -- Appendix: Technical Details -- 8.10 References -- 8.5 Block Coordinate Descent Algorithms -- Chapter 9. Augmented Lagrangian Methods for Learning, Selecting, and Combining Features -- 9.1 Introduction -- 9.2 Background -- 9.3 Proximal Minimization Algorithm -- 9.4 Dual Augmented Lagrangian (DAL) Algorithm -- 9.5 Connections -- 9.6 Application -- 9.7 Summary -- Acknowledgment -- Appendix: Mathematical Details -- 9.9 References -- Chapter 10. The Convex Optimization Approach to Regret Minimization -- 10.1 Introduction -- 10.2 The RFTL Algorithm and Its Analysis -- 10.3 The "Primal-Dual" Approach -- 10.4 Convexity of Loss Functions -- 10.5 Recent Applications -- 10.6 References -- Appendix: The FTL-BTL Lemma -- Chapter 11. Projected Newton-type Methods in Machine Learning -- 11.1 Introduction -- 11.2 Projected Newton-type Methods -- 11.3 Two-Metric Projection Methods -- 11.4 Inexact Projection Methods -- 11.5 Toward Nonsmooth Objectives -- 11.6 Summary and Discussion -- 11.7 References -- Chapter 12. Interior-Point Methods in Machine Learning -- 12.1 Introduction -- 12.2 Interior-Point Methods: Background -- 12.3 Polynomial Complexity Result.

12.4 Interior-Point Methods for Machine Learning -- 12.5 Accelerating Interior-Point Methods -- 12.6 Conclusions -- 12.7 References -- Chapter 13. The Tradeoffs of Large-Scale Learning -- 13.1 Introduction -- 13.2 Approximate Optimization -- 13.3 Asymptotic Analysis -- 13.4 Experiments -- 13.5 Conclusion -- 13.6 References -- Chapter 14. Robust Optimization in Machine Learning -- 14.1 Introduction -- 14.2 Background on Robust Optimization -- 14.3 Robust Optimization and Adversary Resistant Learning -- 14.4 Robust Optimization and Regularization -- 14.5 Robustness and Consistency -- 14.6 Robustness and Generalization -- 14.7 Conclusion -- 14.8 References -- Chapter 15. Improving First and Second-Order: Methods by Modeling Uncertainty -- 15.1 Introduction -- 15.2 Optimization Versus Learning -- 15.3 Building a Model of the Gradients -- 15.4 The Relative Roles of the Covariance and the Hessian -- 15.5 A Second-Order Model of the Gradients -- 15.6 An Efficient Implementation of Online Consensus Gradient: TONGA -- 15.7 Experiments -- 15.8 Conclusion -- 15.9 References -- Chapter 16. Bandit View on Noisy Optimization -- 16.1 Introduction -- 16.2 Concentration Inequalities -- 16.3 Discrete Optimization -- 16.4 Online Optimization -- 16.5 References -- Chapter 17. Optimization Methods for Sparse Inverse Covariance Selection -- 17.1 Introduction -- 17.2 Block Coordinate Descent Methods -- 17.3 Alternating Linearization Method -- 17.4 Remarks on Numerical Performance -- 17.5 References -- Chapter 18. A Pathwise Algorithm for Covariance Selection -- 18.1 Introduction -- 18.2 Covariance Selection -- 18.3 Algorithm -- 18.4 Numerical Results -- 18.5 Online Covariance Selection -- Acknowledgements -- 18.6 References.