Cover -- Half-title -- Title -- Copyright -- Contents -- Preface -- 1 Getting Started -- 1.1 Artificial neural networks -- 1.2 A simple example -- 1.3 General setup -- 1.4 Problems -- 2 Perceptron Learning - Basics -- 2.1 Gibbs learning -- 2.2 The annealed approximation -- 2.3 The Gardner analysis -- 2.4 Summary -- 2.5 Problems -- 3 A Choice of Learning Rules -- 3.1 The Hebb rule -- 3.2 The perceptron rule -- 3.3 The pseudo-inverse rule -- 3.4 The adaline rule -- 3.5 Maximal stability -- 3.6 The Bayes rule -- 3.7 Summary -- 3.8 Problems -- 4 Augmented Statistical Mechanics Formulation -- 4.1 Maximal stabilities -- 4.2 Gibbs learning at non-zero temperature -- 4.3 General statistical mechanics formulation -- 4.4 Learning rules revisited -- 4.5 The optimal potential -- 4.6 Summary -- 4.7 Problems -- 5 Noisy Teachers -- 5.1 Sources of noise -- 5.2 Trying perfect learning -- 5.3 Learning with errors -- 5.4 Refinements -- 5.5 Summary -- 5.6 Problems -- 6 The Storage Problem -- 6.1 The storage capacity -- 6.2 Counting dichotomies: the Cover analysis -- 6.3 Galilean pastiche: the Ising perceptron -- 6.4 The distribution of stabilities -- 6.5 Beyond the storage capacity -- 6.6 Problems -- 7 Discontinuous Learning -- 7.1 Smooth networks -- 7.2 The Ising perceptron -- 7.3 The reversed wedge perceptron -- 7.4 The dynamics of discontinuous learning -- 7.5 Summary -- 7.6 Problems -- 8 Unsupervised Learning -- 8.1 Supervised versus unsupervised learning -- 8.2 The deceptions of randomness -- 8.3 Learning a symmetry-breaking direction -- 8.4 Clustering through competitive learning -- 8.5 Clustering by tuning the temperature -- 8.6 Discussion -- 8.7 Problems -- 9 On-line Learning -- 9.1 Stochastic gradient descent -- 9.2 Specific examples -- 9.3 Optimal on-line learning -- 9.4 Perceptron with a smooth transfer function -- 9.5 Queries.

9.6 Unsupervised on-line learning -- 9.7 The natural gradient -- 9.8 Discussion -- 9.9 Problems -- 10 Making Contact with Statistics -- 10.1 Convergence of frequencies to probabilities -- 10.2 Sauer's lemma -- 10.3 The Vapnik-Chervonenkis theorem -- 10.4 Comparison with statistical mechanics -- 10.5 The Cramér-Rao inequality -- 10.6 Discussion -- 10.7 Problems -- 11 A Bird's Eye View: Multifractals -- 11.1 The shattered coupling space -- 11.2 The multifractal spectrum of the perceptron -- 11.3 The multifractal organization of internal representations -- 11.4 Discussion -- 11.5 Problems -- 12 Multilayer Networks -- 12.1 Basic architectures -- 12.2 Bounds -- 12.3 The storage problem -- 12.4 Generalization with a parity tree -- 12.5 Generalization with a committee tree -- 12.6 The fully connected committee machine -- 12.7 Summary -- 12.8 Problems -- 13 On-line Learning in Multilayer Networks -- 13.1 The committee tree -- 13.2 The parity tree -- 13.3 Soft committee machine -- 13.4 Back-propagation -- 13.5 Bayesian on-line learning -- 13.6 Discussion -- 13.7 Problems -- 14 What Else? -- 14.1 Support vector machines -- 14.2 Complex optimization -- 14.3 Error-correcting codes -- 14.4 Game theory -- Appendix 1 Basic Mathematics -- A1.1 Gaussian integrals -- A1.2 Jensen's inequality -- A1.3 The Delta-and Theta-functions -- A1.4 The saddle point method -- Appendix 2 The Gardner Analysis -- Appendix 3 Convergence of the Perceptron Rule -- Appendix 4 Stability of the Replica Symmetric Saddle Point -- First step -- Second step -- Third step -- Appendix 5 One-step Replica Symmetry Breaking -- Appendix 6 The Cavity Approach -- Appendix 7 The VC theorem -- Bibliography -- Index.