On-Line Learning in Neural Networks

doi:10.1017/CBO9780511569920

On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.

Review of the hardback:‘I recommend this book to readers with a theoretical, analytical, or mathematical interest in neural networks, especially online learning.’

Source: Computing Reviews

- Aa Reduce text
- Aa Enlarge text

View selected items
Save to my bookmarks
Export citations
Download PDF (zip)
Save to Kindle
Save to Dropbox
Save to Google Drive
Save content to
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to .

To save content items to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Please be advised that item(s) you selected are not available.
You are about to save
Your Kindle email address

Please provide your Kindle email.

@free.kindle.com @kindle.com (service fees apply)

By using this service, you agree that you will only keep content for personal use, and will not openly distribute them via Dropbox, Google Drive or other file sharing services

Contents

Frontmatter
pp i-iv
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

Contents
pp v-vi
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

Acknowledgements
pp vii-viii
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

List of contributors
pp ix-x
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

Foreword
pp 1-2
- By Christopher M. Bishop
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

1 - Introduction
pp 3-8
- By David Saad, Neural Computing Research Group, Aston University Birmingham B4 7ET, UK.
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

2 - On-line Learning and Stochastic Approximations
pp 9-42
- By Léon Bottou, AT&T Labs-Research Red Bank, NJ07701, USA
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

3 - Exact and Perturbation Solutions for the Ensemble Dynamics
pp 43-62
- By Todd K. Leen, Dept. of Computer Science and Engineering and Dept. of Electrical and Computer Engineering Oregon Graduate Institute of Science & Technology P.O. Box 91000 Portland, Oregon, USA
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

4 - A Statistical Study of On-line Learning
pp 63-92
- By Noboru Murata, Lab. for Information Synthesis, RIKEN Brain Science Institute Wako, Saitama 351–0198, Japan
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

5 - On-line Learning in Switching and Drifting Environments with Application to Blind Source Separation
pp 93-110
- By Klaus-Robert Müller, GMD-First, Rudower Chaussee 5, D-12489 Berlin, Germany, Andreas Ziehe, GMD-First, Rudower Chaussee 5, D-12489 Berlin, Germany, Noboru Murata, Brain Science Institute, RIKEN, Wako-shi, Saitama 351–0198, Japan, Shun-ichi Amari, Brain Science Institute, RIKEN, Wako-shi, Saitama 351–0198, Japan
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

6 - Parameter Adaptation in Stochastic Optimization
pp 111-134
- By Luís B. Almeida, INESC R. Alves Redol, 9 1000 Lisboa, Portugal, Thibault Langlois, Jose D. Amaral, Alexander Plakhov, INESC R. Alves Redol, 9 1000 Lisboa, Portugal
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

7 - Optimal On-line Learning in Multilayer Neural Networks
pp 135-164
- By David Saad, Neural Computing Research Group, Aston University Birmingham B4 7ET, UK, Magnus Rattray, Neural Computing Research Group, Aston University Birmingham B4 7ET, UK
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

8 - Universal Asymptotics in Committee Machines with Tree Architecture
pp 165-182
- By Mauro Copelli, Limburgs Universitair Centrum B-3590 Diepenbeek, Belgium, Nestor Caticha, Instituto de Física, Universidade de São Paulo Caixa Postal 66318, 05389–970 São Paulo, SP, Brazil
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

9 - Incorporating Curvature Information into On-line Learning
pp 183-208
- By Magnus Rattray, Neural Computing Research Group, Aston University Birmingham B4 7ET, UK, David Saad, Neural Computing Research Group, Aston University Birmingham B4 7ET, UK
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

10 - Annealed On-line Learning in Multilayer Neural Networks
pp 209-230
- By Siegfried Bös, Brain Science Institute, RIKEN Wako–shi, Saitama 351–0198, Japan, Shun-Ichi Amari, Brain Science Institute, RIKEN Wako–shi, Saitama 351–0198, Japan
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

11 - On-line Learning of Prototypes and Principal Components
pp 231-250
- By Michael Biehl, Institut für Theoretische Physik Universität Würzburg Am Hubland D-97074 Würzburg, Germany, Ansgar Freking, Matthias Hölzer, Georg Reents, Enno Schlösser, Institut für Theoretische Physik Universität Würzburg Am Hubland D-97074 Würzburg, Germany
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

12 - On-line Learning with Time-Correlated Examples
pp 251-278
- By Tom Heskes, RWCP Theoretical Foundation, SNN, Department of Medical Physics and Biophysics, University of Nijmegen, Geert Grooteplein 21, 6525 EZ Nijmegen, The Netherlands, Wim Wiegerinck, RWCP Theoretical Foundation, SNN, Department of Medical Physics and Biophysics, University of Nijmegen, Geert Grooteplein 21, 6525 EZ Nijmegen, The Netherlands.
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

13 - On-line Learning from Finite Training Sets
pp 279-302
- By David Barber, Department of Medical Biophysics, University of Nijmegen, 6525 EZ Nijmegen, The Netherlands, Peter Sollich, Department of Physics, University of Edinburgh, Edinburgh EH9 3JZ, U.K.
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

14 - Dynamics of Supervised Learning with Restricted Training Sets
pp 303-344
- By A. C. C. Coolen, Department of Mathematics, King's College, University of London, Strand, London WC2R 2LS, U.K., D. Saad, Department of Computer Science and Applied Mathematics, Aston University, Aston Triangle, Birmingham B4 7ET, U.K.
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

15 - On-line Learning of a Decision Boundary with and without Queries
pp 345-362
- By Yoshiyuki Kabashima, Dept. of Comp. Intelligence and Systems Science Graduate School of Science and Engineering Tokyo Institute of Technology Yokohama 226, Japan, Shigeru Shinomoto, Dept. of Physics, Kyoto University Sakyo-ku, Kyoto, 606-8502, Japan
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

16 - A Bayesian Approach to On-line Learning
pp 363-378
- By Manfred Opper, Neural Computing Research Group, Aston University Birmingham B4 7ET, UK.
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

17 - Optimal Perceptron Learning: an On-line Bayesian Approach
pp 379-398
- By Sara Solla, Physics and Astronomy, Northwestern University, Evanston, IL 60208,; Physiology, Northwestern University Medical School, Chicago, IL 60611, USA, Ole Winther, CONNECT, The Niels Bohr Institute, 2100 Copenhagen Ø, Denmark; Theoretical Physics II, Lund University, S-223 62 Lund, Sweden
- - Get access
    
    Check if you have access via personal or institutional login
    
    Log in Register
- Export citation

Metrics

Altmetric attention score

Total number of HTML views: 0

Total number of PDF views: 0 *

Loading metrics...

Total views: 0 *

Loading metrics...

* Views captured on Cambridge Core between #date#. This data will be updated every 24 hours.

Usage data cannot currently be displayed.

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.

On-Line Learning in Neural Networks

This Book has been cited by the following publications. This list is generated based on data provided by Crossref.

Book description

Reviews

Refine List

Actions for selected content:

Contents

Frontmatter
pp i-iv

Contents
pp v-vi

Acknowledgements
pp vii-viii

List of contributors
pp ix-x

Foreword
pp 1-2

1 - Introduction
pp 3-8

2 - On-line Learning and Stochastic Approximations
pp 9-42

3 - Exact and Perturbation Solutions for the Ensemble Dynamics
pp 43-62

4 - A Statistical Study of On-line Learning
pp 63-92

5 - On-line Learning in Switching and Drifting Environments with Application to Blind Source Separation
pp 93-110

6 - Parameter Adaptation in Stochastic Optimization
pp 111-134

7 - Optimal On-line Learning in Multilayer Neural Networks
pp 135-164

8 - Universal Asymptotics in Committee Machines with Tree Architecture
pp 165-182

9 - Incorporating Curvature Information into On-line Learning
pp 183-208

10 - Annealed On-line Learning in Multilayer Neural Networks
pp 209-230

11 - On-line Learning of Prototypes and Principal Components
pp 231-250

12 - On-line Learning with Time-Correlated Examples
pp 251-278

13 - On-line Learning from Finite Training Sets
pp 279-302

14 - Dynamics of Supervised Learning with Restricted Training Sets
pp 303-344

15 - On-line Learning of a Decision Boundary with and without Queries
pp 345-362

16 - A Bayesian Approach to On-line Learning
pp 363-378

17 - Optimal Perceptron Learning: an On-line Bayesian Approach
pp 379-398

Metrics

Altmetric attention score

Full text views

Book summary page views

Accessibility standard: Unknown

Why this information is here

Accessibility Information

On-Line Learning in Neural Networks

Book description

Reviews

Refine List

Actions for selected content:

Save Search

Contents

Metrics

Altmetric attention score

Full text views

Book summary page views

Accessibility standard: Unknown

Why this information is here

Accessibility Information