The "echo state" approach looks at RNNs from a new angle. Large RNNs are interpreted as "reservoirs" of complex, excitable dynamics. Output units "tap" from this reservoir by linearly combining the desired output signal from the rich variety of excited reservoir signals. This idea leads to training algorithms where only the network-to-output connection weights have to be trained. This can be done with known, highly efficient linear regression algorithms.

Numerous dynamical systems have been learnt easily by echo state networks, which were difficult to learn with existing methods. They include (long) periodic sequence generators, multistable switches, tunable frequency generators, frequency measurement devices, controllers for nonlinear plants, long short-term memories, dynamical pattern recognizers, and notably, long-term predictors of chaotic attractors.

The basic idea of having a dynamical "reservoir", from which target dynamics of interest are read out by trainable mechanisms, has been independently explored under the name of "Liquid State Machines" (LSM) by Wolfgang Maass et al. Their main research objective is modeling of biological systems; therefore, the LSM approach typically employs reservoir networks (called "liquids") made from more biologically adequate, spiking neuron models.

Summer 2005: ESN special session at IJCNN 2005, organized by Jose Principe.

Fall 2006: ESN workshop at NIPS 2006

Spring 2007: Special issue of Neural Networks on Echo State Networks and Liquid State Machines

Spring 2007: Special session on Reservoir Computing at ESANN 2007

Fall 2007: Jacobs University graduate seminar wins an international financial forecasting competition using ESNs.

Fall 2008: The ORGANIC European FP7 project on speech recognition with reservoir computing systems is granted; Jacobs University is the coordinator.

International patents are pending for the ESN method, claimed by the Fraunhofer Institute for Intelligent Analysis and Information Systems (Fraunhofer IAIS).

A Scholarpedia article for a first impression.

Extensive survey paper: M. Lukosevicius and H. Jaeger (2009), Reservoir Computing Approaches to Recurrent Neural Network Training. Computer Science Review 3(3), 127-149 (preprint pdf)

Highlight paper: H. Jaeger and H. Haas, Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science 304, 2 April 2004, pp. 78-80 (preprint pdf) (Matlab code zip)

The 2009 PhD thesis by David Verstraeten (University Gent) provides an extensive and very accessible overview of the state of the art with particular emphasis of application-relevant insights - indispensable reading for serious end-users.

A survey (pdf) of recursive least squares (RLS) methods for ESN training, helpful for online adaptive signal processing applications (Master thesis of A. U. Kücükemre, University of Applied Sciences Bonn-Rhein-Sieg 2006)

The web portal www.reservoir-computing.org is jointly maintained by the leading reservoir computing groups in Europe and provides information on

• programming toolboxes,
• research groups and publications,
• projects and open positions,
• the reservoir computing mailing list (moderated).