Learning: From Synapses to Systems

WS 04/05: Irchel Campus, 22 F 62/68, Thursdays 17:15- 19:00, Sebastian Seung

The class will cover mathematical concepts that lie at the interface between machine learning and theoretical neuroscience. The aim is to bring the audience to the frontiers of research as quickly as possible. Basic knowledge of multivariable calculus, linear algebra, differential equations, and probability theory will be assumed.

There is an accompanying math meeting to brush up on the maths used in the lecture. It takes place on mondays at 10:00 in the Elkroom at INI. There will be exercises mondays at 12:00 in the Elkroom. Please download the exercises from this site in the lecture-schedule. If you would like to be added to the mailing list of the math review meeting or have questions about the exercises, write to Fabian Roth.

Lecture schedule / Reading List

Oct. 21: Neural networks: spiking and nonspiking models
- Definitions of computational neuroscience and neural networks. Classical neural network equations. Integrate-and-Fire model neurons and reduction by the method of averaging.
- Christof Koch, Biophysics of Computation, Section 14.2: 335-341, Oxford University Press, New York, Oxford (1999).
- Bard Ermentrout, Reduction of Conductance-Based Models with Slow Synapses to Neural Nets, Neural Computation. 6,679-695 (1995).
- Lecture notes
- Problem Set 1
- Solutions Problem Set 1
Oct. 28: Simple perceptrons and gradient learning
- The problem of credit assignment. Perceptron learning rule. Convergence theorem. Learning by gradient following. Online learning.
- J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley, 1991, Chapter 5
- Lecture notes: a,b
- Problem Set 2
- Solutions Problem Set 2
Nov. 4: Multilayer perceptrons and backpropagation learning
- Multilayer perceptrons and backpropagation.
- Hertz, Krogh, and Palmer Chapter 6.
- Y. LeCun, L. Bottou, G. B. Orr, K.-R. Muller. Efficient backprop, in G. Orr and K. Muller, Neural networks: tricks of the trade, Springer, 1998.
- Problem Set 3
- Solutions Problem Set 3
Nov. 11: Learning by weight and node perturbation
- Gert Cauwenberghs, A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization, 1993
- A. Dembo, T. Kailath, Model-Free Distributed Learning, IEEE Transactions of Neural Networks, 1, 58-70 (1990)
- M. Jabri, B. Flower, Weight Perturbation: An Optimal Architecture and Learning Technique for Analog VLSI Feedforward and Recurrent Multilayer Networks, IEEE Transactions of Nerual Networks, 3, 154-157 (1992)
- R. H. R. Hahnloser, A. A. Kozhevnikov, M. S. Fee. An ultra-sparse code underlies the generation of neural sequences in a songbird, Nature, 419, 65-70 (2002).
- I. R. Fiete, R. H. R. Hahnloser, M. S. Fee, H. S. Seung, Temporal Sparseness of the Premotor Drive Is Important for Rapid Learning in a Neural Network Model of Birdsong, J. Neurophysiol, 92, 2274-2282(2004)
- Problem Set 4
- Solutions Problem Set 4
Nov. 18: Matching and maximizing
- L. P. Sugrue, G. S. Corrado, W. T. Newsome, Matching behavior and the representation of value in the parietal cortex, Science, 304, 1782-1787(2004)
- R. J. Herrnstein, D. Prelec, Melioration, A Theory of Distributed Choice, Journal of Economic Perspecives, 5, 137-156(1991)
- Problem Set 5
- Solutions Problem Set 5
Nov. 25: Markov decision processes and policy gradient learning, hedonistic synapses
- X. Xie, S. Seung, Learning in neural networks by reinforcement of irregular spiking, Physical Review E, 69, 041909(2004)
- R. J. Williams, Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning, Machine Learning, 8, 229-256(1992)
- J. Baxter, P. L. Bartlett, Infinite-Horizon Policy-Gradient Estimation, Journal of Artificial Intelligence Research, 15 319-350(2001)
- D. Prlec, H. S. Seung, Matching and maximizing are two ends of a spectrum of gradient learning algorithms, 2004
- H. S. Seung, Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission, Neuron, 40, 1063-1073(2003)
- Problem Set 6
- Solutions Problem Set 6
Dec. 2: Neural basis of reward learning
- W. Schultz, P. Dayan, P. R. Montague, A Neural Substrate of Prediction and Reward, Science, 275(1997) 1593-9. Review
- W. Schultz, Predictive reward signal of dopamine neurons, J Neurophysiol., 80(1998), 1-27. Review
- L. Kaelbling, M. Littman, A. Moore, Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research, 4(1996), 237-285
- Problem Set 7
- Solutions Problem Set 7
Dec. 9: Artificial economies as learning systems
- E. Baum, Toward a Model of Intelligence as an Economy of Agents, Machine Learning, 35, 155-185(1999)
- Hal R. Varian, Microeconomic Analysis, Chapter 17, Exchange, W. W. Norton & Company; 1992
- No problem set this week - enjoy your christmas shopping
Dec. 16: Hybrid analog-digital computation
- Richard H. R. Hahnloser, Rahul Sarpeshkar, Misha A. Mahowald, Rodney J. Douglas, H. Sebastian Seung, Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit, Nature, 405, 947-951(2000)
- Problem Set 8
- Solutions Problem Set 8
Dec. 23: No Lecture
Jan. 13: Nonlinear network theory
- Richard H. R. Hahnloser, H. Sebastian Seung, J.-J. Slotine, Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks , Neural Computation
- Problem Set 9 (corrected)
Jan. 20: Combining bottom-up and top-down processes in perception
- James L. McClelland, David E. Rumelhart, An Interactive Activation Model of Context Effects in Letter Perception: Part 1. An Account of Basic Findings, Psychological Review, 88, 375-407(1981)
Jan. 27: Learning perceptual hierarchies, last lecture
- Daniel D. Lee, H. Sebastian Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401, 788-791(1999)