Brown University Neural Prosthesis Project

Humans and other animate creatures survive in an environment by continually sensing and acting. Illness or injury however may impair or destroy the neural pathways connecting the brain with the external world. These include auditory and visual impairments as well as motor impairment due to stroke, spinal cord injury, Amyotrophic Lateral Sclerosis, or Multiple Sclerosis. There are 250,000 cases of spinal cord injury alone in the United States of America with 11,000 new cases each year.

Research on neural prostheses seeks an engineering solution to restoring lost function by providing new, alternate, pathways which restore, to varying degrees, the ability to sense and act on the world. While neural prosthetic research takes many forms, our work focuses on direct cortical control of external devices such as computer displays and robots.

Building a direct, artificial, connection between the brain and the world, requires answers to the following questions:

1. What ``signals'' can we measure from the brain? From what regions? With what technology?
2. How is information represented (or encoded) in the brain?
3. What algorithms can we use to infer (or decode) the internal ``state'' of the brain?
4. How can we build practical interfaces that exploit the available technology?
Our work exploits neural activity recorded in primary motor cortex using an array of chronically implanted microelectrodes. We adopt a Bayesian formulation of the encoding/decoding problem and have developed a variety of real-time methods for reconstructing hand motion from neural activity. This reconstruction is sufficiently accurate to permit the control of unconstrained 2D cursor movement or simple robotic functions.

Please see the papers and talks below for details of our research effort.


Brain-Machine Interface Research Projects

Statistical Modeling of Neural Activity in Motor Cortex

Statistical learning and probabilistic inference techniques are used to infer the hand position of a subject from multi-electrode recordings of neural activity in motor cortex. First, an array of electrodes provides training data of neural firing conditioned on hand kinematics. We learn a non-parametric representation of this firing activity using a Bayesian model and rigorously compare it with previous models using cross-validation.

Bayesian Inference of Hand Motion from Multi-electrode Recordings

We infer a posterior probability distribution over hand motion conditioned on a sequence of neural test data using Bayesian inference. We are exploring a variety of methods from Kalman filtering to particle filtering. In the latter case, the learned firing models of multiple cells are used to define a non-Gaussian likelihood term which is combined with a prior probability for the kinematics. A particle filtering method is used to represent and propagate the posterior distribution over time. The approach is compared with traditional linear filtering methods; the results suggest that it may be appropriate for neural prosthetic applications.

Neural Control of Robotic Devices

We are working to directly connect the central nervous system of humans to robotic devices to restore lost function. Our current focus is on the neural control of semi-autonomous mobile robots.

Current ollaborators

John Donoghue, Dept. of Neuroscience, Brown University.
Leigh Hochberg
John Simeral,
Gregrory Shakhnarovich, Department of Computer Science, Brown University.
Sung-Phil Kim, Department of Computer Science, Brown University.
Carlos Vargas-Irwin
Frank Wood, Department of Computer Science, Brown University.
Elie Bienenstock, Depts. of Applied Math and Neuroscience, Brown University.

Publications

Bayesian population coding of motor cortical activity using a Kalman filter.
Wu, W., Gao, Y., Bienenstock, E., Donoghue, J. P., Black, M. J.,
Neural Computation, 18:80-118, 2005.
(pdf preprint), (pdf from publisher)

Assistive technology and robotic control using MI ensemble-based neural interface systems in humans with tetraplegia,
Donoghue, J. P., Nurmikko, A., Black, M., J., and Hochberg, L.,
Journal of Physiology, Special Issue on Brain Computer Interfaces, 579:603-611, 2007.
(pdf preprint) (pdf from publisher)

Neuromotor prosthesis development,
Donoghue, J.P., Hochberg, L.R., Nurmikko, A.V., Black, M.J., Simeral, J.D., and Friehs, G.,
Medicine & Health Rhode Island, Vol. 90, No. 1, pp. 12-15, Jan. 2007.
(pdf reprint)

Probabilistically modeling and decoding neural population activity in motor cortex
Black, M. J. and Donoghue, J. P.,
in Toward Brain-Computer Interfacing, G. Dornhege, J. del R. MillŽan, T. Hinterberger, D. McFarland, K.-R. Muller (eds.), MIT Press, pp. 147-159, 2007.

Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia,
Kim, S.-P., Simeral, J., Hochberg, L., Donoghue, J. P., Friehs, G., Black, M. J.,
The 3rd International IEEE EMBS Conference on Neural Engineering, pp. 486-489, May 2-5, 2007.
(pdf)

Decoding grasp aperture from motor-cortical population activity,
Artemiadis, P., Shakhnarovich, G., Vargas-Irwin, C., Black, M. J., Donoghue, J. P.,
The 3rd International IEEE EMBS Conference on Neural Engineering, pp. 518-521, May 2-5, 2007.
(pdf)

Nonlinear physically-based models for decoding motor-cortical population activity,
Shakhnarovich, G., Kim, S.-P. and Black, M. J.,
to appear: Advances in Neural Information Processing Systems, NIPS-2006.
(pdf)

A non-parametric Bayesian approach to spike sorting,
Wood, F., Goldwater, S., and Black, M. J.,
International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, Aug-Sep, pp. 1165-1169, 2006.
(pdf)

Statistical analysis of the non-stationarity of neural population codes.
Kim, S.-P., Wood, F., Fellows, M., Donoghue, J. P., Black, M. J.,
BioRob 2006, The first IEEE / RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics,  pp. 295-299, Pisa, Italy, Feb. 2006.
(pdf)

Modeling neural population spiking activity with Gibbs distributions.
Wood, F., Roth, S., and Black, M. J.,
Advances in Neural Information Processing Systems,  8, pp. 1537-1544, 2005.
(pdf)

Inferring attentional state and kinematics from motor cortical firing rates,
Wood, F., Prabhat, Donoghue, J. P., Black, M. J.,
Proc. IEEE Engineering in Medicine and Biology Society, pp 1544-1547, Sept. 2005.
(pdf)

Motor cortical decoding using an autoregressive moving average model,
Fisher, J and Black, M. J.,
Proc. IEEE Engineering in Medicine and Biology Society, pp. 1469-1472, Sept. 2005.
(pdf)

Development of neural motor prostheses for humans.
Donoghue, J., Nurmikko, A., Friehs, G., Black, M.,
(Invited paper) Advances in Clinical Neurophysiology (Supplements to Clinical Neurophysiology, Vol. 57) Editors: M. Hallett, L.H. Phillips, II, D.L. Schomer, J.M. Massey. 2004.
(pdf).

On the variability of manual spike sorting,
Wood, F., Black, M. J., Vargas-Irwin, C., Fellows, M., Donoghue, J. P.,
IEEE Trans. Biomedical Engineering, 51(6):912-918, June 2004.
(pdf).

Modeling and decoding motor cortical activity using a switching Kalman filter,
Wu, W., Black, M. J., Mumford, D., Gao, Y., Bienenstock, E., Donoghue, J. P.,
IEEE Trans. Biomedical Engineering, 51(6):933-942, June 2004.
(pdf).

A switching Kalman filter model for the motor cortical coding of hand motion,
Wu, W., Black, M. J., Mumford, D., Gao, Y., Bienenstock, E., Donoghue, J. P.,
Proc. IEEE Engineering in Medicine and Biology Society, pp. 2083-2086, Sept. 2003
(pdf).

Connecting brains with machines: The neural control of 2D cursor movement,
Black, M. J., Bienenstock, E., Donoghue, J. P., Serruya, M., Wu, W., Gao, Y.,
1st International IEEE/EMBS Conference on Neural Engineering, pp. 580-583, Capri, Italy, March 20-22, 2003.
(pdf).

A quantitative comparison of linear and non-linear models of motor cortical activity for the encoding and decoding of arm motions,
Gao, Y., Black, M. J., Bienenstock, E., Wu, W., Donoghue, J. P.,
1st International IEEE/EMBS Conference on Neural Engineering, pp. 189-192, Capri, Italy, March 20-22, 2003.
(pdf).

Neural decoding of cursor motion using a Kalman filter,
Wu, W., Black, M. J., Gao, Y., Bienenstock, E., Serruya, M., Shaikhouni, A., Donoghue, J. P.,
Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun and K. Obermayer (Eds.), MIT Press, pp. 117-124, 2003.
(pdf).

Inferring hand motion from multi-cell recordings in motor cortex using a Kalman filter,
Wu, W., Black, M. J., Gao, Y., Bienenstock, E., Serruya, M., and Donoghue, J. P.,
SAB'02-Workshop on Motor Control in Humans and Robots: On the Interplay of Real Brains and Artificial Devices, August 10, 2002, Edinburgh, Scotland (UK), pp. 66-73.
(abstract), (postscript), (pdf).

Probabilistic inference of arm motion from neural activity in motor cortex,
Gao, Y., Black, M. J., Bienenstock, E., Shoham, S., Donoghue, J.,
Advances in Neural Information Processing Systems 14, T. G. Dietterich, S. Becker, and Z. Ghahramani (Eds.), pp. 221-228, MIT Press, 2002.
(abstract), (postscript), (pdf).

Encoding/decoding of arm kinematics from simultaneously recorded MI neurons,
Gao, Y., Bienenstock, E., Black, M., Shoham, S., Serruya, M., Donoghue, J.,
Society for Neuroscience Abst. Vol. 27, Program No. 572.14 2001.
(abstract)

Related Talks

Models of Neural Coding in Motor Cortex and their Application to Neural Prostheses. plenary talk, Workshop on Neural Coding, Mathematical Biosciences Institute, The Ohio State University, February 2003.

Connecting Brains with Machines: Towards the Neural Control of 2D Cursor Movement. Invited talk, AI Lab, MIT, October 2002.

The Man Who Mistook His Computer for a Hand: Neural Control of Robotic Devices. Invited talk, Center for Autonomous Systems, KTH, Stockholm.

Related Teaching

Topics in Brain Computer Interfaces (CS295-07)

Acknowledgement

This material was supported by the National Science Foundation under Grant No. 0113679, and NIH-NINDS, R01 NS 50967-01, CRCNS: Learning the Neural Code for Prosthetic Control

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.


Home