Method and system for training a visual prosthesis

• US 7,321,796 B2
• Filed: 04/30/2004
• Issued: 01/22/2008
• Est. Priority Date: 05/01/2003
• Status: Active Grant

• Current Assignee: California Institute of Technology, Doheny Eye Institute
• Original Assignee: California Institute of Technology, Doheny Eye Institute
• Inventors: Humayun, Mark, Fink, Wolfgang
• Primary Examiner(s): Evanisko; George R.
• Application Number:
  US10/837,163
• Publication Number:
  US 20040236389A1
• Time in Patent Office: 1,362 Days
• Field of Search: 607/2, 607/53, 607/54, 607/115, 607/116, 607/141, 623/4.1
• US Class Current: 607/54
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: A61N 1/36046 of the eye

Abstract

A method for training a visual prosthesis includes presenting a non-visual reference stimulus corresponding to a reference image to a visual prosthesis patient. The visual prosthesis including a plurality of electrodes. Training data sets are generated by presenting a series of stimulation patterns to the patient through the visual prosthesis. Each stimulation pattern in the series, after the first, is determined at least in part on a previous subjective patient selection of a preferred stimulation pattern among stimulation patterns previously presented in the series and a fitness function optimization algorithm. The presented stimulation patterns and the selections of the patient are stored and presented to a neural network off-line to determine a vision solution.

Claims

1. A method for training a visual prosthesis comprising:

presenting a non-visual reference stimulus corresponding to a reference image to a patient having a visual prosthesis, the visual prosthesis including a plurality of electrodes;

generating a training data set by presenting a series of stimulation patterns to the patient through the visual prosthesis and receiving a corresponding series of subjective patient selections of a preferred stimulation pattern among stimulation patterns in the series of stimulation patterns previously presented to the patient until the patient identifies a successful stimulation pattern, and storing the presented stimulation patterns and the received selections of the patient;

wherein at least one stimulation pattern in the series of stimulation patterns is determined at least in part based on a fitness function optimization algorithm and at least one previously received subjective patient selection in the series of subjective patient selections; and

after identifying the successful stimulation pattern, presenting off-line to a neural network at least the reference image and the successful stimulation pattern from the training data set to determine a vision solution.

2. The method of claim 1 wherein the retinal prosthesis is a retinal implant.

3. The method of claim 1 further comprising:

generating additional training data sets based on additional reference images; and

presenting off-line to the neural network at least reference images and successful stimulation patterns from the additional training data sets.

4. The method of claim 3, comprising utilizing a same fitness function optimization algorithm for determining each series of stimulation patterns corresponding to each given reference image.

5. The method of claim 3, comprising changing the fitness function optimization algorithm between presentation of each series of stimulation patterns corresponding to different reference images.

6. The method of claim 1, comprising utilizing a simulated annealing algorithm as the fitness function optimization algorithm.

7. The method of claim 1, comprising utilizing a genetic algorithm as the fitness function optimization algorithm.

8. The method of claim 1, comprising applying the fitness function optimization algorithm to determine a subsequent stimulation pattern in the series.

9. The method of claim 1, comprising, subsequent to the off-line presentation to the neural network, providing the neural network to the patient to control the visual prosthesis.

10. The method of claim 1, wherein the successful stimulation pattern comprises a stimulation pattern that generates a patient perception that matches the reference image.

11. The method of claim 1, wherein the successful stimulation pattern comprises a stimulation pattern that the patient determines to be a best fit to the reference image.

12. A system for training a visual prosthesis comprising:

a) a processor configured to

i) present a non-visual reference stimulus corresponding to a reference image to a patient having a visual prosthesis via a non-visual stimulus tool, the visual prosthesis including a plurality of electrodes;

ii) generate a training data set by presenting a series of stimulation patterns to the patient through the visual prosthesis and receiving a corresponding series of subjective selections of a preferred stimulation pattern among stimulation patterns in the series of stimulation patterns previously presented to the patient until the patient identifies a successful stimulation pattern, and storing the presented stimulation patterns and the selections of the patient, wherein at least one stimulation pattern in the series of stimulation patterns is determined at least in part based on a fitness function optimization algorithm and at least one previously received subjective patient selection in the series of subjective patient selections; and

iii) after identifying the successful stimulation pattern, present off-line to a neural network at least the reference image and the successful stimulation pattern from the training data set to determine a vision solution; and

b) a user input for selecting between the presented stimulation patterns.

13. The system of claim 12 wherein the processor is configured to present the series of stimulation patterns to the patient through a retinal implant.

14. The system of claim 12 wherein the processor is further configured to:

generate additional training data sets based on additional reference images; and

present off-line to the neural network at least reference images and successful stimulation patterns from the additional training data sets.

15. The system of claim 14 wherein the processor is configured to utilize a same fitness function optimization algorithm for determining each series of stimulation patterns corresponding to each given reference image.

16. The system of claim 14, wherein the processor is configured to changing the fitness function optimization algorithm between presentation of each series of stimulation patterns corresponding to different reference images.

17. The system of claim 12, wherein the processor is configured to utilize a simulated annealing algorithm as the fitness function optimization algorithm.

18. The system of claim 12, wherein the processor is configured to utilize a genetic algorithm as the fitness function optimization algorithm.

19. The system of claim 12, wherein the processor is configured to apply the fitness function optimization algorithm to determine a subsequent stimulation pattern in the series.

20. The system of claim 12, comprising the non-visual stimulus tool.

21. The system of claim 20, wherein the non-visual stimulus tool comprises a plurality of pins.

22. The system of claim 21, wherein the pins of the non-visual stimulus tool are configurable to correspond to respective pixels of the reference image.

23. The system of claim 20, wherein the non-visual stimulus tool comprises an audio output.

24. The system of claim 12, wherein the processor is configured to, subsequent to the off-line presentation of the reference image and successful stimulation pattern from the training data set to the neural network, providing the neural network to the patient to control the visual prosthesis.

25. The system of claim 12, wherein the processor is configured to, subsequent to the off-line presentation to the neural network, provide the neural network to the patient to control the visual prosthesis.

26. The system of claim 12, wherein the successful stimulation pattern comprises a stimulation pattern that generates a patient perception that matches the reference image.

27. The system of claim 12, wherein the successful stimulation pattern comprises a stimulation pattern that the patient determines to be a best fit to the reference image.

28. A method for training a retinal implant comprising:

presenting a non-visual reference stimulus corresponding to a reference image to a patient having a retinal implant, the retinal implant including a plurality of electrodes;

generating a training data set by presenting a series of stimulation patterns to the patient through the retinal implant and receiving a corresponding series of subjective patient selections of a preferred stimulation pattern among stimulation patterns in the series of stimulation patterns previously presented to the patient until the patient identifies a successful stimulation pattern, each stimulation pattern in the series after a first stimulation pattern in the series being determined based at least in part on a fitness function optimization algorithm and a previous subjective patient selection of a preferred stimulation pattern among stimulation patterns previously presented, and storing the presented stimulation patterns and the selections of the patient; and

after identification of the successful stimulation pattern, presenting off-line to a neural network the training data set to determine a vision solution.

29. The method of claim 28, wherein the successful stimulation pattern comprises a stimulation pattern that generates a patient perception that matches the reference image.

30. The method of claim 28, wherein the successful stimulation pattern comprises a stimulation pattern that the patient determines to be a best fit to the reference image.

31. A computer readable medium encoding instructions for causing a computer to carry out a method comprising:

presenting a non-visual reference stimulus corresponding to a reference image to a patient having a visual prosthesis, the visual prosthesis including a plurality of electrodes;

generating a training data set by (i) presenting a series of stimulation patterns to the patient through the visual prosthesis and receiving a corresponding series of subjective selections of a preferred stimulation pattern among stimulation patterns in the series of stimulation patterns previously presented to the patient until the patient identifies a successful stimulation pattern, and storing the presented stimulation patterns and the received selections of the patient;

wherein at least one stimulation pattern in the series of stimulation patterns is determined at least in part based on a fitness function optimization algorithm and at least one previously received subjective patient selection in the series of subjective patient selections; and

presenting off-line to a neural network at least the reference image and the successful stimulation pattern from training data set to determine a vision solution.

32. The computer readable medium of claim 31, wherein the successful stimulation pattern comprises a stimulation pattern that generates a patient perception that matches the reference image.

33. The computer readable medium of claim 31, wherein the successful stimulation pattern comprises a stimulation pattern that the patient determines to be a best fit to the reference image.