Adaptive sensory-motor encoder for visual or acoustic prosthesis

• US 6,400,989 B1
• Filed: 05/30/2000
• Issued: 06/04/2002
• Est. Priority Date: 02/21/1997
• Status: Expired due to Term

• Current Assignee: Pixium Vision SA
• Original Assignee: Intelligent Implants GmbH
• Inventors: Eckmiller, Rolf
• Primary Examiner(s): Jastrzab, Jeffrey R.
• Application Number:
  US09/367,030
• Time in Patent Office: 735 Days
• Field of Search: 607/53-57, 623/4, 623/10
• US Class Current: 607/54
• CPC Class Codes: A61N 1/0543 Retinal electrodes
• CPC Class Codes: A61N 1/36038 Cochlear stimulation
• CPC Class Codes: A61N 1/36046 of the eye

Abstract

The invention describes an adaptive, sensory-motor encoder for a visual prosthesis or for an acoustic prosthesis and equipped with a central control unit for signal processing functions, monitoring functions, control functions and external intervention functions as well as with a group of adaptive spatio-temporal filters for the conversion of sensor signals into stimulation impulse sequences, whereby a bi-directional interface is provided for coupling the encoder with an implantable microstructure (2) for stimulation of nerve or glial tissue on the one hand, and on the other hand for function monitoring of brain function.

Claims

1. A visual or acoustic prosthesis comprising:

an adaptive sensory-motor encoder, comprising;

a central control unit for performing signal processing functions, monitoring functions, control functions, and external pick-up functions, the central control unit including a group of adaptive spatio-temporal filters for converting sensory signals into stimulation pulse sequences;

an implantable microstructure for providing stimulation to nerve or glial tissue and for functional monitoring of neural functions; and

a bi-directional interface coupling the encoder to the microstructure, through which at least one of stimulation and control signals are provided to the microstructure and monitoring signals are provided to the encoder.

2. The prosthesis according to claim 1, wherein the control unit includes in a perception-based dialog process receptive field (RF) filters, said RF filters comprising adjustable spatio-temporal filters with receptive field properties.

3. The prosthesis according to claim 2, wherein the RF filters are associated with time delay elements for a relative temporal sequence of impulse sequences produced.

4. The prosthesis according to claim 2, wherein a monitor is provided that displays imaging functions produced by the RF filters at least approximately inverted and optically or acoustically.

5. The prosthesis according to claim 1, further comprising:

a simulated eye movement system, including head and eye movement detectors.

6. The prosthesis according to claim 1, wherein said implantable microstructure comprises:

a device for application of an active substance, the device being controlled by said central control unit;

wherein the implantable microstructure records and transmits neural activity, in the form of said monitoring signals, to a monitoring system of said central control unit.

7. The prosthesis according to claim 1, wherein said central control unit further comprises:

an adaptive pre-programming module, for simplification of image or sound patterns.

8. The prosthesis according to claim 1, further comprising:

a portable signal transmitter for relaying a position of an object in space, determined by the encoder, to an appropriate sensory organ.

9. The prosthesis according to claim 1, wherein said central control unit includes at least one pattern recognition program.

10. The prosthesis according to claim 1, further comprising:

feedback apparatus for training said encoder.

11. The prosthesis according to claim 10, wherein said feedback apparatus comprises:

means for inputting evaluation inputs, the evaluation inputs representing a comparison between an original stimulus and a detected stimulus; and

a dialog module, the dialog module receiving the evaluation inputs and providing feedback to the encoder for purposes of adaptation.

12. The prosthesis according to claim 11, wherein said dialog module comprises a neural network.

13. An adaptive sensory-motor encoder for use in a visual or acoustic prosthesis, the prosthesis including an implantable microstructure for stimulation of neural or glial tissue and for monitoring of neural function and a bi-directional interface coupled to the implantable microstructure, the bi-directional interface facilitating the transmission of at least one of stimulation and control signals from the encoder to the microstructure and of monitoring signals from the microstructure to the encoder, the encoder comprising:

a central control unit performing signal processing functions, monitoring functions, control functions, and external pick-up functions, wherein the central control unit includes a group of adaptive spatio-temporal filters, the adaptive spatio-temporal filters converting sensory signals into stimulation pulse sequences.

14. The encoder according to claim 13, wherein the control unit includes in a perception-based dialog process receptive field (RF) filters, said RF filters comprising adjustable spatio-temporal filters with receptive field properties.

15. The encoder according to claim 14, wherein the RF filters are associated with time delay elements for a relative temporal sequence of impulse sequences produced.

16. The encoder according to claim 14, wherein a monitor is provided that displays imaging functions produced by the RF filters at least approximately inverted and optically or acoustically.

17. The encoder according to claim 13, further comprising:

a simulated eye movement system, the simulated eye movement system receiving input from head and eye movement detectors.

18. The encoder according to claim 13, wherein said central control unit controls a device for application of an active substance, the device being part of said implantable microstructure;

and wherein the central control unit includes a monitoring system that receives indications of neural activity from said implantable microstructure via said monitoring signals.

19. The encoder according to claim 13, wherein said central control unit further comprises:

an adaptive pre-programming module, for simplification of image or sound patterns.

20. The encoder according to claim 13, wherein said central control unit includes at least one pattern recognition program.

21. A method for use with an adaptive sensory-motor encoder for a visual or acoustic prosthesis, the encoder comprising a central control unit performing signal processing functions, monitoring functions, control functions and external pick-up functions, as well as including a group of adaptive spatio-temporal filters, the prosthesis further including a microstructure for stimulation and monitoring of neural activity and a bi-directional interface coupling the encoder with the microstructure for transmission of at least one of control and stimulation signals from the encoder to the microstructure and of monitoring signals from the microstructure to the encoder, the method comprising the steps of:

supplying a minimum single-channel evaluation entry unit for individual adjustment of said signal processing functions by means of a dialog process, the evaluation entry unit using a subjective evaluation vector, where the evaluation vector represents similarity of a currently perceived pattern to a desired pattern;

transmitting the evaluation vector to a parameter adjustment system, to produce suitable sequences of parameter vectors, said parameter adjustment system comprising a neural network with non-monitored adaptation rules; and

generating, by said evaluation entry unit, a parameter vector at a multi-channel output for a respective signal processing function to be adjusted.

22. The method according to claim 21, wherein said signal processing functions comprise receptive field (RF) filters.

23. The method according to claim 22, further comprising the step of:

establishing, by the evaluation entry unit, appropriate pattern sequences for an adaptation process in a decision system for individual or group optimization of said RF filters.

24. The method according to claim 23, wherein said step of establishing comprises the step of:

generating internally stored sequences of parameter vectors for establishing typical RF filter functions.

25. The method according to claim 24, wherein said RF filters have a spatio-temporal function space that includes a function space of the receptive field properties of neurons at a site at which said microstructure is implanted.

26. The method according to claim 22, further comprising the steps of:

generating at least one signal by the RF filters;

emitting said at least one signal at said microstructure to elicit an actual perception; and

simultaneously passing an associated desired pattern to an output unit for perception by a human.

27. The method according to claim 26, wherein said microstructure includes a given number of micro-contacts, and further comprising the step of:

conducting, by an RF filter, said at least one signal to several locally adjacent micro-contacts for functionally increasing the number and definition of selectively reachable stimulation sites.

28. The method according to claim 26, wherein said at least one signal comprises at least two impulse signals, and further comprising the step of:

varying the at least two impulse signals for the purpose of shifting a focus of stimulation and varying stimulation sites.

29. The method according to claim 28, wherein said step of varying the at least two impulse signals comprises the step of:

effecting a perception-based dialog with a being in which the microstructure is implanted, for facilitating shifting of the stimulation focus so as to lead to selective and well-defined neural stimulation, the step of effecting a perception-based dialog including a step of utilizing optimization means in the central control unit.

30. The method according to claim 29, wherein said step of utilizing optimization means includes the step of utilizing a neural network.

31. The method according to claim 26, wherein said step of generating at least one signal comprises the step of generating at least one impulse signal.

32. The method according to claim 31, further comprising the steps of:

comparing recorded neural impulses to said at least one impulse signal;

distinguishing spontaneously occurring neural impulses from those produced in response to said at least one impulse signal; and

improving selectivity and biocompatibility of neural stimulation effected by said at least one impulse signal using results of said steps of comparing and distinguishing.

33. The method according to claim 21, further comprising the steps of:

detecting head movements and undesirable real eye movements;

simulating eye movements by means of at least one of electronic image pattern shifts, optical variation of direction of vision, and movement of photosensors;

using the detected head and real eye movements, as well as simulated eye movements produced by said step of simulating eye movements, by means of movement control or adjustment, quick and slow eye movements are produced for purposes of pattern recognition, tracking of moving objects, and fast circumspection.

34. The method of claim 33, further comprising the step of:

using said quick and slow eye movements to compensate for undesired eye movements or perceptions of apparent motion.

35. The method of claim 33, further comprising the step of:

generating compensatory eye movements with reference to the human vestibulo-ocular reflex for situational stabilization of an image pattern in the presence of normal head and upper body movements.

36. The method of claim 33, further comprising the step of:

facilitating technical adaptation of a brightness operating range resulting from a function range of a photosensor array extending over several brightness decades, said step of facilitating including the step of;

selecting an operating range for the encoder with respect to magnitude and adaptation brightness.

37. The method of claim 36, wherein said step of selecting an operating range comprises the steps of:

non-linearly evaluating sections of said function range; and

composing said operating range from the resulting non-linearly evaluated sections of the function range.

38. The method of claim 37, further comprising the step of:

transposing visual scenes with local subsets of varying brightness into a common operating range of the encoder, using a suitable imaging system.

39. The method of claim 37, further comprising the step of:

if necessary, shifting RF filter functions for stimulation of perception a bright or dark adapted range using a brightness operating range of the encoder.

40. The method of claim 39, further comprising the step of:

rapidly changing the brightness operating range and the associated RF filter function shift automatically during simulated eye movements or during pattern recognition.

41. The method of claim 36, further comprising the steps of:

sharply adjusting first areas of a visual field by variation of accommodation;

storing the resulting sharply-adjusted first areas;

sharply adjusting, as an image, second areas of said visual field, whereby said initial areas become less sharp; and

blending the stored sharply-adjusted first areas into the image in place of the first areas, which became less sharp in said step of sharply adjusting second areas.

42. The method of claim 36, wherein the method repeats itself cyclically.

43. The method of claim 21, further comprising the step of:

relaying a position of an object in space, determined by said encoder, to an appropriate sensory organ using a portable signal transmitter.

44. The method of claim 21, further comprising the steps of:

executing, in the encoder, at least one pattern recognition program, in connection with automatically running, simulated eye movements;

warning an implant carrier of obstacles or hazards detected based on results of said step of executing; and

reporting a type and position of a technically identified pattern or object detected based on results of said step of executing.