Temporal winner takes all spiking neuron network sensory processing apparatus and methods

• US 9,070,039 B2
• Filed: 02/01/2013
• Issued: 06/30/2015
• Est. Priority Date: 02/01/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Richert, Micah
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US13/757,607
• Publication Number:
  US 20140219497A1
• Time in Patent Office: 879 Days
• Field of Search: 382/100, 382/107, 382122-123, 382155-157, 382/162, 382/168, 382/173, 382/181, 382/191, 382/232, 382/254, 382/263, 382273-276, 382/305, 382/312, 382/103, 382/291, 706/20, 706/48, 706/18, 706/26, 706/34, 235/449
• US Class Current: 1/1
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06V 10/451 with interaction between the filter responses, e.g. cortical complex cells

Abstract

Apparatus and methods for contrast enhancement and feature identification. In one implementation, an image processing apparatus utilizes latency coding and a spiking neuron network to encode image brightness into spike latency. The spike latency is compared to a saliency window in order to detect early responding neurons. Salient features of the image are associated with the early responding neurons. An inhibitory neuron receives salient feature indication and provides inhibitory signal to the other neurons within an area of influence of the inhibitory neuron. The inhibition signal reduces probability of responses by the other neurons to stimulus that is proximate to the feature thereby increasing contrast within the encoded data. The contrast enhancement may facilitate feature identification within the image. Feature detection may be used for example for image compression, background removal and content distribution.

Claims

1. A non-transitory computer-readable apparatus configured to store at least one computer process thereon, the computer process configured to detect an object in a sensory input, the computer process comprising a plurality of instructions configured to, when executed:

based on at least one excitatory signal, generate an inhibitory signal configured to suppress generation of another excitatory signal, and thereby enable the object detection;

wherein;

the sensory input comprises a plurality of spikes associated with a first sensory modality and a second sensory modality;

the at least one excitatory signal comprising one or more pulses generated based on a portion of the sensory input comprising first representation of the object based on the first sensory modality;

the another excitatory signal comprising another one or more pulses configured to be generated based on another portion of the sensory input comprising second representation of the object based on the second sensory modality; and

the first representation of the object being characterized by a parameter having a value that is different from a value of the parameter associated with the sensory input of the first sensory modality in an area proximate to the object.

2. The apparatus of claim 1, wherein:

the first sensory modality comprises an infrared spectrum image comprising the first representation of the object;

the second sensory modality comprises a visible-light image comprising the second representation of the object;

the one or more pulses are characterized by a first range of latencies; and

the another one or more pulses are characterized by a second range of latencies, the latencies within the second range being greater in magnitude than the latencies within the first range.

3. The apparatus of claim 2, wherein:

the parameter is configured to characterize a luminance of pixels within the image; and

individual latencies within the first range of latencies are encoded based on a logarithmic function of pixel luminance.

4. The apparatus of claim 2, wherein the inhibitory signal is effectuated by an inhibition trace, the inhibition trace being incremented responsive to a pulse latency of the one or more pulses being within a latency window.

5. The apparatus of claim 2, wherein:

the inhibitory signal is generated based on another sensory input of the first sensory modality comprising another representation of the object, the another sensory input preceding the sensory input; and

a time interval between the sensory input and the another sensory input is greater than the first range of latencies.

6. The apparatus of claim 1, wherein:

the first sensory modality comprises an audio signal comprising the first representation of the object;

the second sensory modality comprises a visible-light image comprising the second representation of the object;

the at least one excitatory signal comprises one or more pulses having a first range of latencies associated therewith; and

the another excitatory signal is configured based on a second range of latencies, the latencies within the second range being greater than the latencies within the first range.

7. The apparatus of claim 1, wherein:

the sensory input comprises an image comprising the first representation of the object;

the inhibitory signal is configured to transform the image into a spiking representation comprising at least one feed-forward signal;

the inhibitory signal is characterized by an inhibition area within the image;

the first representation of the object is characterized by an object area; and

the spiking representation is characterized by absence of spikes corresponding to pixels of the image that are outside the object area and within the inhibition area.

8. A method of detecting one or more salient features within an image set with a spiking neuron network, the method comprising:

providing a first stimulus comprising a first spectral parameter of a first image of the image set to a first portion of the network;

providing a second stimulus comprising a second spectral parameter of a second image of the image set to a second portion of the network;

based at least in part on the provision of the first stimulus, causing generation of a plurality of pulses by the first portion of the network, the plurality of pulses encoding the first spectral parameter into a pulse latency;

generating an inhibition signal based at least in part on two or more pulses of the plurality of pulses, the two or more pulses being proximate one another within a time interval; and

based at least in part on the inhibition signal, suppressing responses to the second stimulus by at least one neuron of the second portion.

9. The method of claim 8, wherein:

the providing of the second stimulus is characterized by a stimulus duration, the stimulus duration being greater than the time interval; and

the suppressing of the responses is characterized by an absence of responses from the second portion during the stimulus duration.

10. The method of claim 8, wherein:

the providing of the second stimulus is characterized by a stimulus duration, the stimulus duration being greater than the time interval; and

the suppressing of the responses comprises reducing a number of responses from the second portion during the stimulus duration as compared to a number of responses that are generated by the second portion during the stimulus duration and responsive to the second stimulus.

11. The method of claim 8, wherein:

providing of the second stimulus is characterized by a stimulus duration, the stimulus duration being greater than the time interval; and

the suppressing of the responses comprises reducing a probability of response generation by the at least one neuron within the second portion during the stimulus duration and responsive to the second stimulus.

12. The method of claim 8, wherein:

the providing of the second stimulus causes generation of a burst of pulses by at least one neuron of the network; and

the suppressing of the responses comprises suppressing at least a portion of the pulses within the burst from being generated responsive to the second stimulus.

13. The method of claim 8, wherein the suppressing of the responses is based at least in part on at least one of the two or more pulses.

14. A spiking neuron network processing apparatus configured to process sensory input comprising at least two attributes, the apparatus comprising:

one or more encoders configured to;

receive the sensory input; and

encode the sensory input to produce a plurality of pulses;

a detector bank comprising one or more detectors associated with each of the at least two attributes, each of the one or more detectors configured to;

receive an encoded input for an associated attribute; and

based on the encoded input generate a respective feed-forward signal; and

at least one inhibitory unit configured to;

receive the respective feed-forward signal;

in response to the received feed-forward signal, produce a feedback signal; and

transmit the feedback signal to at least one of the one or more detectors;

wherein the feedback signal is configured to alter a response generation parameter of at least one of the one or more detectors.

15. The apparatus of claim 14, wherein the at least two attributes comprise two different sensory modalities.

16. The apparatus of claim 15, wherein the two different sensory modalities comprise a visible-light image and an audio stream.

17. The apparatus of claim 14, wherein the one or more detectors associated with each of the at least two attributes comprises at least one dedicated detector for each of the at least two attributes.

18. The apparatus of claim 14, wherein the one or more detectors associated with each of the at least two attributes comprises a single detector associated with all of the at least two attributes.

19. The apparatus of claim 14, wherein the feedback signal is configured to reduce the probability of response generation by the at least one of the one or more detectors.

20. The apparatus of claim 14, wherein the feedback signal is configured to increase the probability of response generation by the at least one of the one or more detectors.