Low power neuromorphic voice activation system and method

• US 10,157,629 B2
• Filed: 02/06/2017
• Issued: 12/18/2018
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: Brainchip, Inc. (BrainChip Holdings Ltd.)
• Original Assignee: Brainchip, Inc. (BrainChip Holdings Ltd.)
• Inventors: van der Made, Peter A J, Elkhatib, Mouna, Oros, Nicolas Yvan
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US15/425,861
• Publication Number:
  US 20170229117A1
• Time in Patent Office: 680 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 3/10 Interfaces, programming languages or software development kits, e.g. for simulating neural networks
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/30 using neural networks
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

The present invention provides a system and method for controlling a device by recognizing voice commands through a spiking neural network. The system comprises a spiking neural adaptive processor receiving an input stream that is being forwarded from a microphone, a decimation filter and then an artificial cochlea. The spiking neural adaptive processor further comprises a first spiking neural network and a second spiking neural network. The first spiking neural network checks for voice activities in output spikes received from artificial cochlea. If any voice activity is detected, it activates the second spiking neural network and passes the output spike of the artificial cochlea to the second spiking neural network that is further configured to recognize spike patterns indicative of specific voice commands. If the first spiking neural network does not detect any voice activity, it halts the second spiking neural network.

Claims

1. A neuromorphic system for controlling an electronic device through voice control commands, the system comprising:

a spiking neuron adaptive processor comprising a hierarchical arrangement of a first spiking neural network and a second spiking neural network, wherein the first spiking neural network is configured to;

receive output spikes from an artificial cochlea, wherein the artificial cochlea is configured to produce the output spikes based on a pulse code modulation signal, wherein one or more decimation filters convert a pulse density modulation signal into the pulse code modulation signal, and wherein one or more microphones convert a voice signal into the pulse density modulation signal;

perform a cognitive and learning function that detects spike patterns in the output spikes, wherein the spike patterns indicate a presence of a voice activity in the output spikes; and

activate the second spiking neural network through a digital interface from a low energy halted or off state based on the presence of the voice activity in the output spikes.

2. The system of claim 1, wherein the second spiking neural network is configured to perform a second cognitive and learning function that detects the spike patterns in the output spikes from the artificial cochlea are indicative of specific voice commands.

3. The system of claim 1, wherein the electronic device is a portable wireless device.

4. The system of claim 1, wherein the electronic device comprises one or more digital microphones, one or more decimation filters coupled to the digital microphones and the artificial cochlea, and the hierarchical arrangement of the first spiking neural network and the second spiking neural network.

5. The system of claim 1, wherein the first spiking neural network and the second spiking neural network comprise a plurality of artificial neurons interconnected with a plurality of synapses.

6. The system of claim 1, wherein the second spiking neural network comprises a plurality of artificial neurons interconnected with a plurality of synapses, wherein the second spiking neural network is configured to:

recognize the spike patterns as indicative of specific voice commands, wherein the recognizing comprises;

receiving the output spikes from the first spiking neural network representative of the voice activity;

associating the output spikes with the specific voice commands by activating a portion of a plurality of artificial neurons within the second spiking neural network in response to triggering synaptic activity derived from training; and

producing an output that is representative of at least one voice command.

7. The system of claim 1, wherein the second spiking neural network is in the halted or the off state when the spike patterns indicative of the presence of the voice activity are not detected by the first spiking neural network.

8. The system of claim 1, wherein the artificial cochlea is configured to reject background noise by selectively rejecting certain bandwidths of an input signal within the artificial cochlea.

9. The system of claim 1, wherein the system is a hard wired digital logic circuit running at a low clock speed.

10. The system of claim 1, further comprising a labeling and decode logic to generate an interrupt to a digital signal processor based on the activation of the second spiking neural network.

11. A method for allowing a user to control an electronic device by using voice commands, the method comprising:

receiving, by one or more microphones, a voice signal;

converting, by the one or more microphones, the voice signal into a pulse density modulation signal;

converting, by one or more decimation filters, the pulse density modulation signal into a pulse code modulation signal,

producing, by an artificial cochlea, an output of parallel spikes based on the pulse code modulation signal;

detecting, by a first spiking neural network, a presence of voice activity in the output of parallel spikes; and

activating, by the first spiking neural network, a second spiking neural network based on the presence of the voice activity, wherein the second spiking neural network is configured to perform a cognitive and learning function that detects spike patterns in the output of parallel spikes from the artificial cochlea that are indicative of specific voice commands.

12. The method of claim 11, further comprising:

passing, by the first spiking neural network, the output of parallel spikes to the second spiking neural network; and

recognizing, by the second spiking neural network, the spike patterns in the output of parallel spikes that are indicative of the specific voice commands.

13. The method of claim 11, wherein the first spiking neural network deactivates the second spiking neural network when no voice activity is detected.

14. The method of claim 11, wherein the artificial cochlea is configured as a digital emulation of a biological cochlea.

15. The method of claim 11, wherein the first spiking neural network and the second spiking neural network comprise a plurality of artificial neurons interconnected by a plurality of digital synapses.

16. The method of claim 11, wherein the electronic device is a portable wireless device.

17. The method of claim 14, wherein the artificial cochlea is configured to detect frequencies in a bandwidth of the voice signal.