Systems and methods for recognizing events

• US 8,463,721 B2
• Filed: 08/05/2010
• Issued: 06/11/2013
• Est. Priority Date: 08/05/2010
• Status: Active Grant

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Prokhorov, Danil V.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): OLUDE AFOLABI, OLATOYOSI
• Application Number:
  US12/851,207
• Publication Number:
  US 20120036097A1
• Time in Patent Office: 1,041 Days
• Field of Search: 706/20, 382/104, 382/224
• US Class Current: 706/20
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

Systems and methods for recognizing events include a processor for executing machine readable instructions. The processor may be electronically coupled to an electronic memory. A temporal sensor may be electronically coupled to the processor for generating a sequence of temporal signals relating to an unrecognized event. The temporal sensor may transmit the sequence of temporal signals to the processor. The processor may execute the machine readable instructions to: input the sequence of temporal signals relating to the unrecognized event to a recurrent neural network; transform the sequence of temporal signals relating to the unrecognized event to a neural output relating to the unrecognized event with the recurrent neural network; input the neural output relating to the unrecognized event into a random forest classifier; and recognize a recognized event based upon a transformation of the neural output relating to the unrecognized event with the random forest classifier.

Claims

1. A system for recognizing events comprising:

a processor for executing machine readable instructions, wherein the processor is electronically coupled to an electronic memory;

a temporal sensor electronically coupled to the processor for generating a sequence of temporal signals relating to an unrecognized event, wherein the temporal sensor transmits the sequence of temporal signals to the processor and the processor executes the machine readable instructions to;

input the sequence of temporal signals relating to the unrecognized event to a recurrent neural network;

transform the sequence of temporal signals relating to the unrecognized event to a neural output relating to the unrecognized event with the recurrent neural network;

input the neural output relating to the unrecognized event into a random forest classifier; and

recognize a recognized event based upon a transformation of the neural output relating to the unrecognized event with the random forest classifier.

2. The system for recognizing events of claim 1 further comprising a vehicle, wherein the processor is disposed within the vehicle and the temporal sensor detects a driver interaction with the vehicle.

3. The system for recognizing events of claim 1 wherein the neural output is a feature vector representing a plurality of intermediate transformations of the sequence of temporal signals relating to the unrecognized event from a hidden layer of nodes of the recurrent neural network.

4. The system for recognizing events of claim 1 wherein a decision path of the random forest classifier is followed based upon the neural output relating to the unrecognized event.

5. A method for recognizing events, wherein a temporal sequence of electronic data is transformed by a processor into a sequence recognition, the method comprising:

inputting a sequence of temporal signals relating to an unrecognized event into a feedback system, wherein the feedback system comprises a hidden layer of nodes for performing intermediate transformations;

generating a feature vector relating to the unrecognized event from the sequence of temporal signals with the hidden layer of nodes, the feature vector representing a plurality of intermediate transformations of the sequence of temporal signals;

inputting the feature vector into an ensemble classifier for the sequence recognition, wherein the ensemble classifier is a random forest comprising decision trees;

providing a training sequence wherein an unrecognized training event is correlated to a recognized training event;

sampling a sampled sequence for each of the decision trees from the training sequence, wherein the training sequence and the sampled sequence have an equal amount of data points such that some of the training sequence is duplicated in the sampled sequence and some of the training sequence is omitted from the sampled sequence;

recognizing a recognized event based upon a plurality of classifying transformations performed by the ensemble classifier upon the feature vector relating to the unrecognized event; and

providing output relating to the recognized event.

6. The method of claim 5 further comprising assigning a mathematic function that performs an intermediate transformation at a node of the hidden layer of nodes.

7. The method of claim 6 wherein the mathematic function that performs the intermediate transformation is selected from a function library according to a probability distribution function.

8. The method of claim 5 wherein each of the decision trees comprises decision nodes and decision paths, such that one of the decision paths interconnects a pair of the decision nodes, further comprising:

sampling a subset from the sampled sequence for each of the decision nodes; and

training one of the decision paths at each of the nodes with the subset of the sampled sequence.

9. The method of claim 5 wherein the sequence of temporal signals comprises multi-dimensional signals sampled over time.

10. The method of claim 5 wherein the recognized event is sleep.

11. The method of claim 10 wherein the output is an alarm when the sleep meets a preset criteria.

12. A method for recognizing events, wherein a temporal sequence of electronic data is transformed by a processor into a sequence recognition, the method comprising:

inputting a sequence of temporal signals relating to an unrecognized event into a group of classifiers, wherein each of the group of classifiers comprises decision nodes for making determinations based upon the sequence of temporal signals, decision paths that interconnect the decision nodes, and filtering functions that transform the sequence of temporal signals;

assigning the filtering functions to each of the decision nodes, wherein one of the decision nodes is assigned a recurrent neural network;

transforming the sequence of temporal signals relating to the unrecognized event at the decision nodes to a windowed sequence relating to the unrecognized event;

following each of the decision paths based at least in part upon a transformation of the windowed sequence by the filtering functions;

generating a group of sub-recognition events by following the decision paths of the group of classifiers until they terminate; and

recognizing a recognized event based upon the group of sub-recognition events, wherein the recognized event is recognized based upon majority voting of the group of sub-recognition events.

13. The method of claim 12 further comprising training the recurrent neural network to recognize the recognized event from the sequence of temporal signals relating to the unrecognized event before assigning the recurrent neural network to one of the decision nodes of the group of classifiers.

14. The method of claim 12 further comprising training the recurrent neural network to recognize the recognized event from the sequence of temporal signals relating to the unrecognized event after assigning the recurrent neural network to one of the decision nodes.

15. The method of claim 14 wherein the recurrent neural network is trained based at least in part upon at least one of the group of sub-recognition events with an unsupervised learning algorithm or an evolutionary algorithm.

16. The method of claim 12 further comprising selecting one of the filtering functions from a function library with a randomizer.

17. The method of claim 16 wherein the randomizer selects one of the filtering functions according to a probability distribution function.

18. The method of claim 12 wherein the recurrent neural network is untrained.