Method and apparatus for recognition of sensor data patterns

• US 10,540,597 B1
• Filed: 06/25/2015
• Issued: 01/21/2020
• Est. Priority Date: 06/25/2014
• Status: Active Grant

• Current Assignee: Bosch Sensortec GmbH (Robert Bosch GmbH)
• Original Assignee: Bosch Sensortec GmbH (Robert Bosch GmbH)
• Inventors: Tilton, Adam K., Ghiotto, Shane T., Mehta, Prashant G.
• Primary Examiner(s): Starks, Wilbert L
• Application Number:
  US14/751,084
• Time in Patent Office: 1,671 Days
• Field of Search: None
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/00 Computing arrangements based on specific mathematical models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/144 Training of HMMs
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters

Abstract

Methods and systems for learning, recognition, classification and analysis of real-world cyclic patterns using a model having n oscillators, with primary frequency ω₁, ω₂, . . . , ω_n. The state of the oscillators is evolved over time using sensor observations, which are also used to determine the sensor characteristics, or the sensor observation functions. Once trained, a set of activity detection filters may be used to classify a sensor data stream as being associated with an activity.

Claims

1. A method comprising:

receiving sensor data comprising a plurality of sequential sensor measurements associated with an unknown activity;

obtaining, for each of a plurality of activity filters, activity parameters including a set of activity frequencies, wherein the set of activity frequencies corresponds to a plurality of sets of particles, each activity frequency associated with a respective set of particles configured to oscillate in accordance with the associated activity frequency, each set of particles having an associated set of particle phases;

updating, for each set of particles, the associated set of particle phases by updating the phase of each particle of the set of particles according to i) its associated activity frequency and ii) a prediction error based on a difference between at least one of the plurality of sequential observation measurements and an observation prediction from an observation model operating on the associated set of particle phases;

determining, for each of the plurality of activity filters, an activity probability based on the plurality of sets of particle phases for the corresponding activity filter, wherein each activity filter is associated with a predetermined activity, and wherein the activity probability for each activity filter indicates a probability that the received sensor data is indicative of the predetermined activity associated with that activity filter; and

outputting an indicator that identifies an activity filter having a highest activity probability among the plurality of activity filters.

2. The method of claim 1 wherein the activity filter parameters further include a set of observation function coefficients.

3. The method of claim 1 wherein determining the activity probability for each of the plurality of activity filters includes determining a probability update based on a difference between the respective activity filter'"'"'s observation prediction output and an aggregated estimate based on a weighted average of all observation predictions.

4. The method of claim 3 wherein the weightings used in the weighted average of the observation predictions are the activity probabilities.

5. The method of claim 1 wherein determining the activity probability for each of the plurality of activity filters includes determining a probability update based on a difference between a sensor measurement and an aggregated estimate of the observation predictions.

6. The method of claim 1 wherein an observation function of the observation model is updated based on the prediction error.

7. The method of claim 1 wherein updating, for each set of particles, the associated set of particle phases is performed using a particle filter algorithm.

8. The method of claim 1 wherein updating, for each set of particles, the associated set of particle phases is performed using an extended Kalman filter algorithm.

9. The method of claim 1 further comprising outputting an indication of the activity associated with the activity filter having the highest activity probability value.

10. The method of claim 1 further comprising calculating an activity quality based at least in part on an activity probability of an activity-detection filter.

11. The method of claim 10 further comprising outputting, via a user interface, the calculated activity quality.

12. The method of claim 1 wherein the activity filters are gesture-detection filters.

13. The method of claim 12 further comprising transmitting an execute-computer-task signal associated with a gesture-detection filter having the highest activity probability value to a computing device, wherein the execute-computer-task signal instructs the computing device to perform a computer task associated with the gesture-detection filter having the highest activity probability.

14. The method of claim 12 further comprising transmitting an execute-robotic-system-task signal associated with a gesture-detection filter having the highest activity probability value to a robotic system, wherein the execute-robotic-system-task signal instructs the robotic system to perform a robotic system task associated with the gesture-detection filter having the highest activity probability.

15. The method of claim 1 wherein the activity filters are mechanical-system-failure-mode-detection filters.

16. The method of claim 15 further comprising transmitting an indication of a mechanical system failure mode associated with a mechanical-system-failure-mode-detection filter having the highest activity probability value.

17. The method of claim 1 wherein the activity filters are physical-training-activity-detection filters.

18. The method of claim 17 further comprising outputting, via a user interface, a number of repetitions of a physical training activity associated with a physical-training-activity-detection filter.

19. An apparatus comprising a computer readable medium having instructions stored thereon that when executed cause a processor to perform the functions of:

receiving sensor data comprising a plurality of sequential sensor measurements associated with an unknown activity;

obtaining, for each of a plurality of activity filters, activity parameters including a set of activity frequencies, wherein the set of activity frequencies corresponds to a plurality of sets of particles, each activity frequency associated with a respective set of particles configured to oscillate in accordance with the associated activity frequency, each set of particles having an associated set of particle phases;

updating, for each set of particles, the associated set of particle phases by updating the phase of each particle of the set of particles according to i) its associated activity frequency and ii) a prediction error based on a difference between at least one of the plurality of sequential observation measurements and an observation prediction from an observation model operating on the associated set of particle phases;

determining, for each of the plurality of activity filters, an activity probability based on the plurality of sets of particle phases for the corresponding activity filter, wherein each activity filter is associated with a predetermined activity, and wherein the activity probability for each activity filter indicates a probability that the received sensor data is indicative of the predetermined activity associated with that activity filter; and

outputting an indicator that identifies an activity filter having a highest activity probability among the plurality of activity filters.