Human interface device and method

• US 9,625,997 B2
• Filed: 07/10/2014
• Issued: 04/18/2017
• Est. Priority Date: 07/19/2013
• Status: Active Grant

• Current Assignee: Microchip Technology Incorporated
• Original Assignee: Microchip Technology Incorporated
• Inventors: Heim, Axel
• Primary Examiner(s): POLO, GUSTAVO D
• Application Number:
  US14/328,265
• Publication Number:
  US 20150022490A1
• Time in Patent Office: 1,013 Days
• Field of Search: 345174
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method for state tracking based gesture recognition engine for a sensor system has the steps of: defining a plurality of sequential states of a finite-state machine, determining a Sequence Progress Level (SPL) for each state, mapping a state probability distribution to a (single) SPL on run-time, and utilizing the mapped SPL estimate as an output value of the sensor system.

Claims

1. A method for state tracking based gesture recognition engine for a sensor system comprising the steps of:

defining a plurality of sequential states of a finite-state machine, wherein the finite state machine is a first order Markov Model,

determining a Sequence Progress Level (SPL) for each state,

mapping a state probability distribution to an SPL on run-time, and

utilizing the mapped SPL estimate as an output value of the sensor system, wherein a most likely state is computed using a Forward (Baum-Welch) Algorithm.

2. The method according to claim 1, wherein a last state is followed by a first state of the plurality of sequential states.

3. The method according to claim 1, wherein the sensor system is a three-dimensional (3-D) sensing system.

4. The method according to claim 3, wherein the 3-D sensor system is a near-field capacitive sensor systems or a mid/far field sensor system.

5. The method according to claim 4, wherein the mid/far field sensor system uses a video or infrared camera system.

6. The method according to claim 4, wherein the near-field capacitive sensor system is a capacitive non-touching 3-D sensor system based on a quasi static electric field measurements.

7. The method according to claim 1, wherein the sensor system is a two-dimensional (2-D) sensing system, a capacitive or resistive touch system or a touchscreen system.

8. The method according to claim 1, wherein the sensor system is a one-dimensional (1-D) sensing system.

9. The method according to claim 1, wherein the plurality of states are states of a circular gesture.

28. A method of operating a device configured to implement the method according to claim 1, wherein the output value is used to control a function of the device.

29. The method according to claim 28, wherein the function is increasing or decreasing a volume level, controlling a dimmer, or controlling a scroll function.

10. A method for state tracking based gesture recognition engine for a sensor system comprising the steps of:

defining a plurality of sequential states of a finite-state machine, wherein the method uses an N-state Hidden Markov Model (HMM) comprising a state transition probability matrix,

determining a Sequence Progress Level (SPL) for each state,

mapping a state probability distribution to an SPL on run-time,

utilizing the mapped SPL estimate as an output value of the sensor system, wherein for each discrete-time instance the data provided by the sensor system is forwarded to the finite state machine which computes a state probability distribution for the N-state HMM, and wherein, for each discrete-time instance, a state with the maximum probability is selected and an SPL associated with the state is output.

11. The method according to claim 10, wherein the state transition probability matrix comprises in each row a maximum probability value.

12. The method according to claim 11, wherein all probabilities of each row add up to 100%.

13. The method according to claim 10, wherein the HMI further uses an observation matrix which indicates the probability of predefined movement directions.

14. The method according to claim 13, wherein the observation matrix comprises in each row a maximum probability value.

15. The method according to claim 14, wherein all probabilities of each row add up to 100%.

19. The method according to claim 13, wherein, for a plurality of sequential output values a smoothening algorithm is applied during post processing of sequential output values.

30. The method according to claim 10, wherein a last state is followed by a first state of the plurality of sequential states.

31. The method according to claim 10, wherein the sensor system is a non-touching three-dimensional (3-D) near-field capacitive sensor system based on a quasi static electric field measurements.

32. The method according to claim 10, wherein the sensor system is a two-dimensional (2-D) sensing system, a capacitive or resistive touch system or a touchscreen system.

33. The method according to claim 10, wherein the plurality of states are states of a circular gesture.

16. A method for state tracking based gesture recognition engine for a sensor system comprising the steps of:

defining a plurality of sequential states of a finite-state machine, wherein the method uses an N-state Hidden Markov Model (HMM) comprising a state transition probability matrix;

determining a Sequence Progress Level (SPL) for each state,

mapping a state probability distribution to an SPL on run-time,

utilizing the mapped SPL estimate as an output value of the sensor system, wherein for each discrete-time instance the data provided by the sensor system is forwarded to the finite state machine which computes a state probability distribution for the N-state HMM, and, wherein for each time instance an SPL is computed from the state probability distribution and is output.

17. The method according to claim 16, wherein the method distinguishes between a clock-wise circular gesture, a counter-clockwise circular gesture, or a no-movement gesture.

18. The method according to claim 16, wherein a Maximum Likelihood rule is applied to compute the most likely state.

20. A method for state tracking based gesture recognition engine for a sensor system comprising the steps of:

providing a state tracking based gesture recognition method which comprises;

defining a plurality of sequential states of a finite-state machine,

determining a Sequence Progress Level (SPL) for each state,

mapping a state probability distribution to an SPL on run-time, and

utilizing the mapped SPL estimate as an output value of the sensor system,

and providing a pattern based gesture recognition mode, wherein the method switches between the pattern based gesture recognition mode and a state tracking based gesture recognition mode provided by the state tracking based gesture recognition engine when a predefined condition is met,

wherein the state tracking based gesture recognition mode uses an N-state Hidden Markov Model (HMM) comprising a probability matrix,

wherein during the state tracking based gesture recognition mode for each discrete-time instance the data provided by the sensor system is pre-processed and forwarded to the finite state machine which computes a state probability distribution for each state of the N-state HMM, and

wherein, for each discrete-time instance, a state with the maximum probability is selected and an SPL associated with the state is output.

21. The method according to claim 20, wherein each state is assigned to an observation probability distribution.

23. The method according to claim 20, wherein the state tracking based gesture recognition mode distinguishes between a clock-wise circular gesture, a counter-clockwise circular gesture, or a no-movement gesture.

24. The method according to claim 23, wherein the method switches back to the pattern based gesture recognition mode if a predefined time threshold for a no-movement gesture is met.

25. The method according to claim 20, wherein a Maximum Likelihood rule is applied to compute the most likely state.

26. The method according to claim 20, wherein, for a plurality of sequential output values a smoothening algorithm is applied during post processing of sequential output values.

22. A method for state tracking based gesture recognition engine for a sensor system comprising the steps of:

providing a state tracking based gesture recognition method which comprises;

defining a plurality of sequential states of a finite-state machine,

determining a Sequence Progress Level (SPL) for each state,

mapping a state probability distribution to an SPL on run-time, and

utilizing the mapped SPL estimate as an output value of the sensor system,

and providing a pattern based gesture recognition mode, wherein the method switches between the pattern based gesture recognition mode and a state tracking based gesture recognition mode provided by the state tracking based gesture recognition engine when a predefined condition is met,

wherein the state tracking based gesture recognition mode uses an N-state Hidden Markov Model (HMM) comprising a probability matrix,

wherein during the state tracking based gesture recognition mode for each discrete-time instance the data provided by the sensor system is pre-processed and forwarded to the finite state machine which computes a state probability distribution for each state of the N-state HMM, and

wherein for each time instance a SPL is computed from the state probability distribution and is output.

27. The method according to claim 22, wherein, for a plurality of sequential output values a smoothening algorithm is applied during post processing of sequential output values.