SYSTEM AND METHOD FOR GRAB AND DROP GESTURE RECOGNITION

• US 20100071965A1
• Filed: 12/01/2008
• Published: 03/25/2010
• Est. Priority Date: 09/23/2008
• Status: Abandoned Application

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Hu, Nan, Kryze, David, Pathak, Rabindra
• Application Number:
  US12/325,435
• Publication Number:
  US 20100071965A1
• Time in Patent Office: Days
• US Class Current: 178/18.60
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

X-axis and Y-axis sensor arrays detect hand motion. The array data are processed by a trained model gesture recognizer to discriminate between grab and touch gestures. Touch gestures are further processed using touch point classifier, Hidden Markov Model and peak detector to discriminate between single point touch and multiple point touch. A Kalman tracker analyzes the trajectories of the X and Y axis data to determine how to associate X and Y axis data into ordered pairs corresponding to the touch points. The system resolves ambiguities inherent in certain sensor arrays and will also detect grab and drop gestures where the detected hand is sometimes out of sensor range during the gestural sequence.

Claims

1. A system for grab and drop gesture recognition, comprising:

a sensor array that provides gestural detection information that expresses touch point position information;

a gesture recognizer that analyzes the touch point position information using a trained model that discriminates between grab gestures and touch gestures, the gesture recognizer providing an indication of a grab gesture occurrence;

a drop detector configured to monitor gestural detection information in response to recognition by said gesture recognizer of a grab gesture occurrence, the drop detector providing an indication that a drop gesture has occurred in association with said grab gesture occurrence.

2. The system of claim 1 wherein said sensor array provides independent X and Y coordinate values expressing said touch point position information.

3. The system of claim 1 wherein said sensor array is a capacitive sensor array.

4. The system of claim 1 wherein said gesture recognizer employs a Gaussian density classifier.

5. The system of claim 1 wherein said gesture recognizer employs a trained model based on a plurality of statistical features.

6. The system of claim 5 wherein the statistical features are selected from the group consisting of the mean, standard deviation, and the normalized higher order central moments.

7. The system of claim 1 wherein said drop detector ascertains that a drop gesture has occurred by comparing gestural detection information to a predetermined threshold.

8. The system of claim 7 wherein said predetermined threshold corresponds to a weighted average of the maximum and average values of the gestural detection information.

9. The system of claim 1 wherein said gestural detection information is based on capacitance data obtained from the sensor array.

10. A system for touch point gestural analysis, comprising:

a sensor array that provides gestural detection information that expresses touch point position information;

a touch point classifier configured to discriminate between a single touch gesture and a multiple touch gesture, the touch point classifier providing a sequence of classification decisions; and

a model-based probabilistic analyzer, receptive of the sequence of classification decisions, and operative to associate the classification decisions to at least one gestural motion.

11. The system of claim 10 wherein said sensor array provides independent X and Y coordinate values expressing said touch point position information.

12. The system of claim 10 wherein said sensor array is a capacitive sensor array.

13. The system of claim 10 wherein said touch point classifier employs a Gaussian density classifier.

14. The system of claim 10 wherein said touch point classifier employs a trained model based on a plurality of statistical features.

15. The system of claim 14 wherein the statistical features are selected from the group consisting of the mean, standard deviation, and the normalized higher order central moments.

16. The system of claim 10 wherein the probabilistic analyzer employs a Hidden Markov Model.

17. The system of claim 10 further comprising a peak detector that refines the resolution of detected points associated with said at least one gestural motion by identifying maxima in said gestural detection information.

18. The system of claim 10 wherein said sensor array provides independent X and Y coordinate values expressing said touch point position information and further comprising Kalman tracker to resolve ambiguity as to how to associate given X and Y coordinate values into ordered pairs.

19. The system of claim 18 wherein said Kalman tracker evaluates the trajectory of touch point movement and associates given X and Y coordinate values that are most consistent with the observed movement.

20. A method of detecting a grab gesture comprising:

obtaining data from a sensor array that provides gestural detection information that expresses touch point position information;

analyzing the touch point position information using a trained model that discriminates between grab gestures and touch gestures;

using the results of said analyzing step to provide an indication that a grab gesture has occurred.

21. A method of detecting a grab and drop gesture comprising:

obtaining data from a sensor array that provides gestural detection information that expresses touch point position information;

analyzing the touch point position information using a trained model that discriminates between grab gestures and touch gestures and providing an indication of grab gesture occurrence;

monitoring said gestural detection information in response to said grab gesture occurrence to detect that a drop gesture has occurred and providing a corresponding indication of a drop gesture occurrence;

associating said grab gesture occurrence with said drop gesture occurrence.

22. A method of analyzing a touch gesture comprising:

obtaining data from a sensor array that provides gestural detection information that expresses of touch point position information;

classifying said gestural detection information according to whether it expresses a single touch gesture or a multiple touch gesture and providing a sequence of classification decisions; and

analyzing the classification decisions using a model-based probabilistic analyzer to associate the classification decisions to at least one gestural motion;

23. The method of claim 22 further comprising identifying maxima in said gestural detection information to refine the resolution of detected points associated with said at least one gestural motion.

24. The method of claim 22 further comprising developing independent X and Y coordinate values from said gestural detection information and associating given X and Y coordinate values into ordered pairs.

25. The method of claim 24 wherein said associating given X and Y coordinate values into ordered pairs using a Kalman filter.