Gesture cataloging and recognition

• US 8,839,279 B2
• Filed: 02/06/2012
• Issued: 09/16/2014
• Est. Priority Date: 01/11/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Miller, Thomas M. IV
• Primary Examiner(s): Bui, Kieu Oanh T
• Application Number:
  US13/367,089
• Publication Number:
  US 20120135805A1
• Time in Patent Office: 953 Days
• Field of Search: 725/10, 725/133, 725/141, 725/153
• US Class Current: 725/10
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A total variance may be computed for sample values in an input gesture including a plurality of sample values of sample motion data from one or more sensors associated with a control device. The motion data may be related to movement of a control device. A figure of merit may be calculated using the sample values in the gesture, the total variance for the sample values in the input gesture, and sample values in one or more catalog gestures. The figure of merit measures how well the samples in the input gesture match samples in the catalog gesture. Whether an input gesture matches one of the one or more catalog gesture may be determined based on the figure of merit. A state of the system may be changed if it is determined that the input gesture matches the one of the one or more catalog gestures.

Claims

1. In a system having a control device and a processor coupled to the control device, a gesture recognition method comprising:

a) computing with the processor a total variance for sample values in an input gesture including a plurality of sample values of sample motion data from one or more sensors associated with the control device wherein the motion data is related to movement of the control device;

b) calculating a figure of merit using the sample values in the gesture, the total variance for the sample values in the input gesture, and sample values in one or more catalog gestures, wherein the figure of merit measures how well the samples in the input gesture match samples in the catalog gestures;

c) determining whether an input gesture matches one of the one or more catalog gesture based on the figure of merit; and

d) changing a state of the system if it is determined that the input gesture matches the one of the one or more catalog gestures.

2. The gesture recognition method of claim 1, wherein the system is a video game system and d) comprises changing a state of a game implemented with the video game system with the processor if it is determined at c) that the input gesture matches the one of the one or more catalog gestures.

3. The method of claim 1, wherein the sample motion data comprises vector components.

4. The method of claim 3, wherein the total variance is calculated with ranks assigned to components of the total variance corresponding to different ones of the vector components.

5. The method of claim 4, wherein b) includes comparing input gesture component ranks to catalog gesture component ranks.

6. The method of claim 4, wherein b) includes weighting components by ranking of total variance components.

7. The method of claim 4, wherein b) includes calculating a correlation between the input gesture and the catalog gesture.

8. The method of claim 4 further comprising scaling up the input gesture or catalog gesture based on which has fewer samples.

9. The method of claim 4, further comprising comparing total variance rankings for input gesture and catalog gesture and applying a penalty for each of input gesture rank that does not match.

10. The method of claim 9, wherein the figure of merit comprise a correlation scalar calculated as weighted average of scalar correlation with weights based on the total variance rankings.

11. The method of claim 1, wherein the one or more sensors are one or more inertial sensors.

12. An apparatus for gesture recognition comprising:

a control device;

a processor operable to execute a program of a method for gesture recognition, the method comprising;

a) computing a total variance for sample values in an input gesture including a plurality of sample values of sample motion data from one or more sensors associated with the control device wherein the motion data is related to movement of the control device;

b) calculating a figure of merit using the sample values in the gesture and sample values in one or more catalog gestures.

13. The apparatus of claim 12, wherein the sample motion data comprises vector components, wherein the total variance is calculated with ranks assigned to components of the total variance corresponding to different ones of the vector components, wherein b) includes comparing input gesture component ranks to catalog gesture component ranks.

14. The apparatus of claim 12, further comprising comparing total variance rankings for input gesture and catalog gesture and applying a penalty for each of input gesture rank that does not match.

16. The computer readable storage medium of claim 12, wherein b) includes comparing input gesture component ranks to catalog gesture component ranks.

17. The computer readable storage medium of claim 12, further comprising comparing total variance rankings for input gesture and catalog gesture and applying a penalty for each input gesture rank that does not match.

15. A computer readable storage medium having computer readable instructions embodied therein, the computer readable instructions being configured to implement, when executed, a method for gesture recognition, the method comprising:

a) computing a total variance for sample values in an input gesture including a plurality of sample values of sample motion data from one or more sensors associated with a control device wherein the motion data is related to movement of the control device;

b) calculating a figure of merit using the sample values in the gesture, the total variance for the sample values in the input gesture, and sample values in one or more catalog gestures, wherein the figure of merit measures how well the samples in the input gesture match samples in the catalog gesture;

c) determining whether an input gesture matches one of the one or more catalog gestures based on the figure of merit; and

d) changing a state of the apparatus if it is determined that the input gesture matches the one of the one or more catalog gestures.

18. A computer readable storage medium having computer readable instructions embodied therein, the computer readable instructions being configured to implement, when executed, a method for gesture recognition, the method comprising:

a) computing a total variance for sample values in an input gesture including a plurality of sample values of sample motion data from one or more sensors associated with a control device wherein the motion data is related to movement of the control device;

b) calculating a figure of merit using the sample values in the gesture and sample values in one or more catalog gestures, wherein the figure of merit measures how well the samples in the input gesture match samples in the catalog gestures; and

c) determining whether an input gesture matches one of the one or more catalog gestures based on the figure of merit.

19. The computer readable storage medium of claim 18, wherein the sample motion data comprises vector components, wherein the total variance is calculated with ranks assigned to components of the total variance corresponding to different ones of the vector components, wherein b) includes comparing input gesture component ranks to catalog gesture component ranks.

20. The computer readable storage medium of claim 19, wherein the method further comprises comparing total variance rankings for input gesture and catalog gesture and applying a penalty for each of input gesture rank that does not match.