HANDHELD COMPUTER SYSTEMS AND TECHNIQUES FOR CHARACTER AND COMMAND RECOGNITION RELATED TO HUMAN MOVEMENTS

• US 20160018902A1
• Filed: 09/25/2015
• Published: 01/21/2016
• Est. Priority Date: 11/09/2009
• Status: Abandoned Application

• Current Assignee: Invensense Incorporated (TDK Corporation)
• Original Assignee: Invensense Incorporated (TDK Corporation)
• Inventors: Nasiri, Steve, Sachs, David, Castro, Alex, Li, Richard, Gu, Anjia, Lin, Shang-Hung, Nakayami, Takashi
• Application Number:
  US14/866,548
• Publication Number:
  US 20160018902A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2009/2447 Motion detector
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3215 Monitoring of peripheral devices
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 2221/2105 Dual mode as a secondary aspect
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0233 : Character input methods
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 : Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 30/347 : Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G08C 2201/32 : Remote control based on movements, attitude of remote control device
• CPC Class Codes: H04M 1/67 : by electronic means
• CPC Class Codes: H04M 1/724 : User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 : with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/12 : including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/22 : including a touch pad, a touch sensor or a touch detector
• CPC Class Codes: H04W 12/06 : Authentication
• CPC Class Codes: H04W 12/68 : Gesture-dependent or behaviour-dependent
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 52/0254 : detecting a user operation or a tactile contact or a motion of the device
• CPC Class Codes: H04W 52/0264 : by selectively disabling software applications
• CPC Class Codes: H04W 52/027 : by controlling a display operation or backlight unit
• CPC Class Codes: H04W 52/0274 : by switching on or off the equipment or parts thereof
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

Systems and methods for human hand gesture recognition through a training mode and a recognition mode are disclosed. In the training mode, a user can move a handheld device with a hand gesture intended to represent a command. Sensors within the handheld device can record raw data, which can be processed to obtain a set of values corresponding to a set of discrete features, which is stored in a database and associated with the intended command. The process is repeated for various hand gestures representing different commands. In the recognition mode, the user can move the handheld device with a hand gesture. A computer system can compare a set of values corresponding to a set of discrete features derived from the hand gesture with the sets of values stored in the database, select a command with the closest match and displays and/or executes the command.

Claims

1. A hand held electronic device, comprising:

a gyroscope that generates gyroscope data representing a trajectory of a movement of the handheld device;

an accelerometer that generates accelerometer data representing a tilt of the handheld device relative to earth'"'"'s gravity, wherein the trajectory comprises at least one segment; and

a processing component that generates a command to initiate a function that corresponds to the command in the hand held electronic device;

wherein generating a command comprises;

moving the handheld device along a selected trajectory corresponding to the command;

determining the selected trajectory and segments thereof using at least one of the gyroscope and the accelerometer;

processing data from the at least one of the gyroscope and the accelerometer to derive data parameters;

wherein the data parameters include any of at least one of an angle of the trajectory, an inverse tangent of a change in pitch divided by a change in yaw, a location of a crossing point or a number of times in which the trajectory changes in direction; and

determining the command from the determined selected trajectory and segments thereof based on the derived data parameters to initiate the function in the handheld device.

2. The handheld device of claim 1, wherein the processing component stores at least a portion of the data generated by the at least one of the gyroscope and the accelerometer in a data store.

3. The handheld device of claim 1, further comprising a display for displaying the trajectory of the movement.

4. The handheld device of claim 1, wherein the processing component executes a command associated with an application program associated with the handheld device or an operating system associated with the handheld device.

5. The handheld device of claim 4, wherein the command triggers an event in a video game.

6. The handheld device of claim 4, wherein the command opens or closes the application program or controls a set of applications running within the operating system.

7. A method of generating a command to initiate a function that corresponds to the command in a hand held electronic device;

the method comprising;

moving the handheld device along a selected trajectory corresponding to the command;

determining the selected trajectory and segments thereof using at least one of a gyroscope and an accelerometer contained within the handheld device;

processing data from the at least one of the gyroscope and the accelerometer to derive data parameters;

wherein the data parameters include any of at least one of an angle of the trajectory, an inverse tangent of a change in pitch divided by a change in yaw, a location of a crossing point or a number of times in which the trajectory changes in direction; and

determining the command from the determined selected trajectory and segments thereof based on the derived data parameters to initiate the function in the handheld device.

8. The method of claim 7, wherein the processing component stores at least a portion of the data generated by the at least one of the gyroscope and the accelerometer in a data store.

9. The method of claim 7, further comprising displaying the trajectory of the movement on a display in the hand held electronic device.

10. The method of claim 7, wherein the processing component executes a command associated with an application program associated with the handheld device or an operating system associated with the handheld device.

11. The method of claim 10, wherein the command triggers an event in a video game.

12. The method of claim 10, wherein the command opens or closes the application program or controls a set of applications running within the operating system.

13. A hand held electronic device, comprising:

a gyroscope that generates gyroscope data representing a trajectory of a movement of the handheld device;

an accelerometer that generates accelerometer data representing a tilt of the handheld device relative to earth'"'"'s gravity, wherein the trajectory comprises at least one segment;

a compass that generates compass data representing a direction of the handheld device relative to the earth'"'"'s magnetic pole; and

a processing component that generates a command to initiate a function that corresponds to the command in the hand held electronic device;

wherein generating a command comprises;

moving the handheld device along a selected trajectory corresponding to the command;

determining the selected trajectory and segments thereof using at least one of the gyroscope, the accelerometer and the compass;

processing data from the at least one of the gyroscope, the accelerometer and the compass to derive data parameters;

wherein the data parameters include any of at least one of an angle of the trajectory, an inverse tangent of a change in pitch divided by a change in yaw, a location of a crossing point or a number of times in which the trajectory changes in direction, and a direction of the handheld device relative to the earth'"'"'s magnetic pole; and

determining the command from the determined selected trajectory and segments thereof based on the derived data parameters to initiate the function in the handheld device.

14. The handheld device of claim 13, wherein the processing component stores at least a portion of the data generated by the at least one of the gyroscope, the accelerometer and the compass in a data store.

15. The handheld device of claim 13, further comprising a display for displaying the trajectory of the movement.

16. The handheld device of claim 13, wherein the processing component executes a command associated with an application program associated with the handheld device or an operating system associated with the handheld device.

17. The handheld device of claim 16, wherein the command triggers an event in a video game.

18. The handheld device of claim 16, wherein the command opens or closes the application program or controls a set of applications running within the operating system.