Methods and Apparatus for Recognition of a Plurality of Gestures Using Roll Pitch Yaw Data

• US 20150241985A1
• Filed: 03/03/2015
• Published: 08/27/2015
• Est. Priority Date: 01/07/2014
• Status: Active Grant

• Current Assignee: Nod, Inc. (Advanced Micro Devices, Inc.)
• Original Assignee: Nod, Inc. (Advanced Micro Devices, Inc.)
• Inventors: Elangovan, Anusankar, Menon, Harsh
• Granted Patent: US 10,725,550 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: G06F 1/3259 Power saving in cursor control device, e.g. mouse, joystick, trackball
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Described are apparatus and methods for reconstructing a plurality of gestures using roll pitch and yaw data, typically in combination with data for recognition of start and/or stop portions of the gesture using an auxiliary sensor, such as a capacitive touch sensor or a MEMS sensor.

Claims

1. An apparatus capable of interacting with at least one controllable device based upon a pose of at least a portion of a human body, the apparatus comprising:

one or more sensors that are sized for wearing on the human body, each of the one or more sensors emitting sensor data;

and

a detection unit that operates upon the sensor data to determine the pose of at least the portion of the human body and is capable of interacting with the at least one controllable device, the detection unit including;

a memory that stores at least one or more characteristics of human anatomy that are associated with the human body using at least a partial skeletal rendering of a human; and

a detection processor, automatically operating under software control, that inputs, aggregates and fuses the sensor data from each of the one or more sensors using the at least one or more characteristics of human anatomy stored in the memory to determine the pose of at least the portion of the human body based upon a locality of said one or more sensors, and wherein the pose is determined using at least a determination of which one of a change in roll, a change in pitch and a change in yaw is greatest during a period of time

2. The apparatus according to claim 1 further including an auxiliary sensor sized for wearing on the human body that receives a first specific input based on one of a tactile switch and capacitive touch input and generates a gesture start signal;

and wherein the period of time is determined by when the detection processor begins to input, aggregate and fuse the sensor data upon receipt of the gesture start signal and ceases to input the sensor data upon receipt of a gesture stop signal; and

wherein the auxiliary sensor receives a second specific input based on one of the tactile switch and the capacitive touch input and generates the gesture stop signal.

3. The apparatus according to claim 2 wherein the detection processor determines that the roll change is greatest, and that the roll change is greater than 0, thereby determining a clockwise gesture signal.

4. The apparatus according to claim 2 wherein the detection processor determines that the roll change is greatest, and that the roll change is less than 0, thereby determining a counter clockwise gesture signal.

5. The apparatus according to claim 2 wherein the detection processor determines that the yaw change is greatest, and that the yaw change is greater than 0, thereby determining a right gesture signal.

6. The apparatus according to claim 2 wherein the detection processor determines that the yaw change is greatest, and that the yaw change is less than 0, thereby determining a left gesture signal.

7. The apparatus according to claim 2 wherein the detection processor determines that the pitch change is greatest, and that the pitch change is greater than 0, thereby determining an up gesture signal.

8. The apparatus according to claim 2 wherein the detection processor determines that the pitch change is greatest, and that the pitch change is less than 0, thereby determining a down gesture signal.

9. The apparatus according to claim 2 wherein the detection processor determines that an initial movement is a false gesture, and sets another gesture start signal usable as the gesture start signal thereafter.

10. The apparatus according to claim 1, wherein the change in roll, the change in yaw, and the change in pitch are each determined based upon usage of an euler angle.

12. The method according to claim 9 further including the steps of:

sensing, using an auxiliary sensor sized for wearing on the human body a first specific input based on one of a tactile switch and capacitive touch input and generating a gesture start signal; and

generating a gesture stop signal from the auxiliary sensor sized for wearing on the human body, the gesture stop signal obtained based upon a second specific input based on one of the tactile switch and the capacitive touch input, wherein the period of time is determined between the generation of the gesture start signal and the generation of the gesture stop signal.

13. The method according to claim 12 wherein the step of automatically detecting determines that the roll change is greatest, and that the roll change is greater than 0, thereby determining a clockwise gesture signal.

14. The method according to claim 12 wherein the step of automatically detecting determines that the roll change is greatest, and that the roll change is less than 0, thereby determining a counter clockwise gesture signal.

15. The method according to claim 12 wherein the step of automatically detecting determines that the yaw change is greatest, and that the yaw change is greater than 0, thereby determining a right gesture signal.

16. The method according to claim 12 wherein the step of automatically detecting determines that the yaw change is greatest, and that the yaw change is less than 0, thereby determining a left gesture signal.

17. The method according to claim 12 wherein the step of automatically detecting determines that the pitch change is greatest, and that the pitch change is greater than 0, thereby determining an up gesture signal.

18. The method according to claim 12 wherein the step of automatically detecting determines that the pitch change is greatest, and that the pitch change is less than 0, thereby determining a down gesture signal.

19. The method according to claim 12 wherein the step of automatically detecting determines that an initial movement is a false gesture, and sets another gesture start signal usable as the gesture start signal thereafter.

20. The method according to claim 12, wherein the change in roll, the change in yaw, and the change in pitch are each determined based upon usage of an euler angle.

11. An method for interacting with at least one controllable device based upon a pose of at least a portion of a human body, the method comprising:

sensing, using one or more sensors that are sized for wearing on the human body, sensor data from each of the one or more sensors;

and

determining the pose of at least the portion of the human body based upon the sensor data, under processor and software control, the step of determining operating to;

associate at least one or more characteristics of human anatomy with the human body using at least a partial skeletal rendering of a human; and

automatically determine, under the processor and software control the pose of at least the portion of the human body based upon a locality of said one or more sensors, the step of automatically determining including inputting, aggregating and fusing the sensor data from each of the one or more sensors using the at least one or more characteristics of human anatomy to determine the pose, wherein the pose is determined using at least a determination of which one of a change in roll, a change in pitch and a change in yaw is greatest during a period of time.