Methods and apparatus for recognition of start and/or stop portions of a gesture using an auxiliary sensor

• US 10,338,678 B2
• Filed: 01/07/2015
• Issued: 07/02/2019
• Est. Priority Date: 01/07/2014
• Status: Active Grant

• Current Assignee: Nod, Inc. (Advanced Micro Devices, Inc.)
• Original Assignee: Nod, Inc. (Advanced Micro Devices, Inc.)
• Inventors: Elangovan, Anusankar, Bradley, William Mitchell, Menon, Harsh
• Primary Examiner(s): Wills-Burns, Chineyere D
• Application Number:
  US14/591,878
• Publication Number:
  US 20150220145A1
• Time in Patent Office: 1,637 Days
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/044 by capacitive means

Abstract

Described are apparatus and methods for reconstructing a gesture by aggregating various data from various sensors, including data for recognition of start and/or stop portions of the gesture using an auxiliary sensor, such as a capacitive touch sensor or a MEMS sensor. In a specific embodiment, power savings features are included to preserve the energy stored on the battery of a sensing device.

Claims

1. An apparatus capable of interacting with at least one controllable device based upon a pose of at least a portion of a human body, the apparatus comprising:

one or more sensors that are sized for wearing on the human body, each of the one or more sensors emitting sensor data;

an auxiliary sensor sized for wearing on the human body that receives a first specific input based on one of a tactile switch and capacitive touch input and generates a gesture start signal; and

a detection unit that operates upon the sensor data to determine the pose of at least the portion of the human body and is capable of interacting with the at least one controllable device, the detection unit including;

a memory that stores at least one or more characteristics of human anatomy that are associated with the human body using at least a partial skeletal rendering of a human; and

a detection processor, automatically operating under software control, that inputs, aggregates and fuses the sensor data from each of the one or more sensors using the at least one or more characteristics of human anatomy stored in the memory to determine the pose of at least the portion of the human body based upon a locality of said one or more sensors, wherein the detection processor begins to input, aggregate and fuse the sensor data upon receipt of the gesture start signal and ceases to input the sensor data upon receipt of a gesture stop signal;

wherein at least some of the one or more sensors, the auxiliary sensor, and the detection unit are packaged in an integrated mechanical assembly.

2. The apparatus according to claim 1 wherein the auxiliary sensor receives a second specific input based on one of the tactile switch and the capacitive touch input and generates the gesture stop signal.

3. The apparatus according to claims 2 wherein the first specific input and the second specific input are the same.

4. The apparatus according to claims 2 wherein the first specific input and the second specific input are different.

5. The apparatus according to claim 1 further including a timer that is used to create the gesture stop signal a predetermined period of time after generation of the gesture start signal.

6. The apparatus according to claim 1 wherein the first specific input is further used to change a power mode associated with the detection processor.

7. The apparatus according to claim 1 wherein the first specific input changes the detection processor into a regular power mode and the gesture stop signal causes the one or more sensors to be turned off.

8. The apparatus according to claim 1, wherein the apparatus is further configured to interact with a first device and a second device, and wherein an orientation of the apparatus relative to the first device causes the pose to be used to signal the first device, and wherein orientation of the apparatus relative to the second device causes the pose to be used to signal the second device.

9. The apparatus according to claim 1 wherein the detection processor receives further sensor data that represents a stop time, and the gesture stop signal is generated therefrom.

10. The apparatus according to claim 1 wherein the detection processor receives further sensor data that represents a start time, and the gesture start signal is also generated therefrom.

11. An apparatus according to claim 1, wherein the portion of the human body is a finger, and wherein the integrated mechanical assembly is sized as a ring for wearing on the finger.

12. A method for interacting with at least one controllable device based upon a pose of at least a portion of a human body, the method comprising:

sensing, using one or more sensors that are sized for wearing on the human body, sensor data from each of the one or more sensors;

sensing, using an auxiliary sensor sized for wearing on the human body a first specific input based on one of a tactile switch and capacitive touch input and generating a gesture start signal; and

determining the pose of at least the portion of the human body based upon the sensor data, under processor and software control, the step of determining operating to;

associate at least one or more characteristics of human anatomy with the human body using at least a partial skeletal rendering of a human; and

automatically determine, under the processor and software control the pose of at least the portion of the human body based upon a locality of said one or more sensors and the input from the auxiliary sensor, the step of automatically determining including inputting, aggregating and fusing the sensor data from each of the one or more sensors using the at least one or more characteristics of human anatomy to determine the pose, wherein the step of automatically determining begins to input, aggregate and fuse the sensor data upon receipt of the gesture start signal and ceases to input the sensor data upon receipt of a gesture stop signal, and wherein the at least one or more characteristics of human anatomy that are associated with the human body that are stored in a memory include at least one of (a) a range of motion of human skeletal joints and (b) limitations in the speed human bones can move relative to each other; and

wherein at least some of the one or more sensors, the auxiliary sensor, the processor and the software are packaged in an integrated detection unit mechanical assembly.

13. The method according to claim 12 further including the step of generating the gesture stop signal from the auxiliary sensor sized for wearing on the human body, the gesture stop signal obtained based upon a second specific input based on one of the tactile switch and the capacitive touch input.

14. The method according to claim 12 wherein the auxiliary sensor receives a second specific input based on one of the tactile switch and the capacitive touch input and generates the gesture stop signal.

15. The method according to claims 14 wherein the first specific input and the second specific input are the same.

16. The method according to claims 14 wherein the first specific input and the second specific input are different.

17. The method according to claim 12 wherein a timer is used to create the gesture stop signal a predetermined period of time after generation of the gesture start signal.

18. The method according to claim 12 wherein the first specific input is further used to change a power mode associated with the processor.

19. The method according to claim 12 wherein the first specific input changes the processor into a regular power mode and the gesture stop signal causes the one or more sensors to be turned off.

20. The method according to claim 12, wherein the integrated detection unit mechanical assembly interacts with a first device and a second device, and wherein an orientation of the an integrated detection unit mechanical assembly relative to the first device causes the pose to be used to signal the first device, and wherein orientation of the apparatus relative to the second device causes the pose to be used to signal the second device.

21. The method according to claim 12 wherein the processor receives further sensor data that represents a stop time, and the gesture stop signal is generated therefrom.

22. The method according to claim 12 wherein the processor receives further sensor data that represents a start time, and the gesture start signal is also generated therefrom.

23. A method according to claim 12, wherein the portion of the human body is a finger, and wherein the integrated mechanical assembly is sized as a ring for wearing on the finger.