Methods and apparatus recognition of start and/or stop portions of a gesture using relative coordinate system boundaries

• US 10,338,685 B2
• Filed: 04/27/2015
• Issued: 07/02/2019
• Est. Priority Date: 01/07/2014
• Status: Active Grant

• Current Assignee: Nod, Inc. (Advanced Micro Devices, Inc.)
• Original Assignee: Nod, Inc. (Advanced Micro Devices, Inc.)
• Inventors: Elangovan, Anusankar
• Primary Examiner(s): Wills-Burns, Chineyere D
• Application Number:
  US14/697,446
• Publication Number:
  US 20150301610A1
• Time in Patent Office: 1,527 Days
• Field of Search: 345158
• CPC Class Codes: G06F 1/3259 Power saving in cursor control device, e.g. mouse, joystick, trackball
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Described are apparatus and methods for reconstructing a gesture by aggregating various data from various sensors, including data for recognition of start and/or stop portions of the gesture using a detection of an intersection with a relative coordinate system boundary.

Claims

1. An apparatus capable of interacting with at least one controllable device based upon a pose of at least a portion of a human body, the apparatus comprising:

one or more sensors that are sized for wearing on the human body, each of the one or more sensors emitting sensor data and being packaged in an integrated mechanical assembly;

and

a detection unit that operates upon the sensor data to determine the pose of at least the portion of the human body and is capable of interacting with the at least one controllable device, the detection unit including;

a memory that stores at least one or more characteristics of human anatomy that are associated with the human body using at least a partial skeletal rendering of a human; and

a detection processor, automatically operating under software control, that inputs, aggregates and fuses the sensor data from each of the one or more sensors using the at least one or more characteristics of human anatomy stored in the memory to determine the pose of at least the portion of the human body based upon a locality of said one or more sensors, wherein the detection processor begins to input, aggregate and fuse the sensor data for gesture recognition upon certain sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected into a relative coordinate system boundary to thereby initiate a gesture start signal and wherein the detection processor ceases to input the sensor data for gesture recognition upon receipt of a gesture stop signal.

2. The apparatus according to claim 1 wherein the gesture stop signal is generated by one of (a) other sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected out of the relative coordinate system boundary and (b) receipt of the gesture stop signal from an auxiliary sensor sized for wearing on the human body that receives a specific input based on one of a tactile switch and capacitive touch input.

3. The apparatus according to claim 2 wherein the detection unit is also packaged in the integrated mechanical assembly.

4. The apparatus according to claim 2 wherein the gesture stop signal is generated by other sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected out of the relative coordinate system boundary.

5. The apparatus according to claim 2 wherein the relative coordinate system boundary is related to a hand and arm, with a portion of the arm between the elbow and wrist serving as one axis of the relative coordinate system.

6. The apparatus according to claim 5 wherein the relative coordinate system boundary is associated with an electronic device disposed remote from the integrated mechanical assembly.

7. The apparatus according to claim 6 wherein a distance between at least one of the sensors on the integrated mechanical assembly and the electronic device is sensed to determine the sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected into a relative coordinate system boundary to thereby initiate the gesture start signal.

8. The apparatus according to claim 2 wherein at least one of the gesture start signal and the gesture stop signal is triggered by a detected body turn toward or away from, respectively, an electronic device with which communication is capable.

9. The apparatus according to claim 2 wherein the relative coordinate system boundary is based on a body part to which the integrated mechanical assembly will attach.

10. The apparatus according to claim 1 wherein the relative coordinate system boundary is related to a hand and arm, with a portion of the arm between the elbow and wrist serving as one axis of the relative coordinate system.

11. The apparatus according to claim 1 wherein the relative coordinate system boundary is associated with an electronic device disposed remote from the integrated mechanical assembly.

12. The method according to claim 11 further including the step of generating the gesture stop signal from an auxiliary sensor sized for wearing on the human body and packaged within the integrated mechanical assembly, the gesture stop signal is generated by one of (a) other sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected out of the relative coordinate system boundary and (b) receipt of the gesture stop signal from the auxiliary sensor.

13. The method according to claim 12 wherein at least one of the gesture start signal and the gesture stop signal is triggered by a detected body turn toward or away from, respectively, an electronic device with which communication is capable.

14. The method according to claim 12 wherein the relative coordinate system boundary is based on a body part to which the integrated mechanical assembly will attach.

15. The apparatus according to claim 1 wherein a distance between at least one of the sensors on the integrated mechanical assembly and the electronic device is sensed to determine the sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected into a relative coordinate system boundary to thereby initiate the gesture start signal.

16. The method according to claim 15 wherein the relative coordinate system boundary is related to a hand and arm, with a portion of the arm between the elbow and wrist serving as one axis of the relative coordinate system.

17. The method according to claim 16 wherein the relative coordinate system boundary is associated with an electronic device disposed remote from the integrated mechanical assembly.

18. The method according to claim 17 wherein a distance between at least one of the sensors on the integrated mechanical assembly and the electronic device is sensed to determine the sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected into a relative coordinate system boundary to thereby initiate the gesture start signal.

19. The apparatus according to claim 1 wherein at least one of the gesture start signal and the gesture stop signal is triggered by a detected body turn toward or away from, respectively, an electronic device with which communication is capable.

20. The apparatus according to claim 1 wherein the relative coordinate system boundary is based on a body part to which the integrated mechanical assembly will attach.

21. The apparatus according to claim 1 further including a timer that is used to create the gesture stop signal a predetermined period of time after generation of the gesture start signal.

22. The apparatus according to claim 1 wherein the gesture start signal is further used to change a power mode associated with the detection processor.

23. The apparatus according to claim 1 wherein the an auxiliary sensor sized for wearing on the human body that receives a first specific input based on one of a tactile switch and capacitive touch input changes the detection processor into a regular power mode and the gesture stop signal causes the one or more sensors to be turned off.

24. The apparatus according to claim 1, wherein the apparatus is further configured to interact with a first device and a second device, and wherein an orientation of the apparatus relative to the first device causes the pose to be used to signal the first device, and wherein orientation of the apparatus relative to the second device causes the pose to be used to signal the second device.

25. The apparatus according to claim 1 wherein the detection processor receives further sensor data that represents a stop time, and the gesture stop signal is generated therefrom.

26. The apparatus according to claim 1 wherein the detection processor receives further sensor data that represents a start time, and the gesture start signal is also generated therefrom.

27. The apparatus according to claim 1, wherein the portion of the human body is a finger, and wherein the integrated mechanical assembly is sized as a ring for wearing on the finger.

28. An method for interacting with at least one controllable device based upon a pose of at least a portion of a human body, the method comprising:

sensing, using one or more sensors that are sized for wearing on the human body, sensor data from each of the one or more sensors, wherein the one or more sensors are packaged in an integrated mechanical assembly; and

determining the pose of at least the portion of the human body based upon the sensor data, under processor and software control, the step of determining operating to;

associate at least one or more characteristics of human anatomy with the human body using at least a partial skeletal rendering of a human; and

automatically determine, under the processor and software control the pose of at least the portion of the human body based upon a locality of said one or more sensors and the input from the auxiliary sensor, the step of automatically determining including inputting, aggregating and fusing the sensor data from each of the one or more sensors using the at least one or more characteristics of human anatomy to determine the pose, wherein the step of automatically determining begins to input, aggregate and fuse the sensor data for gesture recognition upon certain sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected into a relative coordinate system boundary to thereby initiate a gesture start signal and wherein the detection processor ceases to input the sensor data for gesture recognition upon receipt of a gesture stop signal.

29. The method according to claim 28 wherein the relative coordinate system boundary is related to a hand and arm, with a portion of the arm between the elbow and wrist serving as one axis of the relative coordinate system.

30. The method according to claim 28 wherein the relative coordinate system boundary is associated with an electronic device disposed remote from the integrated mechanical assembly.

31. The method according to claim 28 wherein a distance between at least one of the sensors on the integrated mechanical assembly and the electronic device is sensed to determine the sensor data indicating that the at least one of the one or more sensors within the integrated mechanical assembly have intersected into a relative coordinate system boundary to thereby initiate the gesture start signal.

32. The method according to claim 28 wherein at least one of the gesture start signal and the gesture stop signal is triggered by a detected body turn toward or away from, respectively, an electronic device with which communication is capable.

33. The method according to claim 28 wherein the relative coordinate system boundary is based on a body part to which the integrated mechanical assembly will attach.

34. The method according to claim 28 wherein a timer is used to create the gesture stop signal a predetermined period of time after generation of the gesture start signal.

35. The method according to claim 28 wherein the gesture start signal is further used to change a power mode associated with the processor.

36. The method according to claim 28 wherein the gesture start signal changes the processor into a regular power mode and the gesture stop signal causes the one or more sensors to be turned off.

37. The method according to claim 28, wherein the integrated detection unit mechanical assembly interacts with a first device and a second device, and wherein an orientation of the an integrated detection unit mechanical assembly relative to the first device causes the pose to be used to signal the first device, and wherein orientation of the apparatus relative to the second device causes the pose to be used to signal the second device.

38. The method according to claim 28 wherein the processor receives further sensor data that represents a stop time, and the gesture stop signal is generated therefrom.

39. The method according to claim 28 wherein the processor receives further sensor data that represents a start time, and the gesture start signal is also generated therefrom.

40. The method according to claim 28, wherein the portion of the human body is a finger, and wherein the integrated mechanical assembly is sized as a ring for wearing on the finger.