METHODS AND SYSTEMS FOR MAPPING POINTING DEVICE ON DEPTH MAP

• US 20130010071A1
• Filed: 07/04/2012
• Published: 01/10/2013
• Est. Priority Date: 07/04/2011
• Status: Abandoned Application

• Current Assignee: 3divi Company
• Original Assignee: 3DIVI
• Inventors: Valik, Andrey, Zaitsev, Pavel, Morozov, Dmitry, Argutin, Alexander
• Application Number:
  US13/541,684
• Publication Number:
  US 20130010071A1
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Disclosed are methods for determining and tracking a current location of a handheld pointing device, such as a remote control for an entertainment system, on a depth map generated by a gesture recognition control system. The methods disclosed herein enable identifying a user'"'"'s hand gesture, and generating corresponding motion data. Further, the handheld pointing device may send motion, such as acceleration or velocity, and/or orientation data such as pitch, roll, and yaw angles. The motion data of user'"'"'s hand gesture and motion data (orientation data) as received from the handheld pointing device are then compared, and if they correspond to each other, it is determined that the handheld pointing device is in active use by the user as it is held by a particular hand. Accordingly, a location of the handheld pointing device on the depth map can be determined.

Claims

1. A computer-implemented method for determining a position of a handheld pointing device, the method comprising:

determining one or more motions of one or more user hands on a depth map;

generating motion data associated with the one or more motions of the one or more user hands;

acquiring motion data of a handheld pointing device;

determining that the motion of the handheld pointing device is associated with the one or more motions of the one or more user hands;

and determining a current position of the handheld pointing device on the depth map.

2. The method of claim 1, further comprising generating the depth map by capturing a series of images.

3. The method of claim 1, further comprising generating a virtual skeleton of the user, the virtual skeleton comprising at least one virtual limb of the user.

4. The method of claim 3, further comprising:

determining coordinates of the one or more user hands, the coordinates to be associated with the virtual skeleton; and

generating motion data of the one or more user hands.

5. The method of claim 4, wherein determining that the motion of the handheld pointing device is associated with the one or more motions of the one or more user hands comprises comparing motion data of the one or more user hands and motion data of the handheld pointing device.

6. The method of claim 5, further comprising determining which hand is holding the pointing device.

7. The method of claim 6, further comprising selectively assigning the coordinates of the hand holding the handheld pointing device to the handheld pointing device.

8. The method of claim 7, further comprising determining an orientation of the handheld pointing device based upon the coordinates of various virtual skeleton joints related to the hand holding the handheld pointing device.

9. The method of claim 8, further comprising generating a vector associated with the orientation of the handheld pointing device.

10. The method of claim 1, further comprising acquiring orientation data of the handheld pointing device, wherein the orientation data comprising one or more of the following:

a pitch angle, a roll angle, and a yaw angle.

11. The method of claim 1, wherein the motion data associated with a motion of the handheld pointing device comprises one or more of acceleration data, velocity data, and inertial data.

12. The method of claim 1, further comprising determining that the handheld pointing device is in active use by the user.

13. The method of claim 12, wherein the handheld pointing device is in active use by the user when the handheld pointing device is held and moved by the user and when the user is identified on the depth map.

14. The method of claim 1, further comprising identifying the user on the depth map.

15. The method of claim 1, further comprising tracking motions of the user hand.

16. A system for determining a position of a handheld pointing device, the system comprising:

a depth sensing device configured to generate a depth map;

a communication module configured to acquire motion data of a handheld pointing device; and

a computing unit communicatively coupled to the depth sensing device, the computing unit configured to;

determine one or more motions of one or more user hands on a depth map;

generate motion data associated with the one or more motions of the one or more user hands;

determine that the motion of the handheld pointing device is associated with the one or more motions of the one or more user hands; and

determine a current position of the handheld pointing device on the depth map.

17. The system of claim 16, further comprising a video camera communicatively coupled to the computing unit, the video camera being configured to facilitate generation of the depth map.

18. The system of claim 16, wherein the computing unit is further configured to generate a virtual skeleton of the user, the virtual skeleton comprising at least one virtual hand of the user.

19. The system of claim 16, wherein the computing unit is further configured to:

determine coordinates of the one or more user hands, the coordinates to be associated with the virtual skeleton;

generate motion data of the one or more user hands; and

compare motion data of the one or more user hands and motion data of the handheld pointing device.

20. A processor-readable nontransitory medium having instructions stored thereon, which when executed by one or more processors, cause the one or more processors to:

determine one or more motions of one or more user hands on a depth map;

generate motion data associated with the one or more motions of the one or more user hands;

acquire motion data of a handheld pointing device;

determine that the motion of the handheld pointing device is associated with the one or more motions of the one or more user hands; and

determine a current position of the handheld pointing device on the depth map.