POINTING-DIRECTION DETECTING DEVICE AND ITS METHOD, PROGRAM AND COMPUTER READABLE-MEDIUM

• US 20150062010A1
• Filed: 02/24/2014
• Published: 03/05/2015
• Est. Priority Date: 09/05/2013
• Status: Abandoned Application

• Current Assignee: Utechzone Co., Ltd.
• Original Assignee: Utechzone Co., Ltd.
• Inventors: LIN, Chieh-Yu, CHEN, Yi-Wen
• Application Number:
  US14/188,326
• Publication Number:
  US 20150062010A1
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

The present invention provides a pointing-direction detecting device, comprising an image capturing unit for capturing a user'"'"'s image to acquire the user'"'"'s skeleton data in the world coordinate system; and a processing unit connected to the image capturing unit. The processing unit comprises an image analyzing module and a pointing-direction analyzing module. The image analyzing module acquires the skeleton data from the image capturing unit and captures an upper limb image of the skeleton data to obtain respectively a hand-terminal region and an elbow or shoulder region. The pointing-direction analyzing module obtains the hand-terminal region and the elbow or shoulder region from the image analyzing module to identify through calculation a hand-terminal position and an elbow or shoulder position, obtaining a pointing-direction vector from the hand-terminal position and the elbow or shoulder position for calculating the intersection of the extension of the pointing-direction vector and the target plane.

Claims

1. A pointing-direction detecting device, comprising:

an image capturing unit for capturing an image of a user to acquire the user'"'"'s skeleton data in the world coordinate system; and

a processing unit, comprising an image analyzing module and a pointing-direction analyzing module, wherein the image analyzing module acquires the skeleton data from the image capturing unit and captures an upper limb image of the skeleton data to obtain respectively a hand terminal region, and an elbow or shoulder region, wherein the pointing-direction analyzing module obtains the hand-terminal region and the elbow or shoulder region from the image analyzing module to calculate a hand-terminal position and an elbow or shoulder position and to obtain a pointing-direction vector from the hand-terminal position and the elbow or shoulder position for calculating the intersection of the extension of the pointing-direction vector and the target plane, wherein the intersection is the user'"'"'s pointing-direction that the user'"'"'s hand points toward.

2. The device of claim 1, wherein the image capturing unit is at least one selected from the group consisting of an active depth camera and a stereo camera.

3. The device of claim 1, wherein the hand-terminal region is at least one selected from a group consisting of a user'"'"'s wrist and palm.

4. A pointing-direction detecting method, comprising:

a) acquiring and keeping track of a user'"'"'s skeleton data in the world coordinate system through an image capturing unit;

b) capturing the user'"'"'s upper limb image to obtain respectively a hand-terminal position and an elbow or shoulder position;

c) calculating a pointing-direction vector from the elbow or shoulder position to the hand-terminal position; and

d) calculating an intersection between an target plane and an extension of the pointing-direction vector to obtain a pointing-direction.

5. The method of claim 4, wherein the hand-terminal position of step b is obtained by the following steps:

sifting a plurality of effective depth parameters nearby a hand-terminal region; and

averaging the effective depth parameters to obtain the hand-terminal position and keep tracking.

6. The method of claim 5, wherein the elbow position of step b is obtained by the following steps:

sifting a plurality of effective depth parameters nearby an elbow region connected to the hand-terminal region; and

averaging the effective depth parameters to obtain the elbow position.

7. The method of claim 5, wherein the shoulder position of step b is obtained by the following step:

sifting a plurality of effective depth parameters nearby a shoulder region connected to the hand-terminal region; and

averaging the effective depth parameters to obtain the shoulder position.

8. The method of claim 4, wherein the method further comprises a step d1 after step d:

d1) determining whether the pointing-direction is projected onto a display;

if yes, return the pointing-direction to move a target object to the pointing-direction.

9. The method of claim 4, wherein the method further comprises a step a0 and a1 before step a:

a0) continuously detecting whether there is at least one person entering;

if yes, continue to the next step; and

a1) continuously detecting whether the person is in the range of an operation region;

if yes, execute step a.

10. The method of claim 4, wherein the pointing-direction of step d is obtained by the following steps:

setting up a virtual reference coordinate system according to the projection of the elbow position on a plane of a display screen, and acquiring the vertical distances between the elbow position and the plane of the display screen;

calculating a projection of the pointing-direction vector respectively on the y-z plane and x-z plane according to the pointing-direction vector to acquire two angles between the two projections and the z axis;

acquiring a temporary coordinate relative to the virtual reference coordinate through the vertical distances and the two angles; and

adding the temporary coordinate to the virtual reference coordinate to obtain the pointing-direction toward the plane of the display screen based on the world coordinates.

11. The method of claim 10, wherein the position of the pointing-direction on the pixel matrix of the display screen is transformed by the following steps:

correcting the pointing-direction according to the position of the image capturing unit relative to the origin of the pixel matrix to acquire a relative coordinate of the pointing-direction relative to the origin of the pixel matrix; and

inputting the relative coordinate into the pixel proportion transformation function to calculate the unit of the relative coordinate according to the aspect ratio of a pixel on the display screen to acquire the display position of the relative coordinate on the pixel matrix of the display screen.

12. The method of claim 10, wherein the position of the pointing-direction on the pixel matrix is transformed by the following steps:

inputting the pointing-direction into the pixel proportion transformation function to calculate the unit of the relative coordinate according to the aspect ratio of a pixel on the display screen to acquire a temporary display position; and

correcting the temporary display position according to the position of the image capturing unit relative to the origin of the pixel matrix to acquire the display position of the temporary display position on the pixel matrix of the display screen.

13. The method of claim 4, wherein the pointing-direction of step d is obtained by the following steps:

setting up a virtual reference coordinate system according to the projection of the hand-terminal position on a plane of a display screen, and acquiring the vertical distances between the hand-terminal position and the plane of the display screen;

calculating a projection of the pointing-direction vector respectively on the y-z plane and x-z plane according to the pointing-direction vector to acquire two angles between the two projections and the z axis;

acquiring a temporary coordinate relative to the virtual reference coordinate through the vertical distances and the two angles; and

adding the temporary coordinate to the virtual reference coordinate to obtain the pointing-direction toward the plane of the display screen based on the world coordinates.

14. The method of claim 13, wherein the position of the pointing-direction on the pixel matrix is transformed by the following steps:

correcting the pointing-direction according to the position of the image capturing unit relative to the origin of the pixel matrix to acquire a relative coordinate of the pointing-direction position relative to the origin of the pixel matrix; and

inputting the relative coordinate into the pixel proportion transformation function to calculate the of the relative coordinate according to the aspect ratio of a pixel on the display screen to acquire the display position of the relative coordinate on the pixel matrix of the display screen.

15. The method of claim 13, wherein the position of the pointing-direction on the pixel matrix is transformed by the following steps:

inputting the pointing-direction position into the pixel proportion transformation function to calculate the unit of the relative coordinate according to the aspect ratio of a pixel on the display screen to acquire a temporary display position; and

correcting the temporary display position according to the position of the image capturing unit relative to the origin of the pixel matrix to acquire the display position of the temporary display position on the pixel matrix of the display screen.

16. The method of claim 4, wherein the hand-terminal region is at least one selected from a group consisting of a user'"'"'s wrist and palm.

17. A computer-readable storage medium installed on an electronic device, wherein the computer-readable storage medium is stored with a pointing-direction detecting method according to claim 4.

18. A pointing-direction detecting program, comprising:

an image analyzing module for capturing an upper limb image of skeleton data after acquiring the skeleton data of a user to respectively identify a hand-terminal region and an elbow or shoulder region of the skeleton data; and

a pointing-direction analyzing module for acquiring the hand-terminal region and the elbow or shoulder region from the image analyzing module to identify through calculation a hand-terminal position and an elbow or shoulder position for obtaining a pointing-direction vector from the elbow or shoulder position to the hand-terminal position, identifying through calculation an intersection between a target plane and an extension of the pointing-direction vector, wherein intersection is the pointing-direction that the hand points toward.

19. The program of claim 18, wherein the hand-terminal region is at least one selected from a group consisting of a user'"'"'s wrist and palm.