ELECTRONIC DEVICE AND METHOD FOR CONTROLLING THE ELECTRONIC DEVICE

• US 20160357319A1
• Filed: 05/26/2016
• Published: 12/08/2016
• Est. Priority Date: 06/02/2015
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: CHO, Min-su, MOON, Joong-hee, BAEK, Kun-woo, LEE, Tahk-guhn, KIM, Hyun-woo
• Application Number:
  US15/165,538
• Publication Number:
  US 20160357319A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

An electronic device and a method for controlling the same are provided. The method includes obtaining a depth image using a depth camera, extracting a hand area including a hand of a user from the obtained depth image, modeling fingers and a palm of the user included in the hand area into a plurality of points, and sensing a touch input based on depth information of one or more of the plurality of modeled points.

Claims

1. A method for controlling an electronic device, the method comprising:

obtaining a depth image using a depth camera;

extracting a hand area including a hand of a user from the obtained depth image;

modeling fingers and a palm of the user included in the hand area into a plurality of points; and

sensing a touch input based on depth information of one or more of the plurality of modeled points.

2. The method according to claim 1, wherein the modeling comprises:

modeling each of an index finger, middle finger, and ring finger of the fingers of the user into a plurality of points;

modeling each of a thumb and little finger of the fingers of the user into one point; and

modeling the palm of the user into one point.

3. The method according to claim 2, wherein the sensing comprises:

in response to sensing that only an end point of at least one finger from among the plurality of points of the index finger and middle finger has been touched, sensing a touch input at the touched point; and

in response to sensing that a plurality of points of at least one finger from among the plurality of points of the index finger and middle finger have been touched, not sensing the touch input.

4. The method according to claim 2, wherein the sensing comprises:

in response to sensing that only end points of two fingers from among the plurality of points of the thumb and index finger have been touched, sensing a multi touch input at the touched point; and

in response to sensing that the plurality of points of the index finger and the one point of the thumb have all been touched, not sensing the touch input.

5. The method according to claim 2, wherein the sensing comprises:

in response to sensing that only end points of two fingers from among the plurality of points of the index fingers of both hands of the user have been touched, sensing a multi touch input at the touched point.

6. The method according to claim 2, wherein the sensing comprises:

in response to sensing that only end points of all fingers from among the plurality of points of all fingers of both hands of the user have been touched, sensing a multi touch input.

7. The method according to claim 1, further comprising:

analyzing a movement direction and speed of the hand included in the hand area,

wherein the extracting comprises extracting the hand of the user based on a movement direction and speed of the hand analyzed in a previous frame.

8. The method according to claim 1, further comprising:

determining whether an object within the obtained depth image is a hand or thing by analyzing the obtained depth image; and

in response to determining that the object within the depth image is a thing, determining a type of the thing.

9. The method according to claim 8, further comprising:

performing functions of the electronic device based on the determined type of the thing and touch position of the thing.

10. An electronic device comprising:

a depth camera configured to obtain a depth image; and

a controller configured to;

extract a hand area including a hand of a user from the obtained depth image,

model the fingers and palm of the user included in the hand area into a plurality of points, and

sense a touch input based on depth information of one or more of the plurality of modeled points.

11. The electronic device according to claim 10, wherein the controller is further configured to:

model each of an index finger, middle finger, and ring finger from among the fingers of the user into a plurality of points;

model each of a thumb and little finger of the fingers of the user into one point; and

model the palm of the user into one point

12. The electronic device according to claim 11, wherein the controller is further configured to:

in response to sensing that only an end point of at least one finger from among the plurality of points of the index finger and middle finger have been touched, sense a touch input at the touched point; and

in response to sensing that a plurality of points of at least one finger from among the plurality of points of the index finger and middle finger have been touched, not sense the touch input.

13. The electronic device according to claim 11, wherein the controller is further configured to:

in response to sensing that only end points of two fingers from among the plurality of points of the thumb and index finger have been touched, sense a multi touch input at the touched point; and

in response to sensing that the plurality of points of the index finger and one point of the thumb have all been touched, not sense the touch input.

14. The electronic device according to claim 11, wherein the controller is further configured to:

in response to sensing that only end points of two fingers from among the plurality of points of the index fingers of both hands of the user have been touched, sense a multi touch input at the touched point.

15. The electronic device according to claim 11, wherein the controller is further configured to:

in response to sensing that only end points of all fingers from among the plurality of points of all fingers of both hands of the user have been touched, sense a multi touch input.

16. The electronic device according to claim 10, wherein the controller is further configured to:

analyze a movement direction and speed of the hand included in the hand area; and

extract the hand of the user based on a movement direction and speed of the hand analyzed in a previous frame.

17. The electronic device according to claim 10, wherein the controller is further configured to determine whether an object within the obtained depth image is the of the user hand or a thing by analyzing the obtained depth image;

and

in response to determining that the object within the depth image is a thing, determine a type of the thing.

18. The electronic device according to claim 17, wherein the controller is further configured to perform functions of the electronic device based on the determined type of the thing and touch position of the thing.

19. The electronic device according to claim 10, further comprising:

an image projector configured to project an image onto a touch area.