Device and method for controlling mouse pointer

• US 8,957,857 B2
• Filed: 01/06/2011
• Issued: 02/17/2015
• Est. Priority Date: 02/05/2010
• Status: Active Grant

• Current Assignee: Industry Academic Cooperation Foundation, Samsung Electronics Co. Ltd.
• Original Assignee: Industry Academic Cooperation Foundation, Samsung Electronics Co. Ltd.
• Inventors: Lee, Dong-Hyuk, Kwon, Mu-Sik, Lee, Sangyoun, Toh, Kar-Ann, Lee, Hyobin, Kim, Youngsung, Oh, Beom-Seok, Kim, Sooyeon
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Brown, Jr., Howard D
• Application Number:
  US12/985,915
• Publication Number:
  US 20110193778A1
• Time in Patent Office: 1,503 Days
• Field of Search: 382/103, 382/107, 345/158
• US Class Current: 345/158
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Disclosed is a device for controlling a mouse pointer, providing a display unit; an image photographing unit for photographing images of a first object and a second object; and a controller for setting a point between the first object and the second object detected from the photographed images as a position of a mouse pointer on the display unit, and when a distance between the first object and the second object is less than a predetermined distance, determining that a user selection instruction for the point has been input. The device detects movement of fingers using differential images according to the movement of the fingers, so that even when a continuously changing surrounding lighting or a user face having the similar color with the finger is included in a background, it is possible to accurately identify the movement of the fingers.

Claims

1. A mobile communication terminal for controlling a pointer, the mobile communication terminal comprising:

a display unit;

an image photographing unit for photographing images of a first finger and a second finger; and

a controller configured for;

setting a point between the first finger and the second finger detected from the photographed images as a position of the pointer on the display unit, when the first finger and the second finger are not in contact with each other, and

when a distance between the first finger and the second finger is less than a predetermined distance, determining that a user selection instruction for the point has been input.

2. The mobile communication terminal of claim 1, further comprising a storage unit for storing the photographed images,

wherein the controller obtains differential images including pixels, in which a brightness value is determined according to a difference of brightness values of pixels arranged on an identical position of a before image and an after image between two sequential images among the photographed images and detects the first finger and the second finger using the differential images.

3. The mobile communication terminal of claim 1, wherein the controller is configured for dragging and dropping an icon displayed on the display unit when the first and second fingers are pursed, moved, and then opened by at least the predetermined distance.

4. The mobile communication terminal of claim 1, wherein the controller is configured for double-clicking an icon displayed on the display unit when the first and second fingers are pursed, and then opened by at least the predetermined distance.

5. The mobile communication terminal of claim 2, wherein the controller groups the differential images in a predetermined number of images based on time, obtains an average differential image including pixels determined by an average value of brightness values of pixels arranged on an identical position of the differential images within each of the groups, and detects the first finger and the second finger using the average differential image.

6. The mobile communication terminal of claim 5, wherein the controller divides the average differential image into equal blocks in a predetermined ratio, compares a sum of brightness values of pixels included in each block with a predetermined brightness value, maintains the brightness value of each pixel within an effective block having a sum equal to or larger than the predetermined brightness value, sets the brightness value of each pixel within a non-effective block having the sum less than the predetermined brightness value as zero, groups adjacent effective blocks among the effective blocks into a single group, and detects the first finger and the second finger form the group.

7. The mobile communication terminal of claim 6, wherein the controller determines candidates of the first finger and the second finger from the grouped effective blocks, extracts a boundary portion of each of the candidates of the first finger and the second finger with a brightness value of pixels included in the determined candidates of the first finger and the second finger, detects a portion in which a sign of a derivative changes in a curve of the extracted boundary portion as end portions of the first finger and the second finger, and sets a point between the end portions of the first finger and the second finger as the position of the pointer.

8. The mobile communication terminal of claim 7, wherein in the detection of the end portions of the first finger and the second finger, the controller compares the end portions of the first finger and the second finger with previously detected two end portions, and when a range of a movement is larger than a predetermined range, the controller returns the previously detected two end portions.

9. A method for controlling a pointer by a mobile communication terminal, the method comprising the steps of:

photographing images of a first finger and a second finger;

setting a point between the first finger and the second finger detected from the photographed images as a position of the pointer, when the first finger and the second finger are not in contact with each other; and

when a distance between the first finger and the second finger is less than a predetermined distance, determining that a user selection instruction for the point has been input.

10. The method of claim 9, further comprising obtaining differential images including pixels, in which a brightness value is determined according to a difference of brightness values of pixels arranged on an identical position of a before image and an after image between two sequential images among the photographed images and detecting the first finger and the second finger using the differential images.

11. The method of claim 9, further comprising dragging and dropping an icon displayed on a display unit of the mobile communication terminal, when the first and second fingers are pursed, moved, and then opened by at least the predetermined distance.

12. The method of claim 9, further comprising double-clicking an icon displayed on the display unit, when the first and second fingers are pursed, and then opened by at least the predetermined distance.

13. The method of claim 10, further comprising grouping the differential images in a predetermined number of images based on time and obtaining an average differential image including pixels determined by an average value of brightness values of pixels arranged on an identical position of the differential images within each of the groups, and detecting the first finger and the second finger using the average differential image.

14. The method of claim 13, further comprising dividing the average differential image into equal blocks in a predetermined ratio, comparing a sum of brightness values of pixels included in each block with a predetermined brightness value, maintaining the brightness value of each pixel within an effective block having a sum equal to or larger than the predetermined brightness value, setting the brightness value of each pixel within a non-effective block having the sum less than the predetermined brightness value as zero, grouping adjacent effective blocks among the effective blocks into a single group, and detecting the first finger and the second finger form the group.

15. The method of claim 14, further comprising determining candidates of the first finger and the second finger from the grouped effective blocks, extracting a boundary portion of each of the candidates of the first finger and the second finger with a brightness value of pixels included in the determined candidates of the first finger and the second finger, detecting a portion in which a sign of a derivative changes in a curve of the extracted boundary portion as end portions of the first finger and the second finger, and setting a point between the end portions of the first finger and the second finger as the position of the pointer.

16. The method of claim 15, wherein in the detection of the end portions of the first finger and the second finger, the end portions of the first finger and the second finger are compared with previously detected two end portions, and when a range of a movement is larger than a predetermined range, the previously detected two end portions are returned.