Apparatus and method for inputting command using gesture

• US 9,836,127 B2
• Filed: 04/22/2011
• Issued: 12/05/2017
• Est. Priority Date: 02/23/2011
• Status: Active Grant

• Current Assignee: LG Innotek Company Limited (LG Corporation)
• Original Assignee: LG Innotek Company Limited (LG Corporation)
• Inventors: Jung, Woo Kil
• Primary Examiner(s): Sherman, Stephen
• Application Number:
  US14/000,699
• Publication Number:
  US 20130328769A1
• Time in Patent Office: 2,419 Days
• Field of Search: 345156, 715863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Disclosure is a method of inputting commands into displays such as TVs or image processing devices. User'"'"'s hands have been photographed through a camera to recognize the motion of the user'"'"'s hands, so that commands are input according to the motion of user'"'"'s hands instead of conventional input devices such as a mouse and a keyboard.

Claims

1. An input apparatus using a gesture comprising:

at least one camera photographing a gesture image of a user'"'"'s hands introduced into a predetermined detection region near the input apparatus;

a controller displaying content, a virtual image of a device to which a gesture of the user is applied, and a user gesture image that has been photographed in a gesture recognition mode, and executing a command according to the gesture of the user;

a storage unit storing a command code used to run an operation of the controller;

a micro-phone receiving a voice of the user; and

a display outputting the images;

wherein, when a keyboard input mode is executed according to the voice received through the micro-phone, a keyboard input window is displayed, wherein a keyboard virtual image and input values are displayed on the keyboard input window;

wherein, in the keyboard mode, the controller is configured to;

recognize a finger position on the user gesture image based on a position of a user'"'"'s finger in the detection region;

compare the finger position with a position of the keyboard virtual image to recognize the finger position on the keyboard virtual image;

display a feedback image corresponding to the finger position on the keyboard virtual image;

set points to positions of finger joints of the finger position;

detect a key input gesture position on the keyboard virtual image when a key input gesture is detected by recognizing a movement of each of the points; and

display a key image corresponding to the key input gesture position on the keyboard virtual image;

wherein the user'"'"'s finger does not touch any real input device or the virtual image of the device;

wherein, when a mouse input mode is executed according to the voice received through the micro-phone, a virtual mouse image is displayed;

wherein, when the user makes a gesture to grip the virtual mouse image in the mouse mode, a hand image corresponding to the user'"'"'s hand grips the virtual mouse image, and, in this state, when the user moves the user'"'"'s hand, the virtual mouse image moves accordingly;

wherein, when a phone mode is executed according to the voice received through the micro-phone, an input window to display a key pad and phone numbers, which have been input, is displayed; and

wherein all of the key pad and the input window are displayed as semi-transparent images.

2. The input apparatus of claim 1, wherein the controller activates or changes the gesture recognition mode according to the voice received through the micro-phone.

3. The input apparatus of claim 1, wherein the camera is an infrared camera or a visible light camera.

4. The input apparatus of claim 1, wherein the device to which the gesture of the user is applied includes at least one of a keyboard, a mouse, and a phone number key pad.

5. The input apparatus of claim 1, wherein the feedback image is a focus-on image corresponding to the finger position when the key input gesture is not yet detected.

6. The input apparatus of claim 1, wherein the feedback image comprises a plurality of focus-on images displaying a plurality of positions of a plurality of fingers.

7. The input apparatus of claim 1, wherein the at least one camera comprises a plurality of cameras installed at a predetermined position of a display.

8. The input apparatus of claim 1, wherein there is no real input device in the predetermined detection region, and

wherein the at least one camera comprises two cameras to serve as a stereo camera.

9. The input apparatus of claim 1, wherein, in the mouse input mode, when the user makes a gesture to click a right or left button of the mouse, the hand image in which the user'"'"'s hand performs the right-click or left-click of the mouse is displayed, and the operation corresponding to the click is executed.

10. An input method using a gesture of a user, the input method comprising:

photographing a gesture of a user introduced into a predetermined detection region;

simultaneously displaying content and a virtual image of a device to which the gesture of the user is applied in a gesture recognition mode;

displaying a photographed image of the gesture of the user;

displaying a feedback image according to the gesture of the user;

displaying a key image according to the gesture of the user; and

executing a command according to the gesture of the user;

wherein, when a keyboard input mode is executed according to a voice received through a micro-phone, a keyboard input window is displayed, wherein a keyboard virtual image and input values are displayed on the keyboard input window;

wherein, in the keyboard mode, displaying the feedback image comprises;

recognizing a finger position on the photographed image based on a position of a user'"'"'s finger in the detection region;

comparing the finger position with a position of the keyboard virtual image to recognize the finger position on the keyboard virtual image;

displaying the feedback image corresponding to the finger position on the keyboard virtual image; and

setting points to positions of finger joints of the finger position;

wherein displaying the key image comprises;

detecting a key input gesture position on the keyboard virtual image when a key input gesture is detected by recognizing a movement of each of the points; and

displaying the key image corresponding to the key input gesture position on the keyboard virtual image;

wherein the user'"'"'s finger does not touch any real input device or the virtual image of the device;

wherein, when a mouse input mode is executed according to the voice received through the micro-phone, a virtual mouse image is displayed;

wherein, when the user makes a gesture to grip the virtual mouse image in the mouse mode, a hand image corresponding to the user'"'"'s hand grips the virtual mouse image, and, in this state, when the user moves the user'"'"'s hand, the virtual mouse image moves accordingly;

wherein, when a phone mode is executed according to the voice received through the micro-phone, an input window to display a key pad and phone numbers, which have been input, is displayed; and

wherein all of the key pad and the input window are displayed as semi-transparent images.

11. The input method of claim 10, wherein the device to which the gesture of the user is applied includes at least one of a keyboard, a mouse, and a phone number key pad.

12. The input method of claim 10, wherein the feedback image is a focus-on image corresponding to the finger position when the key input gesture is not yet detected.

13. The input method of claim 10, wherein the feedback image comprises a plurality of focus-on images displaying a plurality of positions of a plurality of fingers.

14. The input method of claim 10, wherein photographing the gesture of the user is performed by at least one camera.

15. The input method of claim 14, wherein the at least one camera comprises a plurality of cameras installed at a predetermined position of a display.

16. The input apparatus of claim 10, wherein there is no real input device in the predetermined detection region, and

wherein the at least one camera comprises two cameras to serve as a stereo camera.

17. The input method of claim 10, wherein, in the mouse input mode, when the user makes a gesture to click a right or left button of the mouse, the hand image in which the user'"'"'s hand performs the right-click or left-click of the mouse is displayed, and the operation corresponding to the click is executed.