Apparatus and method for remote control using camera-based virtual touch

• US 10,234,954 B2
• Filed: 02/23/2015
• Issued: 03/19/2019
• Est. Priority Date: 02/22/2014
• Status: Active Grant

• Current Assignee: Vtouch Co Ltd
• Original Assignee: Vtouch Co Ltd
• Inventors: Kim, Seok-joong
• Primary Examiner(s): Hong, Richard J
• Application Number:
  US15/120,664
• Publication Number:
  US 20170075427A1
• Time in Patent Office: 1,485 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A remote control apparatus comprises an image acquisition unit for acquiring an image of the body of a user by photographing the image, a location calculation unit for calculating coordinate data representing the locations of a first coordinate (eye) and a second coordinate (a finger or a pointer) of the body of the user by means of the image received from the image acquisition unit, and a match confirmation unit for checking whether the locations of the first and second coordinates of the body of the user match the extended connecting straight line, the coordinates having been confirmed on the basis of the camera lens-based coordinate data calculated in the location calculation unit, and if the match is confirmed, then concluding the setup intention of the user to be an affirmative.

Claims

1. A remote control apparatus using a camera-based virtual touch, comprising:

an image obtaining unit obtaining two-dimensional (2D) image of a user'"'"'s body through image capturing;

a position calculator calculating coordinate data indicating positions of first coordinates (an eye) and second coordinates (a finger or pointer) of the user'"'"'s body on the 2D image of the user'"'"'s body using the 2D image received from the image obtaining unit; and

a matching identifier determining whether the coordinate data indicating the positions of the first coordinates of the user'"'"'s body is substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body, and determining that the user has a setting intention in response to determination that the coordinate data indicating the position of the first coordinates of the user'"'"'s body is substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body,

wherein the coordinate data produced by the position calculator is two-dimensional (2D) data and is coordinate data at a position viewed from a camera lens of the image obtaining unit.

2. The remote control apparatus of claim 1, further comprising:

a virtual touch setting unit determining that the user has the setting intention if the matching is identified by the matching identifier, determining whether there is a variation in the coordinate data of the first and second coordinates, and performing an operation set to correspond to varied contact coordinate data; and

a driving controller, if a command code generated from the virtual touch setting unit is inputted, converting the command code into a control command signal for controlling driving of a corresponding, electronic device, and sending the control command signal to the electronic device.

3. The remote control apparatus of claim 2, wherein the control command signal sent from the driving controller is sent through a short-range wireless communication scheme including an infrared (IR) signal, WiFi, Bluetooth, or Zigbee.

4. The remote control apparatus of claim 1, wherein the virtual touch setting unit includes a gesture detector, if the matching is identified by the matching identifier, detecting the variation in the coordinate data of the first and second coordinates obtained by the image obtaining unit and detecting the user'"'"'s finger motion (gesture), an electronic device detector comparing electronic device information previously stored in an electronic device information database (DB) based on the gesture detected by the gesture detector to detect the corresponding electronic device, an electronic device setting unit setting a control signal according to control of driving of the electronic device detected based on the gesture detected by the gesture detector, and a virtual touch processor generating a command code for controlling driving of the electronic device based on the control signal set by the electronic device setting unit.

5. The remote control apparatus of claim 4, wherein the electronic device information DB stores product information including a manufacturer and model name of an electronic device provided in a home, protocol information applied to per-device operation control, and per-device control method information.

6. A remote control method using a camera-based virtual touch, comprising the steps of:

(A) obtaining a two-dimensional (2D) image of a user'"'"'s body by a camera and calculating coordinate data indicating positions of first coordinates (an eye) and second coordinates (a finger or pointer) of a user'"'"'s body on the 2D image of the user'"'"'s body;

(B) determining whether the coordinate data indicating the position of the first coordinates of the user'"'"'s body is substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body;

(C) detecting a variation in the obtained coordinate data of the first and second coordinates to detect the user'"'"'s motion (gesture) in response to determination that the coordinate data indicating the position of the first coordinates of the user'"'"'s body is substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body; and

(D) immediately determining that the user has a setting intention to perform a view setting distinctively for the view setting in response to determination that the coordinate data indicating the position of the first coordinates of the user'"'"'s body substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body,

wherein the calculated coordinate data is two-dimensional (2D) data and is coordinate data at a position viewed from a camera lens of the camera.

7. A remote control method using a camera-based virtual touch, comprising the steps of:

(a) obtaining a two-dimensional (2D) image of a user'"'"'s body by a camera and calculating coordinate data indicating positions of first coordinates (an eye) and second coordinates (a finger or pointer) of a user'"'"'s body on the 2D image of the user'"'"'s body;

(b) determining whether the coordinate data indicating the position of the first coordinates of the user'"'"'s body is substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body;

(c) detecting a variation in the obtained coordinate data of the first and second coordinates to detect the user'"'"'s motion (gesture) in response to determination that the coordinate data indicating the position of the first coordinates of the user'"'"'s body is substantially identical to the coordinate data indicating the position of the second coordinates of the user'"'"'s body; and

(d) generating a control command signal for controlling driving of an electronic device based on the detected motion of a finger and sending the control command signal to the electronic device,

wherein the calculated coordinate data is two-dimensional (2D) data and is coordinate data at a position viewed from a camera lens of the camera.

8. The remote control method of claim 7, wherein step (d) includes the steps of (d1) comparing electronic device information previously stored in an electronic device information database (DB) based on the detected gesture by performing the view setting and a consecutive operation to detect a corresponding electronic device and to set an electronic device to be controlled for operation through a virtual touch, (d2) setting a control signal according to control of driving of the electronic device set through the user'"'"'s motion (gesture) detected based on the 2D coordinate data, (d3) converting into a control command signal for controlling of driving of the electronic device based or the set control signal and generating a control command code for a corresponding electronic device, and (d) converting the generated command code into a control command signal for controlling operation of the electronic device and sending the control command signal to the electronic device.

9. The remote control method of claim 8, wherein step (d2) includes setting the second coordinates of the user moving in at least one direction of up, down, left, and right with respect to the extension line of the camera lens, the first coordinates, and the second coordinates as a control signal indicating selection or control of a particular icon displayed on a display unit or a motion of a joystick, setting the second coordinates of the user moving along a straight line from left to right (or from right to left) or from down to up (or from up to down) with respect to the extension line of the camera lens, the first coordinates, and the second coordinates as a control signal for controlling a left-to-right turnon/off switch or up-to-down turnon/off switch, and setting the second coordinates of the user moving along a curved line from down to up (or from up to down) with respect to the extension line of the camera lens, the first coordinates, and the second coordinates as a control signal for controlling a degree of rotation for adjusting a magnitude through a rotating wheel.

10. The remote control method of claim 8, wherein the control command signal sent in step (d4) is sent through a short-range wireless communication scheme including an infrared (IR) signal, Wi-Fi, Bluetooth, or Zigbee.