VIRTUAL TOUCH APPARATUS AND METHOD WITHOUT POINTER ON SCREEN

• US 20120206333A1
• Filed: 06/17/2011
• Published: 08/16/2012
• Est. Priority Date: 02/16/2011
• Status: Abandoned Application

• Current Assignee: Seok-Joong Kim
• Original Assignee: Seok-Joong Kim
• Inventors: KIM, Seok-Joong
• Application Number:
  US13/162,984
• Publication Number:
  US 20120206333A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72

Abstract

Provided is a virtual touch apparatus and method for remotely controlling electronic equipment having a display surface. The virtual touch apparatus includes a three-dimensional coordinate calculator and a controller. The three-dimensional coordinate calculator extracts a three-dimensional coordinate of a user'"'"'s body. The controller includes a touch location calculation unit and a virtual touch processing unit. The touch location calculation unit calculates a contact point coordinate where a straight line connecting between a first spatial coordinate and a second spatial coordinate meets the display surface using the first and second spatial coordinates received from the three-dimensional coordinate calculator. The virtual touch processing unit creates a command code for performing an operation corresponding to the contact coordinate received from the touch location calculation unit and inputs the command code into a main controller of the electronic equipment.

Claims

1. A virtual touch apparatus for remotely controlling electronic equipment having a display surface, comprising:

a three-dimensional coordinate calculator extracting a three-dimensional coordinate of a user'"'"'s body; and

a controller comprising a touch location calculation unit calculating a contact point coordinate where a straight line connecting between a first spatial coordinate and a second spatial coordinate meets the display surface using the first and second spatial coordinates received from the three-dimensional coordinate calculator, and a virtual touch processing unit creating a command code for performing an operation corresponding to the contact coordinate received from the touch location calculation unit and inputting the command code into a main controller of the electronic equipment.

2. The virtual touch apparatus of claim 1, wherein the three-dimensional calculator acquires the three-dimensional coordinate using time of flight.

3. The virtual touch apparatus of claim 1, wherein the three-dimensional coordinate calculator is configured to acquire the three-dimensional coordinate by projecting a coded pattern image on the user'"'"'s body and processing an image on which structured light is projected.

4. The virtual touch apparatus of claim 1, wherein the three-dimensional coordinate calculator comprises:

a lighting assembly comprising a light source and a light diffuser and projecting a speckle pattern on the user'"'"'s body;

an image acquisition unit comprising an image sensor to capture the speckle pattern projected on the user'"'"'s body by the lighting assembly; and

a spatial coordinate calculation unit for calculating the three-dimensional coordinate of the user'"'"'s body from the captured speckle pattern.

5. The virtual touch apparatus of claim 4, wherein the three-dimensional coordinate calculator is disposed in plurality on different locations.

6. The virtual touch apparatus of claim 1, wherein the first spatial coordinate is a three-dimensional coordinate of a tip of one user'"'"'s finger or a tip of a pointer gripped by user'"'"'s finger, and the second spatial coordinate is a three-dimensional coordinate of a central point of one of user'"'"'s eyes.

7. The virtual touch apparatus of claim 6, wherein when the virtual touch processing unit determines whether there is a change in the contact point coordinate for a predetermined time or more after the initial contact point coordinate is calculated and there is no change in the contact point coordinate for the predetermined time or more, the virtual touch processing unit creates a command code for performing an operation corresponding to the contact point coordinate, and inputs the command code into the main controller of the electronic equipment.

8. The virtual touch apparatus of claim 6, wherein, when the virtual touch processing unit determines whether there is a change in the contact point coordinate for a predetermined time or more after the initial contact point coordinate is calculated and there is no change in the contact point coordinate for the predetermined time or more, and then the virtual touch processing unit determines whether there is a distance change between the first spatial coordinate and the second spatial coordinate beyond a predetermined distance and there is a distance change beyond the predetermined distance, the virtual touch processing unit creates a command code for performing an operation corresponding to the contact point coordinate, and inputs the command code into the main controller of the electronic equipment.

9. The virtual touch apparatus of claim 7, wherein, when the change of the contact point coordinate is within a predetermined region of the display surface, the contact point coordinate is determined as unchanged.

10. The virtual touch apparatus of claim 1, wherein the first spatial coordinate comprises three-dimensional coordinates of tips of two or more fingers, and the second spatial coordinate comprises a three-dimensional coordinate of the central point of one of user'"'"'s eyes.

11. The virtual touch apparatus of claim 10, wherein the touch location calculation unit receives two or more first spatial coordinates and one second spatial coordinate from the three-dimensional coordinate calculator and calculates two or more contact point coordinates where straight lines connecting between the respective first spatial coordinates and the second spatial coordinate meet the display surface, and the virtual touch processing unit creates a command code for performing an operation corresponding to the contact point coordinates received from the touch location calculation unit and inputs the command code to the main controller of the electronic equipment.

12. The virtual touch apparatus of claim 11, wherein the virtual touch processing unit determines whether there is a change in the contact point coordinates for a predetermined time or more after the initial contact point coordinates are calculated, and when there is no change in the contact point coordinates for the predetermined time or more, the virtual touch processing unit creates command codes for performing an operation corresponding to the contact point coordinates, and inputs the command codes into the main controller of the electronic equipment.

13. The virtual touch apparatus of claim 11, wherein, when the virtual touch processing unit determines whether there is a change in the contact point coordinates for a predetermined time or more after the initial contact point coordinates are calculated and there is no change in the contact point coordinates for the predetermined time or more, and then the virtual touch processing unit determines whether there is a distance change between the first spatial coordinate and the second spatial coordinate beyond a predetermined distance and there is a distance change beyond the predetermined distance, the virtual touch processing unit creates command codes for performing an operation corresponding to the contact point coordinates, and inputs the command codes into the main controller of the electronic equipment.

14. The virtual touch apparatus of claim 12, wherein, when the changes of the contact point coordinates are within a predetermined region of the display surface, the contact point coordinates are determined as unchanged.

15. The virtual touch apparatus of claim 1, wherein the first spatial coordinate comprises three-dimensional coordinates of tips of one or more fingers provided by two or more users, and the second spatial coordinate comprises three-dimensional coordinates of the central points of one of both eyes of two or more users.

16. The virtual touch apparatus of claim 15, wherein the touch location calculation unit receives one or more first spatial coordinates and one second spatial coordinate for each user and calculates two or more contact point coordinates where straight lines connecting between the one or more first spatial coordinates and the one second spatial coordinate meet the display surface, and the virtual touch processing unit creates a command code for performing an operation corresponding to the contact point coordinates received from the touch location calculation unit and inputs the command code to the main controller of the electronic equipment for each user.

17. The virtual touch apparatus of claim 16, wherein the virtual touch processing unit determines for each user whether there is a change in the contact point coordinates for a predetermined time or more after the initial contact point coordinates are calculated, and when there is no change in the contact point coordinates for the predetermined time or more, the virtual touch processing unit creates command codes for performing an operation corresponding to the contact point coordinates, and inputs the command codes into the main controller of the electronic equipment for each user.

18. The virtual touch apparatus of claim 16, wherein, when the virtual touch processing unit determines for each user whether there is a change in the contact point coordinates for a predetermined time or more after the initial contact point coordinates are calculated and there is no change in the contact point coordinates for the predetermined time or more, and then the virtual touch processing unit determines for each user whether there is a distance change between the first spatial coordinate and the second spatial coordinate beyond a predetermined distance and there is a distance change beyond the predetermined distance, the virtual touch processing unit creates command codes for performing an operation corresponding to the contact point coordinates, and inputs the command codes into the main controller of the electronic equipment for each user.

19. The virtual touch apparatus of claim 17, wherein, when the changes of the contact point coordinates are within a predetermined region of the display surface, the contact point coordinates are determined as unchanged.

20. A virtual touch method for remotely controlling electronic equipment having a display surface, the method comprising:

projecting, by a lightning assembly, a speckle pattern on a user'"'"'s body;

capturing, by an image acquisition unit, the speckle pattern projected on the user'"'"'s body by the lighting assembly;

processing, by a spatial coordinate calculation unit, the captured speckle pattern to calculate a three-dimensional spatial coordinate of the user'"'"'s body;

calculating, by a touch location calculation unit, a contact point coordinate where a straight line connecting between a first spatial coordinate that is a three-dimensional coordinate of a tip of one of fingers or a tip of a pointer gripped by the fingers and a second spatial coordinate that is a three-dimensional coordinate of a central point of one of user'"'"'s eyes meets the display surface; and

creating, by a touch processing unit, a command code for performing an operation corresponding to the contact point coordinate received from the touch location calculation unit to input the command code into a main controller of the electronic equipment.

21. The virtual touch method of claim 20, wherein, in the creating of the command code, when the virtual touch processing unit determines whether there is a change in the contact point coordinate for a predetermined time or more after the initial contact point coordinate is calculated and there is no change in the contact point coordinate for the predetermined time or more, the virtual touch processing unit creates a command code for performing an operation corresponding to the contact point coordinate and inputs the command code into the main controller of the electronic equipment.

22. The virtual touch method of claim 20, wherein, in the creating of the command code, when the virtual touch processing unit determines whether there is a change in the contact point coordinate for a predetermined time or more after the initial contact point coordinate is calculated and there is no change in the contact point coordinate for the predetermined time or more, and then the virtual touch processing unit determines whether there is a distance change between the first spatial coordinate and the second spatial coordinate beyond a predetermined distance and there is a distance change beyond the predetermined distance, the virtual touch processing unit creates a command code for performing an operation corresponding to the contact point coordinate, and inputs the command code into the main controller of the electronic equipment.

23. The virtual touch method of claim 21, wherein, when the change of the contact point coordinate is within a predetermined region of the display surface, the contact point coordinate is determined as unchanged.