Virtual optical input device with feedback and method of controlling the same

• US 8,698,753 B2
• Filed: 02/04/2009
• Issued: 04/15/2014
• Est. Priority Date: 02/28/2008
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Jung, Yung Woo, Shin, Yun Sup
• Primary Examiner(s): Boddie, William
• Assistant Examiner(s): Elahi, Towfiq
• Application Number:
  US12/365,653
• Publication Number:
  US 20090219251A1
• Time in Patent Office: 1,896 Days
• Field of Search: 345/170, 382/174
• US Class Current: 345/170
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting

Abstract

Provided are a virtual optical input device capable of providing feedback and a method of controlling the same. In the method, a portion of an input means such as a finger, and a portion of a shadow of the input means generated by a light source are detected through image processing. Physical variations formed between them are used to detect the touch contact of the input means, calculate the position and motion of the input means, and provide the corresponding feedback.

Claims

1. A method for a mobile device to control a virtual input device, comprising:

displaying information on a display unit;

forming a virtual input pattern for detection of user input on an input region of a display plane outside the mobile device by using visible light;

detecting a distance difference between an input means and an invisible shadow of the input means, wherein the invisible shadow of the input means is generated by an invisible light;

detecting a relative velocity between the input means and the invisible shadow of the input means;

determining a contact of the input means on the virtual input pattern based on both the distance difference and when a sign value of the relative velocity between the input means and the invisible shadow changes, wherein the sign value represents a positive value or a negative value; and

displaying, as a feedback, feedback information on the contact through a projector on a feedback region of the display plane,

wherein the feedback region is apart from the input region.

2. The method according to claim 1,

wherein determining the contact comprises:

capturing an image of the input means on the virtual input pattern;

calculating a portion of the input means, a portion of the invisible shadow, and the related positions from the captured image; and

using the calculated position information to determine the contact of the input means.

3. The method according to claim 2, wherein the portion of the input means and the portion of the invisible shadow are respectively an end of the input means and an end of the invisible shadow.

4. The method according to claim 2, wherein the position related to the portion of the input means and the portion of the invisible shadow is a position that is offset by a predetermined distance from the portion of the input means and the portion of the invisible shadow.

5. The method according to claim 1, wherein the virtual input pattern is a keyboard pattern, and the displaying of the feedback comprises displaying an input value, corresponding to a button pressed by the user among the keyboard pattern, on the feedback region of the display plane.

6. The method according to claim 1, wherein the displaying of the feedback comprises displaying the trace of a movement of the input means.

7. The method according to claim 1, wherein the virtual input pattern and the feedback is provided by one imaging device.

22. The method according to claim 1, wherein determining the contact comprises:

determining the contact based on the distance difference; and

complementing the contact based on when the sign value of the relative velocity changes.

8. A virtual input device comprising:

a display unit displaying information;

a user input detector configured to;

generate a virtual input pattern for detection of user input on an input region of a display plane outside a mobile device;

detect a distance difference between an input means and an invisible shadow of the input means, wherein the invisible shadow of the input means is generated by an invisible light;

detect a relative velocity between the input means and the invisible shadow of the input means; and

determine a contact of the input means on the virtual input pattern based on both the distance difference and when a sign value of the relative velocity between the input means and the invisible shadow changes, wherein the sign value represents a positive value or a negative value;

a projector displaying, as a feedback, feedback information on a contact of a portion of the input means on a feedback region of the display plane, wherein the feedback region is apart from the input region; and

a controller executing, when a portion of the input means contacts the virtual input pattern, a command corresponding to the contact point and controlling an image processor so that the projector displays the feedback.

9. The virtual input device according to claim 8, wherein the user input detector comprises:

an input pattern generator generating the virtual input pattern;

an image receiver capturing the input pattern generated by the input generator, a portion of the input means, and an invisible shadow image corresponding to the portion of the input means; and

an image processor detecting the positions related to a portion of the input means and a portion of the invisible shadow from the image received by the image receiver.

10. The virtual input device according to claim 9, wherein the input pattern generator comprises:

a light source;

a lens condensing light emitted from the light source; and

a filter passing the light emitted from the lens and having a pattern generating the input pattern.

11. The virtual input device according to claim 8, wherein the feedback is an image.

12. The virtual input device according to claim 8, wherein the virtual input pattern is a keyboard pattern, and the projector displays an input value, corresponding to a button pressed by the user among the keyboard pattern, on a portion of the feedback region of the display plane.

13. The virtual input device according to claim 8, wherein the projector displays a movement trace of the input means on the feedback region of the display plane.

14. The virtual input device according to claim 9, wherein the input pattern generator and the projector comprise one imaging device.

15. The virtual input device according to claim 9, wherein the image receiver is disposed under the input pattern generator such that the image receiver is spaced apart from the input pattern generator by a predetermined distance.

16. The virtual input device according to claim 8, wherein the controller determines the contact of the input means by detecting at least one of the distance, the angle, the relative velocity, and the acceleration between the position related to the portion of the input means and the position related to the portion of the invisible shadow.

17. A mobile device comprising:

a display unit displaying information;

a wireless communication unit performing wireless communication with a wireless communication system or another mobile device;

a user input detector configured to;

generate a virtual input pattern for detection of user input on an input region of a display plane outside the mobile device;

detect a distance difference between an input means and an invisible shadow of the input means, wherein the invisible shadow of the input means is generated by an invisible light;

detect a relative velocity between the input means and the invisible shadow of the input means; and

determine a contact of the input means on the virtual input pattern based on both the distance difference and when a sign value of the relative velocity between the input means and the invisible shadow changes, wherein the sign value represents positive value or negative value;

a projector displaying, as a feedback, feedback information on a contact of a portion of the input means on a feedback region of the display plane, wherein the feedback region is apart from the input region;

a memory unit storing the input pattern and the corresponding command; and

a control unit controlling the operations of the above components.

18. The mobile device according to claim 17, wherein the user input detector comprises:

an input pattern generator generating the virtual input pattern;

an image receiver capturing the input pattern generated by the input generator, a portion of the input means, and an invisible shadow image corresponding to the portion of the input means; and

an image processor detecting the position related to a portion of the input means and the position related to a portion of the invisible shadow from the image received by the image receiver.

19. The mobile device according to claim 17, wherein the virtual input pattern is a keyboard pattern, and the projector displays an input value, corresponding to a button pressed by the user among the keyboard pattern, on the feedback region of the display plane.

20. The mobile device according to claim 17, wherein the projector displays a movement trace of the input means on the feedback region of the display plane.

21. The mobile device according to claim 17, wherein the input pattern generator and the projector comprise one imaging device.