System and method for a virtual keyboard

• US 8,497,841 B1
• Filed: 08/23/2012
• Issued: 07/30/2013
• Est. Priority Date: 08/23/2012
• Status: Expired due to Fees

• Current Assignee: Celluon, Inc.
• Original Assignee: Celluon, Inc.
• Inventors: Sze, Cheng-Feng, Cha, Raymyung, Koo, Jae Eul, Jeong, Koang Hui
• Primary Examiner(s): Beck, Alexander S
• Assistant Examiner(s): Truong, Nguyen H
• Application Number:
  US13/593,344
• Time in Patent Office: 341 Days
• Field of Search: 345156-184
• US Class Current: 345/168
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0489 using dedicated keyboard keys or combinations thereof
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis

Abstract

The present disclosure is directed to a method that includes capturing images of a working area that includes indicia of a virtual keyboard having a plurality of keys and detecting a selected key of the plurality of keys. The detecting of the selected key includes detecting an object in the working area by analyzing the captured images for variations in light intensity in the captured images. The detecting of the object includes generating a first row histogram based on the light intensity in each of the rows in each of the captured images, generating a first column histogram based on the light intensity in each of the columns in each of the captured images, and determining an identification of the selected key based on the first row histogram and the first column histogram. The method also includes outputting the identification of the selected key to a host device.

Claims

1. A method, comprising:

providing a plane of light above a working area, the working area including indicia of a virtual keyboard having a plurality of keys;

capturing images of the working area with an image capture device positioned above the plane of light, the images including rows of pixels and columns of pixels;

detecting a selected key of the plurality of keys, the detecting of the selected key including;

detecting an object adjacent to the selected key by analyzing each of the captured images for variations in light intensity, the detecting of the object including;

generating a first histogram by summing the light intensity in each of a first number of columns;

determining a second number of columns corresponding to a portion of the first histogram that is above a first threshold light intensity value;

generating a second histogram by summing the light intensity in each of a first number of rows in the second number of columns;

determining a second number of rows corresponding to a portion of the second histogram that is above a second threshold light intensity value;

determining an identification of the selected key from the light intensity corresponding to the pixels in the second number of rows and the second number of columns; and

outputting the identification of the selected key to a host device.

2. The method of claim 1, further comprising:

detecting if the object touches a working surface in the working area, the detecting including;

subtracting a background image from each captured image;

determining a first moving average for the first histogram and a second moving average for the second histogram;

determining the second number of columns by determining the portion of the first histogram that is greater than the first moving average; and

determining the second number of rows by determining the portion of the second histogram that is greater than the second moving average.

3. The method of claim 2

wherein the plane of light is configured to produce the light intensity when the object interacts with the plane of light.

4. The method of claim 3 wherein detecting if the object touches the working surface further includes:

determining a dimension of the object in each captured image based on the second number of rows; and

comparing the dimension of the object to a thickness of the plane of light.

5. The method of claim 2 wherein detecting if the object touches the working surface further includes:

counting a number of consecutive captured images that include the object; and

determining if the number of consecutive captured images is above a lower touch threshold and below an upper touch threshold.

6. The method of claim 2 wherein detecting if the object touches the working surface further includes:

determining if a dimension of the object is increasing by comparing successive captured images.

7. The method of claim 1 wherein determining the identification of the selected key based on the first histogram and the second histogram includes:

determining a center of the object by analyzing the second number rows and the second number of columns;

accessing a calibration table to determine coordinates of the center of the object; and

determining the identification of the selected key from the coordinates by accessing a key map.

8. The method of claim 7 wherein determining the center of the object includes:

generating a third histogram based on the second number of TOWS;

generating a fourth histogram based on the second number columns;

determining a first peak of the third histogram; and

determining a second peak of the fourth histogram.

9. A system, comprising:

a working area that includes indicia of a virtual keyboard having a plurality of keys;

a device that includes;

a light source configured to produce a plane of light above the working area;

an image sensor module positioned above the plane of light and configured to capture images of the working area, each captured image including rows of pixels and columns of pixels;

a key selection module configured to analyze the captured images for variations in light intensity, the variations of light intensity associated with an object in the working area, the key selection module further configured to;

generate a first histogram based on the light intensity in each of a first number rows in each of the captured images;

determine a second number of rows from the first histogram that correspond to the light intensity being above a first threshold light intensity value;

generate a second histogram based on the light intensity in each of a first number columns in the second number of rows;

determine a second number of columns from the second histogram that correspond to the light intensity being above a second threshold light intensity value;

determine an identification of a selected key of the plurality of keys based on the light intensity in the second number of rows and the second number of columns; and

output the identification of the selected key to a host device.

10. The system of claim 9 wherein the key selection module includes:

an object detection module configured to detect the object in the captured images;

a touch detection module configured to detect if the object touches a working surface of the working area;

a location module configured to determine a coordinate from the captured images that corresponds to a location on the working surface touched by the object;

a key identification module configured to determine the identification of the selected key based on the coordinate from the captured images; and

a key event generation module configured to output a key event identifier to the host device associated with a text input by a user.

11. The system of claim 10 wherein the key event generation module further includes:

a key event storage module configured to store a current key list associated with the selected key and a previous key list associated with a previously selected key; and

a key blocking detection module configured to compare the current key list and the previous key list to determine if the selected key includes an additional key blocked from the image sensor module.

12. The system of claim 10 wherein the object detection module is further configured to:

subtract a background image from each captured image;

determine a first moving average for the first histogram and a second moving average for the second histogram;

determine segments of the first histogram that are greater than the first moving average;

determine the second number of rows associated with the segments of the first histogram that are greater than the first moving average;

determine segments of the second histogram that are greater than the second moving average; and

determine the second number of columns associated with the segments of the second histogram that are greater than the second moving average.

13. The system of claim 10 wherein the plane of light is configured to produce the light intensity in the captured images when the object interacts with the plane of light.

14. The system of claim 13 wherein the touch detection module is further configured to:

determine a dimension of the object in each captured image based on the second number of rows; and

compare the dimension of the object to a thickness of the plane of light.

15. The system of claim 10 wherein the touch detection module is further configured to count a number of consecutive captured images that include the object and determine if the number of consecutive captured images is above a lower touch threshold and below an upper touch threshold.

16. The system of claim 10 wherein the touch detection module is configured to compare successive captured images and determine if a dimension of the object is increasing.

17. The system of claim 9 wherein the key selection module is configured to analyze the second number of rows and the second number of columns, access a calibration table to determine coordinates of a center of the object, access a key map, and determine the identification of the selected key from the coordinates.

18. The system of claim 17 wherein the key selection module is configured to generate a third histogram based on the subset of rows, generate a fourth histogram based on the subset of columns, determine a first peak of the third histogram, and determine a second peak of the fourth histogram.

19. A system, comprising:

a light source configured to produce a plane of light above a working area, the working area including indicia of a virtual keyboard having a plurality of keys;

an image sensor module positioned above the plane of light and configured to capture images of the working area, each captured image including rows of pixels and columns of pixels;

a processor configured to analyze the captured images for variations in light intensity, the variations of light intensity associated with an object in the working area, the processor further configured to;

generate a first histogram based on the light intensity in each of a first number rows in each of the captured images;

generate a first moving average from the first histogram;

determine a second number of rows from the first histogram that have a light intensity above the first moving average;

generate a second histogram based on the light intensity in each of a first number of columns in the second number of rows;

generate a second moving average from the second histogram;

determine a second number of columns from the second histogram that have a light intensity above the second moving average; and

determine an identification of a selected key of the plurality of keys based on the light intensity in the second number of rows and the second number of columns.

20. The system of claim 19 wherein the processor is further configured to detect a dimension of the object in each captured image based on the second number of rows and compare the dimension of the object to a thickness of the plane of light.

21. The system of claim 19 further comprising a housing, the light source and the image sensor being positioned in the housing.

22. The system of claim 19 wherein the image sensor is positioned at an angle to capture the light intensity reflected off the object when the object crosses the plane of light.