Detecting and tracking touch on an illuminated surface using a mean-subtracted image

• US 9,542,045 B2
• Filed: 03/14/2013
• Issued: 01/10/2017
• Est. Priority Date: 03/14/2012
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Sharma, Vinay
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US13/828,641
• Publication Number:
  US 20130241889A1
• Time in Patent Office: 1,398 Days
• Field of Search: 345173-179, 178 1809- 1811
• US Class Current: 1/1
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886

Abstract

A method for touch detection performed by a touch processor in an optical touch detection system is provided that includes receiving an image of an illuminated surface included in the optical touch detection system, wherein the image is captured by a camera included in the optical touch detection system, subtracting a background model from the image to generate a mean-subtracted image, identifying a set of candidate touch locations in the mean-subtracted image, classifying the candidate touch locations in the set of candidate touch locations to generate a set of validated candidate touch locations, and outputting a set of final touch locations determined from the set of validated candidate touch locations.

Claims

1. A method for touch detection performed by a touch processor in an optical touch detection system, the method comprising:

receiving, with one or more processors, an image of an illuminated surface included in the optical touch detection system, wherein the image is captured by a camera included in the optical touch detection system;

subtracting, with the one or more processors, a background model from the image to generate a mean-subtracted image;

filtering the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a central region of the filter surrounded by coefficients of an opposite sign such that a size of the central region corresponds to an expected size of a finger touch;

identifying, with the one or more processors, local extrema in the filtered mean-subtracted image to generate a set of candidate touch locations;

classifying, with the one or more processors, the candidate touch locations by comparing each of the local extrema to one or more detection thresholds to generate a set of validated candidate touch locations; and

outputting, with the one or more processors, a set of final touch locations determined from the set of validated candidate touch locations.

2. The method of claim 1, further comprising:

identifying highly saturated pixels in the image; and

pruning the set of validated candidate touch locations to generate the set of final touch locations, wherein pruning the set of validated candidate touch locations comprises pruning the set of validated candidate touch locations based on the identified highly saturated pixels, wherein a validated candidate touch location is not included in the set of final touch locations if the validated candidate touch location corresponds to a highly saturated pixel.

3. The method of claim 2, wherein identifying highly saturated pixels comprises comparing each pixel in the image to a predetermined high saturation threshold and identifying pixels above the high saturation threshold as highly saturated pixels.

4. The method of claim 2, wherein pruning the set of validated candidate touch locations further comprises:

removing a validated candidate touch location from the set of validated candidate touch locations when the validated candidate touch location is not in a set of foreground pixels in the image; and

removing a validated candidate touch location from the set of validated candidate touch locations when the validated candidate touch location is outside a maximum distance from a boundary of a blob formed by the foreground pixels.

5. The method of claim 1, wherein classifying the candidate touch locations comprises using a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location.

6. The method of claim 5, wherein the combination of features comprises at least one feature selected from a group of features consisting of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

7. The method of claim 5, wherein the combination of features consists of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

8. The method of claim 1, wherein a detection threshold for a first extrema pixel location is different from a detection threshold for a second extrema pixel location.

9. The method of claim 1, wherein the camera is an infrared camera.

10. The method of claim 1, further comprising:

tracking the final touch locations, wherein tracking comprises;

using Kalman filter prediction to compute an expected location of each touch location from a previous image; and

using Munkres'"'"' assignment algorithm to determine which touch locations of the final touch locations correspond to touch locations from the previous image.

11. An optical touch detection system configured for touch detection, the system comprising:

an illuminated surface;

a camera positioned to capture images of the illuminated surface; and

one or more processors configured to;

receive an image of the illuminated surface captured by the camera;

subtract a background model from the image to generate a mean-subtracted image;

identify local extrema in the mean-subtracted image to generate a set of candidate touch locations;

classify the candidate touch locations in the set of candidate touch locations by comparing each of the local extrema to one or more detection thresholds to generate a set of validated candidate touch locations; and

output a set of final touch locations determined from the set of validated candidate touch locations.

12. The optical touch detection system of claim 11, wherein the one or more processors are further configured to:

identify highly saturated pixels in the image; and

prune the set of validated candidate touch locations, based on the identified highly saturated pixels, to generate the set of final touch locations, wherein a validated candidate touch location is not included in the set of final touch locations if the validated candidate touch location corresponds to a highly saturated pixel.

13. The optical touch detection system of claim 11, wherein the one or more processors are further configured to use a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location.

14. The optical touch detection system of claim 11, wherein the combination of features comprises at least one feature selected from a group of features consisting of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

15. The optical touch detection system of claim 11, wherein the one or more processors are further configured to:

compare each pixel to a predetermined detection threshold, the predetermined pixel threshold corresponding to a location of the pixel, wherein a predetermined detection threshold for a first pixel location is different from a predetermined detection threshold for a second pixel location.

16. The optical touch detection system of claim 11, wherein the one or more processors are further configured to:

filter the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a center of the filter surrounded by coefficients of an opposite sign such that a size of a central region corresponds to an expected size of a finger touch; and

identify local extrema in the filtered mean-subtracted image.

17. The optical touch detection system of claim 11, wherein the camera is an infrared camera.

18. A non-transitory computer readable medium storing software instructions that, when executed by a touch processor included in an optical touch detection system, cause the optical touch detection system to:

receive an image of an illuminated surface included in the optical touch detection system, wherein the image is captured by a camera included in the optical touch detection system;

subtract a background model from the image to generate a mean-subtracted image;

identify local extrema in the mean-subtracted image to generate a set of candidate touch locations;

classify the candidate touch locations in the set of candidate touch locations by comparing each of the local extrema to one or more detection thresholds to generate a set of validated candidate touch locations; and

output a set of final touch locations determined from the set of validated candidate touch locations.