Detecting and tracking touch on an illuminated surface using a machine learning classifier

• US 9,098,148 B2
• Filed: 03/24/2013
• Issued: 08/04/2015
• Est. Priority Date: 03/14/2012
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Sharma, Vinay
• Primary Examiner(s): Lao, Lun-Yi
• Assistant Examiner(s): Suteerawongsa, Jarurat
• Application Number:
  US13/828,736
• Publication Number:
  US 20130241890A1
• Time in Patent Office: 863 Days
• Field of Search: 345173-179
• US Class Current: 1/1
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886

Abstract

A method for touch detection that is performed by a touch processor in an optical touch detection system is provided. The method includes receiving an image of an illuminated surface in the optical touch detection system, wherein the image is captured by a camera in the optical touch detection system, identifying a set of candidate touch locations in the image, classifying the candidate touch locations in the set of candidate touch locations to generate a set of validated candidate touch locations, wherein classifying the candidate touch locations includes using a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location, and outputting a set of final touch locations.

Claims

1. A method for touch detection performed by a touch processor in an optical touch detection system, the method comprising:

receiving an image of an illuminated surface comprised in the optical touch detection system, wherein the image is captured by a camera comprised in the optical touch detection system;

identifying a set of candidate touch locations in the image, wherein identifying a set of candidate touch locations comprises subtracting a background model from the image to generate a mean-subtracted image, filtering the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a center of the filter surrounded by coefficients of an opposite sign such that a size of a central region corresponds to an expected size of a finger touch, and identifying local extrema in the filtered mean-subtracted image;

classifying the candidate touch locations in the set of candidate touch locations to generate a set of validated candidate touch locations, wherein classifying the candidate touch locations comprises using a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location; and

outputting a set of final touch locations.

2. The method of claim 1, further comprising:

identifying highly saturated pixels in the image, and

pruning the set of validated candidate touch locations to generate the set of final touch locations, wherein pruning the set of validated candidate touch locations comprises pruning the set of validated candidate touch locations based on the identified highly saturated pixels, wherein a validated candidate touch location is not included in the set of final touch locations if the validated candidate touch location corresponds to a highly saturated pixel.

3. The method of claim 2, wherein identifying highly saturated pixels comprises comparing each pixel in the image to a predetermined high saturation threshold and identifying pixels above the high saturation threshold as highly saturated pixels.

4. The method of claim 2, wherein pruning the set of validated candidate touch locations further comprises:

removing a validated candidate touch location from the set of validated candidate touch locations when the validated candidate touch location is not in a set of foreground pixels in the image; and

removing a validated candidate touch location from the set of validated candidate touch locations when the validated candidate touch location is outside a maximum distance from a boundary of a blob formed by the foreground pixels.

5. The method of claim 1, wherein the combination of features comprises at least one feature selected from a group of features consisting of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

6. The method of claim 1, wherein the combination of features consists of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

7. The method of claim 1, wherein classifying the candidate touch locations comprises comparing each pixel to a predetermined detection threshold.

8. The method of claim 1, wherein classifying the candidate touch locations comprises comparing each pixel to a predetermined detection threshold, the predetermined pixel threshold corresponding to a location of the pixel, wherein a predetermined detection threshold for a first pixel location is different from a predetermined detection threshold for a second pixel location.

9. The method of claim 1, wherein the camera is an infrared camera.

10. The method of claim 1, further comprising:

tracking the final touch locations, wherein tracking comprises;

using Kalman filter prediction to compute an expected location of each touch location from a previous image; and

using Munkres'"'"' assignment algorithm to determine which touch locations of the final touch locations correspond to touch locations from the previous image.

11. An optical touch detection system configured for touch detection, the system comprising:

an illuminated surface;

a camera positioned to capture images of the illuminated surface;

a processor coupled to the camera for executing a method, the method comprises;

receiving an image of the illuminated surface captured by the camera;

identifying a set of candidate touch locations in the image, wherein identifying a set of candidate touch locations comprises subtracting a background model from the image to generate a mean-subtracted image, filtering the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a center of the filter surrounded by coefficients of an opposite sign such that a size of a central region corresponds to an expected size of a finger touch, and identifying local extrema in the filtered mean-subtracted image;

classifying the candidate touch locations in the set of candidate touch locations to generate a set of validated candidate touch locations, wherein classifying the candidate touch locations comprises using a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location; and

outputting a set of final touch locations.

12. The optical touch detection system of claim 11, further comprising:

identifying highly saturated pixels in the image, and

pruning the set of validated candidate touch locations to generate the set of final touch locations, wherein pruning the set of validated candidate touch locations comprises pruning the set of validated candidate touch locations based on the identified highly saturated pixels, wherein a validated candidate touch location is not included in the set of final touch locations if the validated candidate touch location corresponds to a highly saturated pixel.

13. The optical touch detection system of claim 12, wherein identifying highly saturated pixels compares each pixel in the image to a predetermined high saturation threshold and identifying pixels above the high saturation threshold as highly saturated pixels.

14. The optical touch detection system of claim 11, wherein the combination of features comprises at least one feature selected from a group of features consisting of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

15. The optical touch detection system of claim 11, wherein the combination of features consists of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location.

16. The optical touch detection system of claim 11, wherein classifying the candidate touch locations comprises comparing each pixel to a predetermined detection threshold, the predetermined pixel threshold corresponding to a location of the pixel, wherein a predetermined detection threshold for a first pixel location is different from a predetermined detection threshold for a second pixel location.

17. The optical touch detection system of claim 11, wherein the camera is an infrared camera.

18. A non-transitory computer readable medium storing software instructions that, when executed by a touch processor comprised in an optical touch detection system, causes the optical touch detection system to perform a method for touch detection, the method comprising:

receiving an image of an illuminated surface comprised in the optical touch etection system, wherein the image is captured by a camera comprised in the optical touch detection system;

identifying a set of candidate touch locations in the image, wherein identifying a set of candidate touch locations comprises subtracting a background model from the image to generate a mean-subtracted image, filtering the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a center of the filter surrounded by coefficients of an opposite sign such that a size of a central region corresponds to an expected size of a finger touch, and identifying local extrema in the filtered mean-subtracted image;

classifying the candidate touch locations in the set of candidate touch locations to generate a set of validated candidate touch locations, wherein classifying the candidate touch locations comprises using a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location; and

outputting a set of final touch locations.