Detecting and tracking objects in images

• US 7,853,041 B2
• Filed: 01/06/2006
• Issued: 12/14/2010
• Est. Priority Date: 01/07/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Shamaie, Atid
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US11/326,345
• Publication Number:
  US 20060188849A1
• Time in Patent Office: 1,803 Days
• Field of Search: 382/103, 382/159, 382/168, 382/181, 382/199, 382/276, 382/291, 382/305, 382/312, 382/106, 382/116, 382/124, 382/154, 345/156, 345/158, 345/173, 345/180, 345/594, 348/169, 348/207.99, 348/208.14, 434/85
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

According to one disclosed method, coordinates in a multi-dimensional space are determined for an image point characterizing a particular object. An equation describing a model in the multi-dimensional space is provided. The model is characteristic of a set of training images of one or more other objects. The coordinates are applied to the equation to determine a distance between the image point and the model. Based on the determined distance, a determination is made as to whether the particular object matches the one or more other objects.

A set of training images may be received. A multi-dimensional space (e.g., eigenspace) may be determined based on the set of training images. A set of training points may be generated by projecting the set of training images into the multi-dimensional space. An equation describing a model in the multi-dimensional space that is characteristic of the set of training points may be determined.

Claims

1. A method comprising:

using one or more processing devices to determine coordinates for an image point in a multi-dimensional eigenspace, the image point in the multi-dimensional eigenspace characterizing an image of an object, and the multi-dimensional eigenspace having been defined based on a set of training images of one or more fingers;

using the one or more processing devices to access an equation that describes a geometric model in the multi-dimensional eigenspace, the geometric model being characteristic of the set of training images of the one or more other fingers, and the equation that describes the geometric model including an equation that describes a cone in the multi-dimensional eigenspace;

using the one or more processing devices to apply the coordinates determined for the image point in the multi-dimensional eigenspace to the equation that describes the geometric model in the multi-dimensional eigenspace to determine a distance between the image point and the geometric model in the multi-dimensional eigenspace; and

using the one or more processing devices to determine whether the object in the image includes a finger based on the determined distance between the image point and the geometric model in the multi-dimensional eigenspace.

2. The method of claim 1 further comprising:

receiving the image of the object, the image having a number of data elements, and the multi-dimensional eigenspace having a dimensionality that is lower than the number of data elements; and

wherein determining coordinates for the image point in the multi-dimensional eigenspace comprises projecting the image into the multi-dimensional eigenspace to produce the coordinates for the image point in the multi-dimensional eigenspace.

3. The method of claim 2 wherein accessing an equation that describes a geometric model in the multi-dimensional eigenspace includes accessing an equation that describes a geometric model of a set of training points in the multi-dimensional eigenspace, each of the training points in the set corresponding to one or more images in the set of training images.

4. The method of claim 2 further comprising normalizing the image of the object before projecting the image into the multi-dimensional eigenspace.

5. The method of claim 4 wherein normalizing the image of the object includes normalizing the image of the object to account for variations in brightness.

6. The method of claim 4 wherein normalizing the image of the object comprises applying a histogram equalization technique to the image of the object.

7. The method of claim 1 wherein accessing an equation that describes a geometric model in the multi-dimensional eigenspace includes accessing an equation that describes a geometric model in the multi-dimensional eigenspace and that includes both an equation that describes a cone in the multi-dimensional eigenspace and an equation that describes a line in the multi-dimensional eigenspace.

8. The method of claim 7 wherein applying the coordinates determined for the image point in the multi-dimensional eigenspace to the equation that describes the geometric model in the multi-dimensional eigenspace includes:

selecting an equation from among the equation that describes a cone in the multi-dimensional eigenspace and the equation that describes a line in the multi-dimensional eigenspace; and

applying the coordinates determined for the image point in the multi-dimensional eigenspace to the selected equation.

9. The method of claim 1 wherein accessing an equation that describes the geometric model in the multi-dimensional eigenspace and that includes an equation that describes a cone in the multi-dimensional eigenspace includes accessing an equation that takes the form:

H(x, y, z)=(x−

a)²/d²+(y −

b)²/e²−

(z−

c)²/f²

where the terms a, b, c, d, e, and f represent constants.

10. A method comprising:

receiving, at a computer system that includes one or more processing devices, an image of a particular an object, the image having a number of data points;

using the one or more processing devices to project the image into a multi-dimensional eigenspace having a dimensionality that is lower than the number of data points to produce coordinates for an image point in the multi-dimensional eigenspace, the image point in the multi-dimensional eigenspace characterizing the image of the object, and the multi-dimensional eigenspace having been defined based on a training set of images of one or more fingers;

using the one or more processing devices to access an equation that describes a geometric model in the multi- dimensional eigenspace, the geometric model being a model of a set of training points in the multi-dimensional eigenspace, each of the training points in the set corresponding to one or more images in the training set of images of the one or more fingers, and the equation that describes the geometric model including an equation that describes a cone in the multi-dimensional eigenspace;

using the one or more processing devices to apply the coordinates for the image point to the equation that describes the geometric model of the set of training points in the multi-dimensional eigenspace to determine a distance between the image point and the model in the eigenspace; and

using the one or more processing devices to determine whether the object in the image includes a finger based on the determined distance between the image point and the geometric model in the multi-dimensional eigenspace.

11. The method of claim 10 wherein applying the coordinates for the image point to the equation that describes the geometric model of the set of training points in the multi-dimensional eigenspace includes:

selecting an equation from among the equation that describes a cone in the multi-dimensional eigenspace and a different equation that describes a different geometric form; and

applying the coordinates for the image point to the selected equation.

12. A system comprising:

a camera; and

a processing device coupled to the camera and configured to;

determine coordinates for an image point in a multi-dimensional eigenspace, the image point in the multi-dimensional eigenspace characterizing an image of an object, and the multi-dimensional eigenspace having been defined based on a set of training images of one or more fingers;

access an equation that describes a geometric model in the multi-dimensional eigenspace, the geometric model being characteristic of the set of training images of the one or more fingers, and the equation that describes the geometric model including an equation that describes a cone in the multi-dimensional eigenspace;

apply the coordinates determined for the image point in the multi-dimensional eigenspace to the equation that describes the geometric model in the multi-dimensional eigenspace to determine a distance between the image point and the geometric model in the multi-dimensional eigenspace; and

determine whether the object in the image includes a finger based on the determined distance between the image point and the geometric model in the multi-dimensional eigenspace.

13. The system of claim 12 wherein the camera comprises an IR camera.

14. The system of claim 13 further comprising an IR source.

15. The system of claim 12 further comprising a screen disposed in front of the camera.

16. The system of claim 15 wherein the screen is at least translucent such that light reflected by an object disposed on a side of the screen opposite the camera can pass through the screen and be received by the camera.

17. The system of claim 12 wherein the processing device is configured to determine coordinates for an image point in a multi-dimensional eigenspace that has a dimensionality that is lower than a number of data points in the image of the object.

18. A non-transitory computer-readable storage medium having embodied thereon a computer program, the computer program including instructions that, when executed by a computer, cause the computer to:

determine coordinates for an image point in a multi-dimensional eigenspace, the image point in the multi-dimensional eigenspace characterizing an image of an object, and the multi-dimensional eigenspace having been defined based on a set of training images of one or more fingers;

access an equation that describes a geometric model in the multi-dimensional eigenspace, the geometric model being characteristic of the set of training images of the one or more other fingers, and the equation that describes the geometric model including an equation that describes a cone in the multi-dimensional eigenspace;

apply the coordinates determined for the image point in the multi-dimensional eigenspace to the equation that describes the geometric model in the multi-dimensional eigenspace to determine a distance between the image point and the geometric model in the multi-dimensional eigenspace; and

determine whether the object in the image includes a finger based on the determined distance between the image point and the geometric model in the multi-dimensional eigenspace.

19. The non-transitory computer-readable storage medium of claim 18 wherein the computer program further includes instructions that, when executed by a computer, cause the computer to:

receive the set of training images of the one or more fingers, each of the images in the set having a number of data elements;

define, based on the set of training images, the multi-dimensional eigenspace such that the multi-dimensional eigenspace has a dimensionality that is lower than the number of data elements;

generate a set of training points by projecting each image in the set of training images into the multi-dimensional eigenspace to produce coordinates for a corresponding training point in the multi-dimensional eigenspace, each training point in the set of training points corresponding to at least one of the images in the set of training images; and

determine the equation that describes the geometric model in the multi-dimensional eigenspace that is characteristic of the set of training images based on the generated set of training points in the multi-dimensional eigenspace.

20. The non-transitory computer-readable storage medium of claim 19 wherein the computer program further includes instructions that, when executed by a computer, cause the computer to normalize each image in the set of training images before projecting the image into the multi-dimensional eigenspace.

21. The non-transitory computer-readable storage medium of claim 20 wherein the instructions that, when executed by a computer, cause the computer to normalize each image in the set of training images include instructions that, when executed by a computer, cause the computer to normalize each image in the set of training images to account for variations in brightness.

22. The non-transitory computer-readable storage medium of claim 21 wherein the instructions that, when executed by a computer, cause the computer to normalize each image in the set of training images comprise instructions that, when executed by a computer, cause the computer to apply a histogram equalization technique to each image in the set of training images.

23. A method comprising:

using one or more processing devices to determine coordinates for an image point in a multi-dimensional eigenspace, the image point in the multi-dimensional eigenspace characterizing an image of an object, and the multi-dimensional eigenspace having been defined based on a set of training images of one or more fingers;

using the one or more processing devices to access an equation that describes a geometric model in the multi-dimensional eigenspace, the geometric model being characteristic of the set of training images of the one or more fingers, and the equation that describes the geometric model including an equation that describes at least a portion of a hyperboloid in the multi-dimensional eigenspace;

using the one or more processing devices to apply the coordinates determined for the image point in the multi-dimensional eigenspace to the equation that describes the geometric model in the multi-dimensional eigenspace to determine a distance between the image point and the geometric model in the multi-dimensional eigenspace; and

using the one or more processing devices to determine whether the object in the image includes a finger based on the determined distance between the image point and the geometric model in the multi-dimensional eigenspace.

24. The method of claim 23 wherein:

the image of the object has a number of data points; and

determining coordinates for an image point in a multi-dimensional eigenspace includes determining coordinates for an image point in a multi-dimensional eigenspace having a dimensionality that is lower than the number of data points.