Creating a model tree using group tokens for identifying objects in an image

• US 7,680,748 B2
• Filed: 02/02/2006
• Issued: 03/16/2010
• Est. Priority Date: 02/02/2006
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Sharon, Yoav, Heisele, Bernd
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Kennedy; Adrian L
• Application Number:
  US11/347,422
• Publication Number:
  US 20070179918A1
• Time in Patent Office: 1,503 Days
• Field of Search: 706/13, 706/20, 706/12, 382/229, 382/187, 382/181
• US Class Current: 706/13
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram

Abstract

Object recognition techniques are disclosed that provide both accuracy and speed. One embodiment of the present invention is an identification system. The system is capable of locating objects in images by searching for local features of an object. The system can operate in real-time. The system is trained from a set of images of an object or objects. The system computes interest points in the training images, and then extracts local image features (tokens) around these interest points. The set of tokens from the training images is then used to build a hierarchical model structure. During identification/detection, the system, computes interest points from incoming target images. The system matches tokens around these interest points with the tokens in the hierarchical model. Each successfully matched image token votes for an object hypothesis at a certain scale, location, and orientation in the target image. Object hypotheses that receive insufficient votes are rejected.

Claims

1. A computer-implemented method for identifying objects in images, wherein the method is performed by a processor, comprising:

computing one or more interest points in each of a plurality of training images including one or more objects, wherein each interest point represents one pixel, and wherein the one or more interest points in a training image represent a subset of the pixels of the training image;

extracting tokens associated with the interest points, wherein a token associated with an interest point comprises an image feature of an image region surrounding the interest point;

comparing tokens associated with an interest point in a first training image with tokens associated with an interest point in a second training image to find matched tokens, wherein a matched token comprises a first token in the first training image and a second token in the second training image, and wherein the first token is related to the second token;

grouping the matched tokens into sets, wherein a set comprises related matched tokens;

computing a group token to represent each set of matched tokens; and

creating a model tree using the group tokens, where each node of the tree represents an object model for identifying objects in images.

2. The method of claim 1 wherein the method further includes the preliminary step of receiving the plurality of training images including one or more objects, and formatting those images so that they can be processed according to the method.

3. The method of claim 1 further comprising:

identifying objects in a target image using the model tree.

4. The method of claim 3 wherein identifying objects in the target image using the model tree comprises:

computing one or more interest points in the target image;

extracting tokens associated with the target image interest points; and

comparing tokens of the target image with tokens in the model tree to identify matches.

5. The method of claim 4 further comprising:

in response to determining that a token match threshold is satisfied, accepting an object hypothesis; and

in response to determining that a token match threshold is not satisfied, rejecting an object hypothesis.

6. The method of claim 5 further comprising:

repeating the computing, extracting, comparing, and determining for a plurality of target images.

7. The method of claim 3 wherein the method further includes the preliminary step of receiving the target image, and formatting the target image so that it can be processed according to the method.

8. A machine-readable medium encoded with instructions, that when executed by a processor, cause the processor to carry out a process for identifying objects in images, comprising:

computing one or more interest points in each of a plurality of training images including one or more objects, wherein each interest point represents one pixel, and wherein the one or more interest points in a training image represent a subset of the pixels of the training image;

extracting tokens associated with the interest points, wherein a token associated with an interest point comprises an image feature of an image region surrounding the interest point;

comparing tokens associated with an interest point in a first training image with tokens associated with an interest point in a second training image to find matched tokens, wherein a matched token comprises a first token in the first training image and a second token in the second training image, and wherein the first token is related to the second token;

grouping the matched tokens into sets, wherein a set comprises related matched tokens;

computing a group token to represent each set of matched tokens; and

creating a model tree using the group tokens, where each node of the tree represents an object model for identifying objects in images.

9. The machine-readable medium of claim 8 wherein the process further includes receiving the plurality of training images including one or more objects, and formatting those images so that they can be processed according to the process.

10. The machine-readable medium of claim 8, the process further comprising:

identifying objects in a target image using the model tree.

11. The machine-readable medium of claim 10 wherein identifying objects in the target image using the model tree comprises:

computing one or more interest points in the target image;

extracting tokens associated with the target image interest points; and

comparing tokens of the target image with tokens in the model tree to identify matches.

12. The machine-readable medium of claim 11, the process further comprising:

in response to determining that a token match threshold is satisfied, accepting an object hypothesis; and

in response to determining that a token match threshold is not satisfied, rejecting an object hypothesis.

13. The machine-readable medium of claim 12, the process further comprising:

repeating the computing, extracting, comparing, and determining for a plurality of target images.

14. The machine-readable medium of claim 10 wherein the process further includes the preliminary step of receiving the target image, and formatting the target image so that it can be processed according to the process.

15. A hardware system for identifying objects in images, comprising:

an interest point locator module for computing one or more interest points in each of a plurality of training images including one or more objects, wherein each interest point represents one pixel, and wherein the one or more interest points in a training image represent a subset of the pixels of the training image;

a token extraction module for extracting tokens associated with the interest points, wherein a token associated with an interest point comprises an image feature of an image region surrounding the interest point;

a token grouping module for comparing tokens associated with an interest point in a first training image with tokens associated with an interest point in a second training image to find matched tokens, wherein a matched token comprises a first token in the first training image and a second token in the second training image, and wherein the first token is related to the second token, for grouping the matched tokens into sets, wherein a set comprises related matched tokens, and for computing a group token to represent each set of matched tokens; and

a model tree creator module for creating a model tree using the group tokens, where each node of the tree represents an object model for identifying objects in images.

16. The system of claim 15 further comprising:

a run-time recognition module for identifying objects in a target image using the model tree.

17. The system of claim 16 wherein the run-time recognition module further comprises:

an interest point locator module for computing one or more interest points in the target image;

a token extraction module for extracting tokens associated with the target image interest points; and

a token matching module for comparing tokens of the target image with tokens in the model tree to identify matches.

18. The system of claim 17 further comprising:

a hypothesis verification module for determining if a token match threshold is satisfied, and accepting an object hypothesis or rejecting an object hypothesis based on that determination.

19. A hardware system for identifying objects in images, comprising:

a means for computing one or more interest points in each of a plurality of training images including one or more objects, wherein each interest point represents one pixel, and wherein the one or more interest points in a training image represent a subset of the pixels of the training image;

a means for extracting tokens associated with the interest points, wherein a token associated with an interest point comprises an image feature of an image region surrounding the interest point;

a means for comparing tokens associated with an interest point in a first training image with tokens associate with an interest point in a second training image to find matched tokens, wherein a matched token comprises a first token in the first training image and a second token in the second training image, and wherein the first token is related to the second token, for grouping the matched tokens into sets, wherein a set comprises related matched tokens, and for computing a group token to represent each set of matched tokens; and

a means for creating a model tree using the group tokens, where each node of the tree represents an object model for identifying objects in images.

20. The system of claim 19 further comprising:

a means for computing one or more interest points in a target image;

a means for extracting tokens associated with the target image interest points; and

a means for comparing tokens of the target image with tokens in the model tree to identify matches; and

a means for determining if a token match threshold is satisfied, and accepting an object hypothesis or rejecting an object hypothesis based on that determination.