Extending data-driven detection to the prediction of object part locations

• US 9,842,274 B2
• Filed: 03/28/2014
• Issued: 12/12/2017
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Rodriguez-Serrano, Jose-Antonio, Dai, Zhenwen
• Primary Examiner(s): Rush, Eric
• Application Number:
  US14/228,947
• Publication Number:
  US 20150278632A1
• Time in Patent Office: 1,355 Days
• Field of Search: 382100, 382103, 382155, 382159, 382181, 382190, 382195, 382203, 382209, 382216-220, 382224, 382282, 382305, 382160
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Methods and systems for detecting an object part location based on an extended date-driven detection. A specific relevance value between configurations of parts with respect to a set of training images annotated with annotating objects can be defined. A similarity learning algorithm can be applied with respect to the parts to obtain a similarity function based on the similarity between the part configurations. The similarity learning algorithm receives a set of positive pair having similar part configuration and a negative pair having different configuration and returns the similarity function that tends to assign a high score to new positive pairs and a low score to negative pairs. A similarity between a new image and the training images can be measured utilizing the learned similarity function to obtain a neighbor image and a visible and/or non-visible part location with respect to the image can be predicted based on the neighbor image.

Claims

1. A method for detecting an object part location in images captured by at least one camera, said method comprising:

defining with a relevance computing unit, a specific relevance value between a configuration of a plurality of parts and a set of training images annotated via an annotating object;

applying a similarity learning algorithm with respect to said plurality of parts to obtain a learned similarity function based on said specific relevance value, wherein said learned similarity function comprises a real-valued function that quantifies a similarity between at least two objects and wherein said similarity learning algorithm respects a similarity between part configurations, wherein a similarity learning of said similarity learning algorithm comprises a supervised machine learning task, and wherein said learned similarity function assigns a high score to positive pairs and a low score to negative pairs;

measuring a similarity between a new image captured by said at least one camera and said set of training images utilizing said learned similarity function in order to obtain a neighbor image and predict an object part location of a visible object part and a non-visible object part with respect to said new image based on said neighbor image;

capturing with said at least one camera, said set of training images and said new image with respect to said annotating object;

processing said set of training images and said new image utilizing an image processor and wherein said predict said object part location of said visible object part and said non-visible object part with respect to said new irnace based on said neighbor image, further comprises a detection of an object part, wherein said detection is executable as a single query that is independent of a number of object parts by taking into account a global consistency between said object parts;

defining a set of indices corresponding to said visible object part in said training images among said set of training images and thereafter computing said specific relevance value between said training images among said set of training images;

computing a part-by-part overlap score by measuring an overlap between similar parts; and

assigning said specific relevance value to one if said part-by-part overlap score is greater than a threshold otherwise assign zero.

2. The method of claim 1 further comprising:

receiving a set of said positive pairs having similar part configurations and a negative pair having different part configurations by said similarity learning algorithm; and

returning said learned similarity function to assign said high score to said positive pairs and said low score to said negative pair.

3. The method of claim 1 wherein said annotating object with respect to said set of training images and said new image comprises a rectangular shape having at least four coordinates.

4. The method of claim 3 wherein said annotating object comprises an empty value with respect to said object part location of said non-visible object part.

5. The method of claim 1 wherein said threshold depends on constraints with respect to a data driven detection application.

6. The method of claim 1 further comprising assigning said specific relevance value to zero if different parts are visible in said training images among said set of training images.

7. The method of claim 1 further comprising:

outputting said object part location of said visible object part and said non-visible object part with respect to said new image based on said neighbor image for a single neighbor among a plurality of neighbors; and

combining object part locations in a case of said plurality of neighbors.

8. The method of claim 7 further comprising:

computing a weighted average for said visible object part with respect to said plurality of neighbors; and

returning an empty value with respect to said plurality of neighbors having said non-visible object part.

9. A system for detecting an object part location in images captured by at least one camera, said system comprising:

a processor; and

a computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with the processor, said computer program code comprising instructions executable by said processor and configured for;

defining with a relevance computing unit, a specific relevance value between a configuration of a plurality of parts and a set of training images annotated via an annotating object;

applying a similarity learning algorithm with respect to said plurality of parts to obtain a learned similarity function based on said specific relevance value, wherein said learned similarity function comprises a real-valued function that quantifies a similarity between at least two objects and wherein said similarity learning algorithm respects a similarity between part configurations, wherein a similarity learning of said similarity learning algorithm comprises a supervised machine learning task, and wherein said learned similarity function assigns a high score to positive pairs and a low score to negative pairs;

measuring a similarity between a new image captured by said at least one camera and said set of training images utilizing said learned similarity function in order to obtain a neighbor image and predict an object part location of a visible object part and a non-visible object part with respect to said new image based on said neighbor image;

capturing via said at least one camera, said set of training images and said new image with respect to said annotating object;

processing said set of training images and said new image utilizing an image processor and wherein said predict said object part location of said visible object part and said non-visible object part with respect to said new image based on said neighbor image, further comprises a detection of an object part, wherein said detection is executable as a single query that is independent of a number of object parts by taking into account a global consistency between said object parts;

defining a set of indices corresponding to said visible object part in said training images among said set of training images and thereafter computing said specific relevance value between said training images among said set of training images;

computing a part-by-part overlap score by measuring an overlap between similar parts; and

assigning said specific relevance value to one if said part-by-part overlap score is greater than a threshold otherwise assign zero.

10. The system of claim 9 wherein said instructions are further configured for:

receiving a set of said positive pairs having similar part configurations and a negative pair having different part configurations by said similarity learning algorithm; and

returning said learned similarity function to assign said high score to said positive pair and said low score to said negative pair among said negative pairs.

11. The system of claim 10 wherein said annotating object with respect to said training images among said set of training images and said new image comprises a rectangular shape having at least four coordinates.

12. The system of claim 11 wherein said annotating object comprises an empty value with respect to a location of said non-visible object part.

13. A non-transitory processor-readable medium having stored thereon processor executable instructions configured to cause a processor to perform operations for detecting an object part location in images captured by at least one camera, comprising:

defining with a relevance computing unit a specific relevance value between a configuration of a plurality of parts and a set of training images annotated via an annotating object;

applying a similarity learning algorithm with respect to said plurality of parts to obtain a learned similarity function based on said specific relevance value, wherein said learned similarity function comprises a real-valued function that quantifies a similarity between at least two objects and wherein said similarity learning algorithm respects a similarity between part configurations, wherein a similarity learning of said similarity learning algorithm comprises a supervised machine learning task, and wherein said learned similarity function assigns a high score to positive pairs and a low score to negative pairs;

measuring a similarity between a new image captured by said at least one camera and said set of training images utilizing said learned similarity function in order to obtain a neighbor image and predict a location of a visible object part and a non-visible object part with respect to said new image based on said neighbor image;

capture via said at least one camera said set of training images and said new image with respect to said annotating object;

process said set of training images and said new image utilizing an image processor and wherein said predict said object part location of said visible object part and said non-visible object part with respect to said new image based on said neighbor image, further comprises a detection of an object part, wherein said detection is executable as a single query that is independent of a number of object parts by taking into account a global consistency between said object parts;

define a set of indices corresponding to said visible object part in said training images among said set of training images and thereafter computing said specific relevance value between said training images among said set of training images;

compute a part-by-part overlap score by measuring an overlap between similar parts; and

assign said specific relevance value to one if said part-by-part overlap score is greater than a threshold otherwise assign zero.

14. The processor-readable medium of claim 13 wherein said processor executable instructions are configured to cause a processor to perform operations to:

receive a set of said positive pairs having similar part configurations and a negative pair having different part configurations by said similarity learning algorithm; and

return said similarity function to assign said high score to said positive pairs and said low score to said negative pair among said negative pairs.

15. The processor-readable medium of claim 13 wherein said annotating object comprises an empty value with respect to a location of said non-visible object part.