Feature interpolation

• US 9,734,434 B2
• Filed: 06/15/2016
• Issued: 08/15/2017
• Est. Priority Date: 07/18/2014
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Brandt, Jonathan W., Shen, Xiaohui, Lin, Zhe
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US15/183,629
• Publication Number:
  US 20160292537A1
• Time in Patent Office: 426 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations

Abstract

Feature interpolation techniques are described. In a training stage, features are extracted from a collection of training images and quantized into visual words. Spatial configurations of the visual words in the training images are determined and stored in a spatial configuration database. In an object detection stage, a portion of features of an image are extracted from the image and quantized into visual words. Then, a remaining portion of the features of the image are interpolated using the visual words and the spatial configurations of visual words stored in the spatial configuration database.

Claims

1. A computer-implemented method comprising:

receiving a collection of training images;

extracting features from the collection of training images;

quantizing the extracted features into visual words;

determining spatial configurations of the visual words in the training images;

storing the spatial configurations of the visual words in a spatial configuration database; and

extracting a portion of features of an image and utilizing the spatial configurations of the visual words stored in the spatial configuration database to interpolate a remaining portion of features of the image.

2. The computer-implemented method of claim 1, wherein the determining the spatial configurations of the visual words comprises, for each visual word, determining neighboring visual words.

3. The computer-implemented method of claim 2, wherein the storing the spatial configurations of the visual words comprises, for each visual word, storing the neighboring visual words in neighbor lists that are associated with the visual word in the spatial configuration database.

4. The computer-implemented method of claim 3, wherein each neighbor list includes an occurrence field that indicates a number of occurrences of the neighboring visual word in the training images.

5. The computer-implemented method of claim 4, wherein the storing the neighboring visual words in the neighbor lists further comprises, for each neighboring visual word:

determining if the neighboring visual word is listed in a corresponding neighbor list of the visual word;

if the neighboring visual word is listed in the corresponding neighbor list of the visual word, increasing a corresponding occurrence value by 1;

if the neighboring visual word is not listed in the corresponding neighbor list of the visual word, creating a new entry for the neighboring visual word in the corresponding neighbor list and setting a corresponding occurrence value to 1.

6. The computer-implemented method of claim 3, wherein:

the determining neighboring visual words, comprises determining N neighboring visual words for each visual word; and

the storing the neighboring visual words comprises, for each visual word, storing N lists of neighboring visual words, wherein each of the N lists comprises a list of neighboring visual words at a certain position relative to the visual word in the training images.

7. The computer-implemented method of claim 3, wherein:

the determining neighboring visual words, comprises, for each visual word, determining left neighboring visual words, top neighboring visual words, right neighboring visuals words, and bottom neighboring visual words; and

the storing the neighboring visual words comprises, storing the left neighboring visual words in a left neighbor list, storing the top neighboring visual words in a top neighbor list, storing the right neighboring visual words in a right neighbor list, and storing the top neighboring visual words in a bottom neighbor list.

8. The computer-implemented method of claim 3, wherein the training images include positive images that include a target object.

9. A system comprising:

an extraction module configured to receive a collection of training images, and extract features from the collection of training images;

a quantization module configured to quantize the extracted features into visual words;

a training module configured to determine spatial configurations of the visual words in the training images;

a spatial configuration database configured to store the spatial configurations of the visual words; and

an interpolation module configured to extract a portion of features of an image and utilize the spatial configurations of the visual words stored in the spatial configuration database to interpolate a remaining portion of features of the image.

10. The system of claim 9, wherein the training module is configured to determine the spatial configurations of the visual words by determining, for each visual word, neighboring visual words.

11. The system of claim 10, wherein the spatial configuration database is configured to store, for each visual word, the neighboring visual words in neighbor lists that are associated with the visual word.

12. The system of claim 11, wherein each neighbor list in the spatial configuration database includes an occurrence field that indicates a number of occurrences of the neighboring visual word in the training images.

13. The system of claim 12, wherein the training module is further configured to:

determine if the neighboring visual word is listed in a corresponding neighbor list of the visual word;

if the neighboring visual word is listed in the corresponding neighbor list of the visual word, increasing a corresponding occurrence value by 1;

if the neighboring visual word is not listed in the corresponding neighbor list of the visual word, creating a new entry for the neighboring visual word in the corresponding neighbor list and setting a corresponding occurrence value to 1.

14. The system of claim 11, wherein:

the training module is configured to determine neighboring visual words by determining N neighboring visual words for each visual word; and

the spatial configuration database is configured to store N lists of neighboring visual words, wherein each of the N lists comprises a list of neighboring visual words at a certain position relative to the visual word in the training images.

15. The system of claim 11, wherein:

the training module is configured to determine neighboring visual words by determining left neighboring visual words, top neighboring visual words, right neighboring visuals words, and bottom neighboring visual words; and

the spatial configuration database is configured to store the left neighboring visual words in a left neighbor list, store the top neighboring visual words in a top neighbor list, store the right neighboring visual words in a right neighbor list, and store the top neighboring visual words in a bottom neighbor list.

16. The system of claim 11, wherein the training images include positive images that include a target object.

17. One or more computer-readable storage devices comprising instructions stored thereon that, responsive to execution by one or more processors, perform operations comprising:

receiving a collection of training images;

extracting features from the collection of training images;

quantizing the extracted features into visual words;

determining spatial configurations of the visual words in the training images;

storing the spatial configurations of the visual words in a spatial configuration database; and

extracting a portion of features of an image and utilizing the spatial configurations of the visual words stored in the spatial configuration database to interpolate a remaining portion of features of the image.

18. The one or more computer-readable storage devices of claim 17, wherein the determining the spatial configurations of the visual words comprises, for each visual word, determining neighboring visual words.

19. The one or more computer-readable storage devices of claim 18, wherein the storing the spatial configurations of the visual words comprises, for each visual word, storing the neighboring visual words in neighbor lists that are associated with the visual word in the spatial configuration database.

20. The one or more computer-readable storage devices of claim 19, wherein each neighbor list includes an occurrence field that indicates a number of occurrences of the neighboring visual word in the training images.