CONTEXT-BASED IMAGE RECOGNITION FOR CONSUMER MARKET RESEARCH

• US 20160125265A1
• Filed: 10/31/2014
• Published: 05/05/2016
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Xie, Wei, Deng, Kevin Keqiang, Terrazas, Alex, Bivins, Michael Allen, Jones, Steve, Schiller, Brian, Sullivan, Jonathan, Sheppard, Michael
• Granted Patent: US 9,569,692 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/36 Indoor scenes
• CPC Class Codes: G06V 2201/09 Recognition of logos
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Example methods, apparatus, systems to perform context-based image recognition for consumer market research are disclosed. Disclosed example methods include comparing a first subset of feature points in a first region of interest of a reference image with a first subset of feature points of a first image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first image to determine a first match value. Disclosed example methods also include, in response to determining the first match value satisfies a first threshold, comparing a second subset of feature points in a second region of interest of the reference image with a second subset of the feature points of the first image residing within a second region of the first image corresponding spatially to the second region of interest to determine a second match value.

Claims

1. A method to determine whether a first image depicts a product, the method comprising:

comparing, with a processor, a first subset of feature points in a first region of interest of a reference image with a first subset of feature points of the first image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first image to determine a first match value;

in response to determining the first match value satisfies a first threshold, comparing, with the processor, a second subset of feature points in a second region of interest of the reference image with a second subset of the feature points of the first image residing within a second region of the first image corresponding spatially to the second region of interest to determine a second match value; and

in response to determining the second match value satisfies a second threshold lower than the first threshold, associating the first image with product information associated with the reference image.

2. A method as defined in claim 1, wherein the second region of the first image is located at a second location in the first image, the second location being selected relative to the first location based on a spatial relationship between the first region of interest and the second region of interest in the reference image.

3. A method as defined in claim 2, further comprising, in response to determining the first match value does not satisfy the first threshold:

selecting a third subset of feature points of the first image within a third region of the first image corresponding spatially to the first region of interest and located at a third location in the first image to determine a third match value;

in response to determining the third match value satisfies a first threshold, comparing, with the processor, the second subset of feature points in the second region of interest of the reference image with a fourth subset of feature points of the first image residing within a fourth region corresponding spatially to the second region of interest and located at a fourth location in the first image to determine a fourth match value, the fourth location being selected relative to the third location based on the spatial relationship between the first region of interest and the second region of interest in the reference image.

4. A method as defined in claim 1, wherein comparing the first subset of feature points in the first region of interest of the reference image with the first subset of feature points of the first image comprises:

calculating distances between the first subset of feature points in the first region of interest of the reference image and the first subset of feature points of the first image to determine a percentage of the first subset of feature points in the first region of interest of the reference image that match corresponding feature points from the first subset of feature points of the first image; and

calculating the first match value based on the percentage.

5. A method as defined in claim 1, wherein the reference image is associated with a plurality of regions of interest having respective assigned significance values, and further comprising:

selecting the first region of interest having a highest significance value out of the plurality of regions of interest associated with the reference image; and

setting the first threshold to correspond to a first significance value associated with the first region of interest.

6. A method as defined in claim 1, wherein the reference image is a first reference image associated with the product, the product is associated with a plurality of different reference images, and further comprising:

in response to determining the second match value satisfies the second threshold;

incrementing a first index value associated with the first reference image;

decrementing index values associated with respective ones of the plurality of reference images other than the first reference image; and

discarding ones of the plurality of reference images associated with index values that do not satisfy a third threshold.

7. A method as defined in claim 1, further comprising improving accuracy of an image matcher by utilizing different first and second thresholds adjusted to a number of false positive matches between the reference images and captured images depicting products to be identified.

8. An apparatus to determine whether a first image depicts a product, the apparatus comprising:

a feature point extractor to determine feature points in the first image; and

a reference comparator to;

compare a first subset of feature points in a first region of interest of a reference image with a first subset of the feature points of the first image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first image to determine a first match value;

in response to determining the first match value satisfies a first threshold, compare a second subset of feature points in a second region of interest of the reference image with a second subset of the feature points of the first image residing within a second region of the first image corresponding spatially to the second region of interest to determine a second match value; and

in response to determining the second match value satisfies a second threshold lower than the first threshold, associate the first image with product information associated with the reference image.

9. An apparatus as defined in claim 8, wherein the second region of the first image is located at a second location in the first image, the second location being selected relative to the first location based on a spatial relationship between the first region of interest and the second region of interest in the reference image.

10. An apparatus as defined in claim 8, wherein to compare the first subset of feature points in the first region of interest of the reference image with the first subset of feature points of the first image, the reference comparator is further to:

calculate distances between the first subset of feature points in the first region of interest of the reference image and the first subset of feature points of the first image to determine a percentage of the first subset of feature points in the first region of interest of the reference image that match corresponding feature points from the first subset of feature points of the first image; and

calculate the first match value based on the percentage.

11. An apparatus as defined in claim 8, wherein the reference image is associated with a plurality of regions of interest having respective assigned significance values, and the reference comparator is further to:

select the first region of interest having a highest significance value out of a plurality of regions of interest associated with the reference image; and

set the first threshold to correspond to a first significance value associated with the first region of interest.

12. An apparatus as defined in claim 8, wherein the reference image is a first reference image associated with the product, the product is associated with a plurality of different reference images, and further comprising:

a reference image manager to, in response to determining the second match value satisfies the second threshold;

increment a first index value associated with the first reference image;

decrement index values associated with respective ones of the plurality of reference images other than the first reference image; and

discard ones of the plurality of reference images associated with index values that do not satisfy a third threshold.

13. An apparatus as defined in claim 8, wherein, in response to determining the first match value does not satisfy the first threshold, the reference comparator to is further to:

select a third subset of the feature points of the first image within a third region of the first image corresponding spatially to the first region of interest and located at a third location in the first image to determine a third match value;

in response to determining the third match value satisfies a first threshold, compare the second subset of feature points in the second region of interest of the reference image with a fourth subset of the feature points of the first image residing within a fourth region corresponding spatially to the second region of interest and located at a fourth location in the first image to determine a fourth match value, the fourth location being selected relative to the third location based on the spatial relationship between the first region of interest and the second region of interest in the reference image.

14. An apparatus as defined in claim 8, wherein the reference comparator is to improve accuracy of an image matcher by utilizing different first and second thresholds adjusted to a number of false positive matches between the reference images and captured images depicting products to be identified.

15. A tangible computer readable storage medium comprising instructions which, when executed, cause a machine to at least:

compare a first subset of feature points in a first region of interest of a reference image with a first subset of the feature points of the first image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first image to determine a first match value;

in response to determining the first match value satisfies a first threshold, compare a second subset of feature points in a second region of interest of the reference image with a second subset of the feature points of the first image residing within a second region of the first image corresponding spatially to the second region of interest to determine a second match value; and

in response to determining the second match value satisfies a second threshold lower than the first threshold, associate the first image with product information associated with the reference image.

16. A tangible computer readable medium as defined in claim 15, wherein the second region of the first image is located at a second location in the first image, the second location being selected relative to the first location based on a spatial relationship between the first region of interest and the second region of interest in the reference image.

17. A tangible computer readable medium as defined in claim 15, wherein to compare the first subset of feature points in the first region of interest of the reference image with the first subset of feature points of the first image, the instructions, when executed, further cause the machine to:

calculate distances between the first subset of feature points in the first region of interest of the reference image and the first subset of feature points of the first image to determine a percentage of the first subset of feature points in the first region of interest of the reference image that match corresponding feature points from the first subset of feature points of the first image; and

calculate the first match value based on the percentage.

18. A tangible computer readable medium as defined in claim 15, wherein the reference image is associated with a plurality of regions of interest having respective assigned significance values, and the instructions, when executed, further cause the machine to:

select the first region of interest having a highest significance value out of a plurality of regions of interest associated with the reference image; and

set the first threshold to correspond to a first significance value associated with the first region of interest.

19. A tangible computer readable medium as defined in claim 15, wherein the reference image is a first reference image associated with the product, the product is associated with a plurality of different reference images, and the instructions, when executed, further cause the machine to:

increment a first index value associated with the first reference image;

decrement index values associated with respective ones of the plurality of reference images other than the first reference image; and

discard ones of the plurality of reference images associated with index values that do not satisfy a third threshold.

20. A tangible computer readable medium as defined in claim 15, wherein, in response to determining the first match value does not satisfy the first threshold, the instructions, when executed, further cause the machine to:

select a third subset of the feature points of the first image within a third region of the first image corresponding spatially to the first region of interest and located at a third location in the first image to determine a third match value;

in response to determining the third match value satisfies a first threshold, compare the second subset of feature points in the second region of interest of the reference image with a fourth subset of the feature points of the first image residing within a fourth region corresponding spatially to the second region of interest and located at a fourth location in the first image to determine a fourth match value, the fourth location being selected relative to the third location based on the spatial relationship between the first region of interest and the second region of interest in the reference image.

21. A tangible computer readable medium as defined in claim 15, wherein the first region of interest does not overlap the second region of interest.

22. A tangible computer readable storage medium as defined in claim 15, wherein the instructions, when executed, cause the machine further to improve accuracy of an image matcher by utilizing different first and second thresholds adjusted to a number of false positive matches between the reference images and captured images depicting products to be identified.