Context-based image recognition for consumer market research

• US 9,710,723 B2
• Filed: 01/27/2017
• Issued: 07/18/2017
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Sullivan, Jonathan, Sheppard, Michael, Xie, Wei, Bivins, Michael Allen, Jones, Steve, Deng, Kevin Keqiang, Schiller, Brian, Terrazas, Alejandro
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US15/417,864
• Publication Number:
  US 20170132492A1
• Time in Patent Office: 172 Days
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/36 Indoor scenes
• CPC Class Codes: G06V 2201/09 Recognition of logos
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Example methods, apparatus, systems to perform context-based image recognition are disclosed. Disclosed example methods include comparing a first and second reference region of a reference image with a first and second region of a first captured image to determine a first and second match value. In response to determining the first and second match values satisfy a first and second threshold: associating the first captured image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database. Decrementing index values associated with ones of the second reference images. Removing a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing the storage space required to store reference records in the database.

Claims

1. An apparatus, comprising:

a reference comparator to;

compare a first reference region of a first reference image stored in a database with a first region of a first image to determine a first match value, the first region corresponding spatially to the first reference region;

in response to determining the first match value satisfies a first threshold, compare a second reference region of the first reference image with a second region of the first image to determine a second match value, the second region corresponding spatially to the second reference region; and

in response to determining the second match value satisfies a second threshold, associate the first image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database; and

a reference image manager to, in response to determining the second match value satisfies the second threshold;

decrement second index values associated with ones of the second reference images; and

remove a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing a storage space required to store reference records in the database, at least one of the reference comparator or the reference image manager including hardware.

2. The apparatus as defined in claim 1, wherein the first reference region has a first significance value and the second reference region has a second significance value and the first significance value is higher than the second significance value.

3. The apparatus as defined in claim 1, wherein the first condition is satisfied when the one of the second index values does not satisfy a third threshold.

4. The apparatus as defined in claim 1, further including:

in response to determining the second match value satisfies the second threshold;

the reference comparator to designate the first reference image as a match to the first image; and

the reference image manager to select the second index values to decrement based on corresponding ones of the second reference images not being designated as a match to the first image.

5. The apparatus as defined in claim 4, wherein the reference comparator is further to:

before the reference image manager decrements one of the second index values, for a corresponding one of the second reference images;

compare a third reference region of the corresponding second reference image with the first region of the first image to determine a third match value, the first region corresponding spatially to the third reference region;

in response to determining the third match value satisfies a fourth threshold, compare a fourth reference region of the corresponding second reference image with the second region of the first image to determine a fourth match value, the second region corresponding spatially to the fourth reference region; and

in response to determining the fourth match value satisfies a fifth threshold, increment a corresponding one of the index values associated with the corresponding second reference image, and designate the corresponding second reference image as a match to the first image.

6. The apparatus as defined in claim 1, wherein the first condition is satisfied when the one of the second index values is a lowest index value associated with the product.

7. The apparatus as defined in claim 1, wherein a maximum number of reference images stored in the database that are associated with the product is limited to a predetermined number.

8. A tangible computer readable storage medium comprising instructions which, when executed, cause a machine to at least:

compare a first reference region of a first reference image stored in a database with a first region of a first image to determine a first match value, the first region corresponding spatially to the first reference region;

in response to determining the first match value satisfies a first threshold, compare a second reference region of the first reference image with a second region of the first image to determine a second match value, the second region corresponding spatially to the second reference region; and

in response to determining the second match value satisfies a second threshold;

associate the first image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database;

decrement second index values associated with ones of the second reference images; and

remove a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing a storage space required to store reference records in the database.

9. The tangible computer readable medium as defined in claim 8, wherein the first reference region has a first significance value and the second reference region has a second significance value and the first significance value is higher than the second significance value.

10. The tangible computer readable medium as defined in claim 8, wherein the first condition is satisfied when the one of the second index values does not satisfy a third threshold.

11. The tangible computer readable medium as defined in claim 8, wherein the instruction, when executed, further cause the machine to:

in response to determining the second match value satisfies the second threshold;

designate the first reference image as a match to the first image; and

select the second index values to decrement based on corresponding ones of the second reference images not being designated as a match to the first image.

12. The tangible computer readable medium as defined in claim 11, wherein the instruction, when executed, further cause the machine to:

before decrementing one of the second index values, for a corresponding one of the second reference images;

compare a third reference region of the corresponding second reference image with the first region of the first image to determine a third match value, the first region corresponding spatially to the third reference region;

in response to determining the third match value satisfies a fourth threshold, compare a fourth reference region of the corresponding second reference image with the second region of the first image to determine a fourth match value, the second region corresponding spatially to the fourth reference region; and

in response to determining the fourth match value satisfies a fifth threshold, increment a corresponding one of the index values associated with the corresponding second reference image, and designate the corresponding second reference image as a match to the first image.

13. The tangible computer readable medium as defined in claim 8, wherein the first condition is satisfied when the one of the second index values is a lowest index value associated with the product.

14. The tangible computer readable medium as defined in claim 8, wherein a maximum number of reference images stored in the database that are associated with the product is limited to a predetermined number.

15. A method, comprising:

receiving a first image, the first image captured by a mobile device;

comparing, by executing an instruction with a processor, a first reference region of a first reference image stored in a database with a first region of the first image to determine a first match value, the first region corresponding spatially to the first reference region;

in response to determining the first match value satisfies a first threshold, comparing, by executing an instruction with the processor, a second reference region of the first reference image with a second region of the first image to determine a second match value, the second region corresponding spatially to the second reference region; and

in response to determining the second match value satisfies a second threshold;

associating, by executing an instruction with the processor, the first image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database;

decrementing, by executing an instruction with the processor, second index values associated with ones of the second reference images; and

removing, by executing an instruction with the processor, a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing a storage space required to store reference records in the database.

16. The method as defined in claim 15, wherein the first reference region has a first significance value and the second reference region has a second significance value and the first significance value is higher than the second significance value.

17. The method as defined in claim 15, wherein the first condition is satisfied when the one of the second index values does not satisfy a third threshold.

18. The method as defined in claim 15, wherein the first condition is satisfied when the one of the second index values is a lowest index value associated with the product.

19. The method as defined in claim 15, further including:

in response to determining the second match value satisfies the second threshold;

designating the first reference image as a match to the first image; and

selecting the second index values to decrement based on corresponding ones of the second reference images not being designated as a match to the first image.

20. The method as defined in claim 19, further including:

before decrementing one of the second index values, for a corresponding one of the second reference images;

comparing, by executing an instruction with the processor, a third reference region of the corresponding second reference image with the first region of the first image to determine a third match value, the first region corresponding spatially to the third reference region;

in response to determining the third match value satisfies a fourth threshold, comparing, by executing an instruction with the processor, a fourth reference region of the corresponding second reference image with the second region of the first image to determine a fourth match value, the second region corresponding spatially to the fourth reference region; and

in response to determining the fourth match value satisfies a fifth threshold, incrementing, by executing an instruction with the processor, a corresponding one of the index values associated with the corresponding second reference image, and designating the corresponding second reference image as a match to the first image.