Reducing scale estimate errors in shelf images

• US 10,282,607 B2
• Filed: 10/28/2016
• Issued: 05/07/2019
• Est. Priority Date: 10/28/2016
• Status: Active Grant

• Current Assignee: Nielsen Consumer LLC (Nielsen Holdings plc)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Garcia, Antonio Hurtado, Martin, Francisco Javier, Morate, Diego Garcia
• Primary Examiner(s): Fujita, Katrina R
• Application Number:
  US15/317,674
• Publication Number:
  US 20180121721A1
• Time in Patent Office: 921 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: G06V 2201/09 Recognition of logos

Abstract

Example image processing methods, apparatus/systems and articles of manufacture are disclosed herein. An example apparatus includes an image recognition application to identify matches between stored patterns and objects detected in a shelf image, where the shelf image has a shelf image scale estimate. The example apparatus further includes a scale corrector to calculate deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects and reduce an error of the shelf image scale estimate by calculating a scale correction value for the shelf image scale estimate based on the deviation values.

Claims

1. An apparatus to reduce scale estimate errors, the apparatus comprising:

an image recognition application to identify matches between stored patterns and objects detected in a shelf image, the shelf image having a shelf image scale estimate; and

a scale corrector to;

calculate deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects; and

reduce an error of the shelf image scale estimate by calculating a scale correction value for the shelf image scale estimate based on the deviation values.

2. The apparatus as defined in claim 1, wherein the scale corrector is to calculate the deviation values by determining a difference in pixel dimensions of a first scaled version of a first pattern of the first set of the stored patterns and a first object of the first set of the objects matched with the first pattern.

3. The apparatus as defined in claim 2, further including a rescaler to generate the first scaled version of the first pattern by rescaling the first pattern based on the shelf image scale estimate.

4. The apparatus as defined in claim 1, wherein the first set of the stored patterns have optimized scales.

5. The apparatus as defined in claim 4, further including a pattern scale optimizer to determine the optimized scales for the first set of the stored patterns.

6. The apparatus as defined in claim 4, wherein the optimized scales are a median of estimated scales for the first set of the stored patterns.

7. The apparatus as defined in claim 1, wherein the stored patterns include a second set of the stored patterns, the first set of the stored patterns having a number of scale estimates that satisfy a threshold, the second set of the stored patterns having a number of scale estimates that do not satisfy the threshold.

8. The apparatus as defined in claim 1, wherein the stored patterns are image representations of at least a portion of a product or packaging of a product, respective ones of the first set of the stored patterns having stored scale estimates associated with the respective stored pattern.

9. The apparatus as defined in claim 8, wherein each of the first set of the stored patterns has a threshold number of stored scale estimates associated with the respective stored pattern.

10. The apparatus as defined in claim 1, wherein the scale corrector is to calculate additional scale estimates for respective ones of the first set of the stored patterns matched with the first set of the objects based on the sizes of the first set of the objects detected in the shelf image and the scale correction value for the shelf image scale estimate.

11. A method to reduce scale estimate errors, the method comprising:

identifying, by executing an instruction with a processor, matches between stored patterns and objects detected in a shelf image, the shelf image having an shelf image scale estimate;

calculating, by executing an instruction with the processor, deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects; and

reducing an error of the shelf image scale estimate by calculating, by executing an instruction with the processor, a scale correction value for the shelf image scale estimate based on the deviation values.

12. The method as defined in claim 11, wherein calculating the deviation values includes determining a difference in pixel dimensions of a first scaled version of a first pattern of the first set of the stored patterns and a first object of the first set of the objects matched with the first pattern.

13. The method as defined in claim 12, further including generating the first scaled version of the first pattern by rescaling the first pattern based on the shelf image scale estimate.

14. The method as defined in claim 11, wherein the first set of the stored patterns have optimized scales.

15. The method as defined in claim 14, further including determining the optimized scales for the first set of the stored patterns.

16. The method as defined in claim 14, wherein the optimized scales are a median of estimated scales for the first set of the stored patterns.

17. The method as defined in claim 11, wherein the stored patterns include a second set of the stored patterns, the first set of the stored patterns having a number of scale estimates that satisfy a threshold, the second set of the stored patterns having a number of scale estimates that do not satisfy the threshold.

18. A tangible machine readable storage medium comprising instructions that, when executed, cause a machine to at least:

identify matches between stored patterns and objects detected in a shelf image, the shelf image having an shelf image scale estimate;

calculate deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects; and

reduce an error of the shelf image scale estimate by calculating a scale correction value for the shelf image scale estimate based on the deviation values.

19. The tangible machine readable storage medium as defined in claim 18, wherein the instructions, when executed, cause the machine to calculate the deviation values by determining a difference in pixel dimensions of a first scaled version of a first pattern of the first set of the stored patterns and a first object of the first set of the objects matched with the first pattern.

20. The tangible machine readable storage medium as defined in claim 19, wherein the instructions, when executed, further cause the machine to generate the first scaled version of the first pattern by rescaling the first pattern based on the shelf image scale estimate.

21. The tangible machine readable storage medium as defined in claim 18, wherein the first set of the stored patterns have optimized scales.

22. The tangible machine readable storage medium as defined in claim 21, wherein the instructions, when executed, further cause the machine to determine the optimized scales for the first set of the stored patterns.

23. The tangible machine readable storage medium as defined in claim 18, wherein the stored patterns include a second set of the stored patterns, the first set of the stored patterns having a number of scale estimates that satisfy a threshold, the second set of the stored patterns having a number of scale estimates that do not satisfy the threshold.