Realogram Scene Analysis of Images: Superpixel Scene Analysis

• US 20160171707A1
• Filed: 03/06/2015
• Published: 06/16/2016
• Est. Priority Date: 12/10/2014
• Status: Active Grant

• Current Assignee: Filix Medtech Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Schwartz, Edward
• Granted Patent: US 9,483,704 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/15 Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

The techniques include an image recognition system to receive a realogram image including a plurality of organized objects and to detect and identify objects in the realogram image of one or more items on a retail shelf, identify shelf fronts and labels on the shelf fronts, identify empty space under shelves, identify areas where unidentified products may be, and identify areas where products are “out of stock”.

Claims

1. A computer-implemented method comprising:

receiving an image of a plurality of organized objects;

generating superpixels based on groups of pixels in the image of the plurality of organized objects;

labeling superpixels with information and observations from image analysis on the image of the plurality of organized objects; and

identifying regions of the image of the plurality of organized objects based on the superpixels.

2. The computer implemented method of claim 1, further comprising identifying indexed objects in the image, wherein labeling superpixels with information and observations includes labeling superpixels with an object identifier.

3. The computer implemented method of claim 1, further comprising generating region hypotheses using the information and observations.

4. The computer implemented method of claim 3, further comprising validating the hypotheses.

5. The computer implemented method of claim 4, further comprising classifying regions of the image using the validated hypotheses.

6. The computer implemented method of claim 1, wherein the regions include one or more from the group of:

known objects, empty shelf space, shelf features, and labels.

7. The computer implemented method of claim 1, further comprising:

identifying empty areas under a shelf;

classifying superpixels as empty or other;

iterating a segmentation algorithm on the superpixels to reclassify each of the superpixels as empty or other; and

identifying empty shelf space in the image based on the reclassified superpixels.

8. A system comprising:

one or more processors; and

a memory, the memory storing instructions, which when executed cause the one or more processors to;

receive an image of a plurality of organized objects;

generate superpixels based on groups of similar pixels in the image of the plurality of organized objects;

label superpixels with information and observations from image analysis on the image of the plurality of organized objects; and

identify regions of the image of the plurality of organized objects based on the superpixels.

9. The system of claim 8, wherein the instructions further cause the one or more processors to identify indexed objects in the image and label superpixels with an object identifier.

10. The system of claim 8, wherein the instructions further cause the one or more processors to generate region hypotheses using the information and observations.

11. The system of claim 10, wherein the instructions further cause the one or more processors to validate the hypotheses.

12. The system of claim 11, wherein the instructions further cause the one or more processors to classify regions of the image using the validated hypotheses.

13. The system of claim 8, wherein the regions include one or more from the group of:

known objects, empty shelf space, shelf features, and labels.

14. The system of claim 8, wherein the instructions further cause the one or more processors to:

identify empty areas under a shelf;

classify superpixels as empty or other;

iterate a segmentation algorithm on the superpixels to reclassify each of the superpixels as empty or other; and

identify empty shelf space in the image based on the reclassified superpixels.

15. A computer program product comprising a non-transitory computer usable medium including a computer readable program, wherein the computer readable program, when executed on a computer causes the computer to:

receive an image of a plurality of organized objects;

generate superpixels based on groups of similar pixels in the image of the plurality of organized objects;

label superpixels with information and observations from image analysis on the image of the plurality of organized objects; and

identify regions of the image of the plurality of organized objects based on the superpixels.

16. The computer program product of claim 15, wherein the computer readable program further causes the computer to identify indexed objects in the image and label superpixels with an object identifier.

17. The computer program product of claim 15, wherein the computer readable program further causes the computer to generate region hypotheses using the information and observations.

18. The computer program product of claim 15, wherein the computer readable program further causes the computer to classify regions of the image using the region hypotheses.

19. The computer program product of claim 15, wherein the regions include one or more from the group of:

known objects, empty shelf space, shelf features, and labels.

20. The computer program product of claim 15, wherein the computer readable program further causes the computer to:

identify empty areas under a shelf;

classify superpixels as empty or other;

iterate a segmentation algorithm on the superpixels to reclassify each of the superpixels as empty or other; and

identify empty shelf space in the image based on the reclassified superpixels.