METHOD AND APPARATUS FOR EXTRACTING AND PROCESSING PRICE TEXT FROM AN IMAGE SET

• US 20180315065A1
• Filed: 05/01/2017
• Published: 11/01/2018
• Est. Priority Date: 05/01/2017
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Inventors: Zhang, Yan, Beach, Robert E., Fu, Bo, Cheng, Yan-Ming, Varley, Jordan, Segall, Iaacov Coby
• Granted Patent: US 11,367,092 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 30/0206 Price or cost determination based on market factors
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

A method of extracting price text from an image set includes: obtaining input data comprising (i) a plurality of images depicting shelves supporting products, and (ii) for each of the images, a set of text regions and corresponding price text strings; registering the images to a common frame of reference; identifying a subset of the text regions having overlapping locations in the common frame of reference; selecting one of the text regions from the subset; and presenting the price text string corresponding to the one of the text regions for further processing.

Claims

1. A method of extracting price text from an image set, comprising:

obtaining, by an image set processor, input data comprising (i) a plurality of images depicting shelves supporting products, and (ii) for each of the images, a set of text regions and corresponding price text strings;

registering, by the image set processor, the images to a common frame of reference;

identifying, by a selector coupled with the image set processor, a subset of the text regions having overlapping locations in the common frame of reference;

selecting, by the selector, one of the text regions from the subset; and

presenting, by the selector, the price text string corresponding to the one of the text regions for comparison to a reference price.

2. The method of claim 1, wherein identifying the subset of text regions comprises identifying text regions having at least a predefined degree of overlap in the common frame of reference.

3. The method of claim 1, wherein the input data further comprises a confidence value corresponding to each of the price text strings.

4. The method of claim 3, wherein selecting one of the text regions comprises selecting the one of the subset of text regions having the highest confidence value.

5. The method of claim 1, further comprising obtaining a shelf edge location in the common frame of reference;

wherein the identifying includes discarding any text regions that are not coincident with the shelf edge location.

6. The method of claim 5, further comprising:

obtaining a plurality of depth measurements corresponding to the plurality of images; and

responsive to obtaining the shelf edge location, performing a validation of the shelf edge location based on the depth measurements.

7. The method of claim 6, where performing the shelf edge validation comprises:

selecting a subset of the depth measurements falling within the shelf edge location obtained in the common frame of reference;

clustering the subset of the depth measurements into a plurality of clusters;

selecting a primary one of the clusters;

fitting a plane to the primary cluster; and

determining whether the plane has a preconfigured orientation.

8. The method of claim 1, further comprising:

obtaining a reference price;

comparing the price text string corresponding to the one of the subset of text regions to the reference price; and

presenting a result of the comparison.

9. The method of claim 8, wherein obtaining the reference price comprises:

extracting an area of at least one of the images at a location corresponding to the subset of text regions;

providing the area to a barcode decoder;

receiving decoded data from the barcode decoder, the decoded data identifying a product;

generating a price query containing the decoded data; and

receiving, responsive to the price query, the reference price.

10. The method of claim 8, wherein obtaining the reference price comprises:

providing the image to a product recognition engine;

receiving, from the product recognition engine, a location indicator within the image and a product identifier associated with the location;

generating a price query containing the product identifier; and

receiving, responsive to the price query, a reference price.

11. A server for extracting price text from an image set, comprising:

an image set processor configured to obtain, from a memory, input data comprising (i) a plurality of images depicting shelves supporting products, and (ii) for each of the images, a set of text regions and corresponding price text strings;

the image set processor further configured to register the images to a common frame of reference;

a selector coupled to the image set processor and configured to;

identify a subset of the text regions having overlapping locations in the common frame of reference;

select one of the text regions from the subset; and

present the price text string corresponding to the one of the text regions for comparison to a reference price.

12. The server of claim 11, the selector configured to identify the subset of text regions by identifying text regions having at least a predefined degree of overlap in the common frame of reference.

13. The server of claim 11, wherein the input data further comprises a confidence value corresponding to each of the price text strings.

14. The server of claim 13, the selector configured to select one of the text regions by selecting the one of the subset of text regions having the highest confidence value.

15. The server of claim 11, the selector further configured to:

obtain a shelf edge location in the common frame of reference; and

identify the subset of the text regions by discarding any text regions that are not coincident with the shelf edge location.

16. The server of claim 15, the selector further configured to:

obtain a plurality of depth measurements corresponding to the plurality of images; and

responsive to obtaining the shelf edge location, perform a validation of the shelf edge location based on the depth measurements.

17. The server of claim 16, the selector configured to perform the shelf edge validation by:

selecting a subset of the depth measurements falling within the shelf edge location obtained in the common frame of reference;

clustering the subset of the depth measurements into a plurality of clusters;

selecting a primary one of the clusters;

fitting a plane to the primary cluster; and

determining whether the plane has a preconfigured orientation.

18. The server of claim 11, further comprising a comparator configured to:

obtain a reference price;

compare the price text string corresponding to the one of the subset of text regions to the reference price; and

present a result of the comparison

19. The server of claim 18, further comprising an extractor configured to:

extract an area of at least one of the images at a location corresponding to the subset of text regions;

provide the area to a barcode decoder;

receive decoded data from the barcode decoder, the decoded data identifying a product;

wherein the comparator is configured to obtain the reference price by generating a price query containing the decoded data, and receive, responsive to the price query, the reference price.

20. The server of claim 18, the comparator configured to obtain the reference price by:

providing the image to a product recognition engine;

receiving, from the product recognition engine, a location indicator within the image and a product identifier associated with the location;

generating a price query containing the decoded data; and

receiving, responsive to the price query, a reference price.