Image processing methods and systems for barcode and/or product label recognition

• US 9,659,204 B2
• Filed: 06/13/2014
• Issued: 05/23/2017
• Est. Priority Date: 06/13/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Wu, Wencheng, Ct, Adrien P., Venable, Dennis L., Moore, Steven R., Wade, Thomas F., Paul, Peter
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US14/303,724
• Publication Number:
  US 20150363625A1
• Time in Patent Office: 1,075 Days
• CPC Class Codes: G06K 7/1443 locating of the code in an image
• CPC Class Codes: G06K 7/1447 extracting optical codes from image or text carrying said optical code
• CPC Class Codes: G06K 7/146 the method including quality enhancement steps
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

This disclosure provides an image processing method and system for recognizing barcodes and/or product labels. According to an exemplary embodiment, the method uses a multifaceted detection process that includes both image enhancement of a candidate barcode region and other product label information associated with a candidate barcode region to identify a product label, where the candidate barcode region includes a nonreadable barcode. According to one exemplary application, a store profile is generated based on the identifications of the product labels which are associated with a location of a product within a store.

Claims

1. A method of identifying a plurality of barcodes associated with a plurality of respective product labels, the product labels including one or more barcodes and other non-barcode related product label information, the method comprising:

an image capturing device capturing an image of a plurality of product labels and storing the captured image in a memory operatively associated with the image capturing device;

a processor operatively associated with the memory detecting and localizing a plurality of barcode candidate regions within the captured image of the plurality of product labels;

the processor cropping the detected and localized plurality of barcode candidate regions to generate one or more respective sub-images of the captured image including images of the one or more barcode candidate regions, the sub-images including a region of the captured image defined by the size of a detected barcode within the respective barcode candidate region and excluding substantially all of the other product label information associated with the respective detected barcode;

the processor processing each sub-image using two or more independent image quality improvement processes to generate modified versions of each of the plurality of barcode candidate regions and respective detected barcodes within the respective barcode candidate regions;

the processor processing the modified versions of each of the plurality of barcode candidate regions using a binary process to decode readable barcodes included in each of the modified versions of each of the candidate barcode regions to generate a numerical representation of the readable barcodes and identifying barcode candidate regions including barcodes nondecodable using the binary process; and

for each of the plurality of barcode candidate regions including a nondecodable barcode,

the processor processing the captured image of the plurality of product labels to determine auxiliary product information regions including a greater number of pixels than the respective barcode candidate region including a nondecodable barcode;

the processor determining a price-tag layout associated with each of the auxiliary product information regions by matching the auxiliary product information regions with one or more reference price-tag templates including the physical layout of the other non-barcode related product information associated with each price-tag template;

the processor extracting the other non-barcode related product label information from the auxiliary product information regions based on a matched price-tag template; and

the processor determining a numerical representation of the nondecodable barcodes associated with each auxiliary product information region based on the extracted other non-barcode related product label information of the respective auxiliary product information region.

2. The method of identifying a barcode according to claim 1, wherein the two or more independent image quality improvement processes include one or more of contrast enhancement, tone-curve reshaping, gamma-correction, sharpening, de-blurring and morphological filtering.

3. The method of identifying a barcode according to claim 1, wherein the two or more independent image quality improvement processes are performed in serial or parallel.

4. The method of identifying a barcode associated with a product label according to claim 1, wherein the barcode is one of a 1D (Dimensional) and 2D barcode.

5. The method of identifying a barcode associated with a product label according to claim 1, further comprising:

associating each of the plurality of product labels including the respective decoded barcode with a physical location in a retail environment.

6. The method of identifying a barcode associated with a product label according to claim 5, wherein the step of detecting and localizing a plurality of barcode-candidate-regions within the captured image is biased to generate relatively more false-positives than false-negatives.

7. The method of identifying a barcode associated with a product label according to claim 1, further comprising:

fusing the decoded readable barcodes and identifying a respective product label based on the fusion.

8. An image processing system for identifying a plurality of barcodes associated with a plurality of respective product labels, the product labels including one or more barcodes and other non-barcode related product label information, the image processing system comprising:

an image capturing device; and

a memory operatively associated with the image capturing device,

a processor operatively associated with one or both of the image capturing device and memory,

the image capturing device configured to capture an image of a plurality of product labels and storing the captured image in the memory operatively associated with the image capturing device;

the processor configured to detect and localize a plurality of barcode candidate regions within the captured image of the plurality of product labels;

the processor configured to crop the detected and localized plurality of barcode candidate regions to generate one or more respective sub-images of the captured image including images of the one or more barcode candidate regions;

the processor configured to process each sub-image using two or more independent image quality improvement processes to generate modified versions of each of the plurality of barcode candidate regions;

the processor configured to process the modified versions of each of the plurality of barcode candidate regions to decode readable barcodes included in the modified versions of each of the candidate barcode regions; and

for each of the plurality of barcode candidate regions including a nondecodable barcode,

the processor configured to process the captured image of the plurality of product labels to determine auxiliary product information regions including a greater number of pixels than the respective barcode candidate region including a nondecodable barcode;

the processor configured to determine a price-tag layout associated with each of the auxiliary product information regions by matching the auxiliary product information regions with one or more reference price-tag templates including the physical layout of the other non-barcode related product information associated with each price-tag template;

the processor configured to extract the other non-barcode related product label information from the auxiliary product information regions based on a matched price-tag template; and

the processor configured to determine a numerical representation of the nondecodable barcodes associated with each auxiliary product information region based on the extracted other non-barcode related product label information of the respective auxiliary product information region.

9. The image processing system according to claim 8, wherein the two or more independent image quality improvement processes include one or more of contrast enhancement, tone-curve reshaping, gamma-correction, sharpening, de-blurring and morphological filtering.

10. The image processing system according to claim 8, wherein the two or more independent image quality improvement processes are performed in serial or parallel.

11. The image processing system according to claim 8, wherein the barcode is one of a 1D (Dimensional) and 2D barcode.

12. The image processing system according to claim 8, further comprising:

associating each of the plurality of product labels including the respective barcode with a physical location in a retail environment.

13. The image processing system according to claim 12, wherein the step of detecting and localizing a plurality of barcode-candidate-regions within the captured image is biased to generate relatively more false-positives than false-negatives.

14. The image processing system according to claim 8, further comprising:

fusing the decoded readable barcodes and identifying a respective product label based on the fusion.

15. A method of performing product label identification, the product label including one or more barcodes and other product label information, the method comprising:

an image capturing device capturing an image of the product label and storing the captured image in a memory operatively associated with the image capturing device;

a processor operatively associated with the memory detecting and localizing one or more barcode candidate regions within the captured image of the product label;

cropping the detected and localized one or more barcode candidate regions to generate one or more sub-mages including the one or more barcode candidate regions;

processing each sub-image to decode readable barcodes included in the barcode candidate regions and identify barcode candidate regions including an unreadable barcode;

processing all or a portion of the captured image of the product label associated with the unreadable barcode to determine all or part of the other product label information association with the unreadable barcode;

comparing the determined other product label information to a plurality of product label templates to determine a layout associated with the product label including an unreadable barcode candidate region;

processing the captured image of the product label to extract all or part of the other product label information based on the determined layout associated with the product label; and

identifying the captured image of the product label as including one of a plurality of unique predefined product labels.

16. The method of performing product label identification according to claim 15, wherein an estimated orientation of a barcode candidate region associated with a nonreadable barcode is used to apply a correction to all or part of the captured image of the product label.

17. The method of performing product label identification according to claim 16, wherein the correction is associated with one or more of skew and rotation.

18. The method of performing product label identification according to claim 15, wherein the step of processing the captured image of the product label selects and uses one of a plurality of extraction algorithms based on the determined layout associated with the product label.

19. The method of performing product label identification according to claim 15, wherein the step of identifying the captured image of the product label includes associating a relative confidence value with the identification of the product label.

20. The method of performing product label identification according to claim 15, wherein the product label identification includes a SKU (stock keeping unit).

21. The method of performing product label identification according to claim 15, wherein the other product label information includes one or more SKU text, price text, product information text, logo, and color patch.