Color Haar Classifier for Retail Shelf Label Detection

• US 20190180150A1
• Filed: 12/13/2018
• Published: 06/13/2019
• Est. Priority Date: 12/13/2017
• Status: Abandoned Application

• Current Assignee: Bossa Nova Robotics IP, Inc.
• Original Assignee: Bossa Nova Robotics IP, Inc.
• Inventors: Taylor, Jonathan Davis, Natarajan, Ramkumar
• Application Number:
  US16/219,238
• Publication Number:
  US 20190180150A1
• Time in Patent Office: Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06K 7/1413 1D bar codes
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

A method for a multiple camera sensor suite mounted on an autonomous robot to be able to detect and recognize shelf labels using color Haar classifiers is described.

Claims

1. A shelf label detection method, comprising the steps of:

determining a color Haar feature consistent with a shelf label and having a defined scale and location across at least some channels of a chosen color space;

assigning a feature vector having a feature value to the color Haar feature; and

training a color Haar classifier to identify shelf labels using feature values of the feature vectors.

2. The shelf label detection method of claim 1, wherein the shelf label and the color Haar feature are both rectangular.

3. The shelf label detection method of claim 1, wherein the chosen color space is LAB.

4. The shelf label detection method of claim 1, wherein the chosen color space is LAB and wherein an assigned feature value at a location is a sum of squares length of the feature vector with a sign given by a luminance (L) channel.

5. The shelf label detection method of claim 1, wherein the color Haar classifier is a boosted Haar cascade classifier.

5-1. The shelf label detection method of claim 1, wherein the color Haar classifier is a boosted Haar cascade classifier.

5-2. The shelf label detection method of claim 1, wherein the trained color Haar classifier is used in an autonomous robot to monitor inventory by identifying shelf labels.

7. An object detection method, comprising the steps of:

creating a panoramic image including products on a shelf, with possible products being surrounded by bounding boxes;

detecting shelf labels by evaluating a color Haar feature consistent with a shelf label and having a defined a scale and location across at least some channels of a chosen color space;

assigning a feature vector having a feature value to the color Haar feature;

generating a color Haar classifier to identify shelf labels using feature values of feature vectors; and

associating identified shelf labels with the possible products in the bounding boxes.

8. An inventory monitoring method, comprising the steps of:

providing an autonomous robot able to capture images that include products on a shelf and shelf labels;

detecting the shelf labels by evaluating a color Haar feature consistent with a shelf label and having a defined a scale and location across at least some channels of a chosen color space;

assigning a feature vector having a feature value to the color Haar feature;

generating a color Haar classifier to identify shelf labels using feature values of feature vectors; and

associating identified shelf labels with the products on the shelf.

9. An inventory monitoring method, comprising the steps of:

capturing high-resolution source images of an aisle that include shelf associated stock labels;

creating a low-resolution panoramic image from the high-resolution source images;

detecting shelf associated stock labels in the low-resolution panoramic image using color Haar features consistent with a shelf label and having a defined a scale and location across at least some channels of a chosen color space;

mapping position of shelf associated stock labels in the low-resolution panoramic image to a position in the high-resolution source images; and

reading shelf associated stock labels using the high-resolution source images.

10. An inventory monitoring method, comprising the steps of:

capturing high-resolution source images of an aisle comprising product, product labels, and shelf associated stock labels;

creating a panoramic image having a lower resolution and including products from high-resolution source images of an aisle;

detecting shelf associated stock labels in high-resolution source images using a Haar classifier; and

reading product labels or shelf associated stock labels using the high-resolution source images and associating the shelf associated stock labels with product in the panoramic image.