Methods and arrangements for identifying objects

• US 9,367,770 B2
• Filed: 01/25/2013
• Issued: 06/14/2016
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Footen, Marc G.
• Primary Examiner(s): STANFORD, CHRISTOPHER J
• Application Number:
  US13/750,752
• Publication Number:
  US 20130223673A1
• Time in Patent Office: 1,236 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06K 7/10861 sensing of data fields affixed to objects or articles, e.g. coded labels postal sorting B07C3/14, conveying articles B65G47/48
• CPC Class Codes: G06K 7/1456 determining the orientation of the optical code with respect to the reader and correcting therefore
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 20/201 Price look-up processing, e.g. updating
• CPC Class Codes: G06Q 20/208 Input by product or record sensing, e.g. weighing or scanner processing
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06T 2201/0065 Extraction of an embedded watermark; Reliable detection
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/63 : Scene text, e.g. street names
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 2201/09 : Recognition of logos
• CPC Class Codes: G06V 30/19173 : Classification techniques
• CPC Class Codes: G06V 30/224 : of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2247 : Characters composed of bars, e.g. CMC-7
• CPC Class Codes: G07G 1/0036 : Checkout procedures
• CPC Class Codes: G07G 1/0045 : with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
• CPC Class Codes: G07G 1/0054 : with control of supplementary check-parameters, e.g. weight or number of articles
• CPC Class Codes: G07G 1/0063 : with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
• CPC Class Codes: G09G 2340/12 : Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2380/04 : Electronic labels
• CPC Class Codes: G09G 2380/06 : Remotely controlled electronic signs other than labels

Abstract

In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views—further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modelled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. A great variety of other features and arrangements are also detailed.

Claims

1. A method employing an electronic shelf labeling unit, mounted on a front of a store shelf, adjacent a store aisle, the unit having a front facing the aisle, and a rear facing retail items stocked on said shelf, the method comprising the acts:

sensing image data from a retail item while the item is presented for sale on said shelf, using a rear-facing camera on the rear of said shelf labeling unit, said camera capturing said image data from a front surface of said item that faces the aisle;

recognizing the retail item by decoding plural-bit data encoded in a machine-readable indicia printed on said retail item and depicted in the image data captured by the electronic shelf labeling unit;

determining a price for the recognized retail item; and

electronically presenting said price for said item using a display on the front of the shelf labeling unit, by reference to the sensed identifying information;

wherein the method allows store personnel to move items on the shelf as-needed, and the electronic shelf labeling unit adapts automatically—

displaying the price of a retail item that is within view of the rear-facing camera.

2. The method of claim 1 that further includes wirelessly receiving price data for said item from a store computer.

3. The method of claim 1 that further includes sensing identifying information for plural different items that are positioned on the shelf within a field of view of said rear-facing camera.

4. The method of claim 3 that further includes electronically presenting a price for each of said plural different items.

5. The method of claim 3 that further includes

sensing identifying information for plural different items;

and

cyclically presenting a price for each of said plural different items.

6. The method of claim 1 that further includes presenting a customer with map information about the store, based on said sensed image data.

7. A system comprising:

a shelf along an aisle of a retail store, the shelf carrying an inventory of retail items facing the aisle, including a first item having first plural-bit data encoded in a machine-readable indicia printed thereon, and a second item having second plural-bit data encoded in a machine-readable indicia printed thereon, the second plural-bit data being different than the first plural-bit data;

an electronic price label display unit attached to a front of the shelf, said unit having a front and a back, and including;

an electronic display on the front, facing the aisle, for presenting price information for proximate retail items; and

a rear-facing camera on the back, directed away from the aisle, for capturing imagery from a front surface of the first and second items on the shelf;

the system further including;

a hardware processor configured to recognize retail items from imagery captured by the rear-facing camera, by decoding depictions of the machine-readable indicia on the first and second items; and

a wireless network for transmitting price data for each recognized retail item to the display unit;

wherein the system allows store personnel to move items on the shelf as-needed, and the electronic shelf labeling unit adapts automatically—

displaying the price of retail items that are within view of its rear-facing camera.