METHODS AND ARRANGEMENTS FOR IDENTIFYING OBJECTS

• US 20130223673A1
• Filed: 01/25/2013
• Published: 08/29/2013
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Rodriguez, Tony F., Rhoads, Geoffrey B., Lord, John D., Reed, Alastair M., Bai, Yang, Calhoon, Sean, Evans, Eric D., Gerlach, Rebecca L., Stach, John, Filler, Tomas, Footen, Marc G.
• Granted Patent: US 9,367,770 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06K 7/10861 sensing of data fields affixed to objects or articles, e.g. coded labels postal sorting B07C3/14, conveying articles B65G47/48
• CPC Class Codes: G06K 7/1456 determining the orientation of the optical code with respect to the reader and correcting therefore
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 20/201 Price look-up processing, e.g. updating
• CPC Class Codes: G06Q 20/208 Input by product or record sensing, e.g. weighing or scanner processing
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06T 2201/0065 Extraction of an embedded watermark; Reliable detection
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/63 : Scene text, e.g. street names
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 2201/09 : Recognition of logos
• CPC Class Codes: G06V 30/19173 : Classification techniques
• CPC Class Codes: G06V 30/224 : of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2247 : Characters composed of bars, e.g. CMC-7
• CPC Class Codes: G07G 1/0036 : Checkout procedures
• CPC Class Codes: G07G 1/0045 : with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
• CPC Class Codes: G07G 1/0054 : with control of supplementary check-parameters, e.g. weight or number of articles
• CPC Class Codes: G07G 1/0063 : with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
• CPC Class Codes: G09G 2340/12 : Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2380/04 : Electronic labels
• CPC Class Codes: G09G 2380/06 : Remotely controlled electronic signs other than labels

Abstract

In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views—further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modelled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. A great variety of other features and arrangements are also detailed.

Claims

1. A method comprising:

moving an item to be purchased, along a path, the item having an irregular surface;

capturing first 2D image data from the irregular surface when it is at a first position along the path;

capturing second 2D image data from the irregular surface when it is at a second, different, location along said path;

processing the first and second 2D image data, in conjunction with information related to said path, to discern first 3D spatial orientation information for a first patch on the surface;

processing the first and second 2D image data, in conjunction with information related to said path, to discern second 3D spatial orientation information for a second patch on the surface, the second 3D spatial orientation information being different than the first 3D spatial orientation information; and

by reference to said discerned first and second 3D spatial orientation information, decoding a machine-readable identifier on the irregular surface, the identifier spanning at least a portion of the first patch and a portion of the second patch.

2. A supermarket checkout apparatus including a processor coupled to a memory and to an output device, the memory storing instructions and rule data causing the processor to serve as a bagging expert system, the output device providing guidance to a human concerning a manner of packing items into bags, based on application of the rule data by the processor.

3. A method comprising

determining a volumetric configuration of a pile of objects presented at a checkout station by a shopper in a store;

and

determining data about each of plural objects in the pile, said determined data including identification data and orientation data;

wherein said determining includes using collected sensor data to refine an identification hypothesis about a particular object in the pile.

4. The method of claim 3 that further includes outputting information identifying where in the pile an unidentified object is located.

5. The method of claim 3 in which the collected sensor data comprises location-related optical signaling sensed from different light sources in the store.

6. The method of claim 3 in which the collected sensor data comprises data from a sensor used to monitor removal of an object from a store shelf.

7. The method of claim 6 in which the collected sensor data comprises camera data, and the method includes presenting imagery based on said camera data on a display visible to shoppers in the store.

8. A method comprising:

sensing first information from an object presented at a checkout station by a shopper in a store; and

using said sensed first information—

in conjunction with sensed second information—

in identifying the object;

wherein the sensed second information is of a different type that the first sensed information.

9. The method of claim 8 wherein the sensed second information was previously collected in a shopping portion of the store, while the shopper was in said shopping portion of the store.

10. The method of claim 8 that includes sensing the second information through a process that includes blowing air past the object.

11. The method of claim 8 that includes sensing the second information through a process that includes acoustic sensing.

12. The method of claim 8 that includes sensing the second information through a process that includes detecting placement of an item in a shopping cart.

13. The method of claim 8 that includes sensing the second information through a process that includes sensing a pose of a hand.

14. The method of claim 8 wherein one of said first or second sensed information comprises information sensed using a time of flight camera.

15. The method of claim 8 that further includes, after identifying the object, projecting price information for visual association with the object.

16. The method of claim 8 that includes signaling identification of the object by illuminating the object in a distinctive fashion.

17. A method comprising:

capturing imagery from a cylindrical object;

detecting at least a section of an ellipse in the captured imagery, and determining therefrom an orientation of an axis of the cylindrical object; and

attempting to decode a digital watermark from the captured imagery, based on a hypothesis that a watermark on the cylindrical object is aligned with said determined axis orientation.

18. A method comprising:

sensing identifying information from an item presented for sale on a shelf of a store, using a shelf-mounted unit;

electronically presenting a price for said item using a display proximate to the item, by reference to the sensed identifying information.

19. A method comprising:

with a camera in a shopper'"'"'s mobile device, capturing imagery depicting plural product packages on a store shelf;

processing the imagery to decode digital watermark data encoded in said product packages; and

presenting an augmented reality display in which expiration date information for different of the packages is displayed as a graphical overlay on the captured imagery.

20. The method of claim 19 in which the graphical overlay comprises a colored heat map, wherein the shopper can identify newest and oldest inventory among the plural packages.