Methods and arrangements for identifying objects

• US 9,129,277 B2
• Filed: 07/19/2013
• Issued: 09/08/2015
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: MacIntosh, Brian T.
• Primary Examiner(s): LEE, SEUNG H
• Application Number:
  US13/946,968
• Publication Number:
  US 20140052555A1
• Time in Patent Office: 781 Days
• Field of Search: 235/375, 235/378, 235/383, 235/385
• US Class Current: 1/1
• CPC Class Codes: G06Q 20/208 Input by product or record sensing, e.g. weighing or scanner processing
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G07G 1/0009 Details of the software in the checkout register, electronic cash register [ECR] or point of sale terminal [POS]
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: G07G 1/0063 with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
• CPC Class Codes: G07G 1/06 with provision for the noting of the money to be paid
• CPC Class Codes: G07G 1/12 electronically operated digital data processing aspects G06Q20/00

Abstract

In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views—further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modelled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. Logos may be identified and used—or ignored—in product identification. A great variety of other features and arrangements are also detailed.

Claims

1. A method of identifying a retail product, based at least in part on assessing correspondence between image recognition features associated with the retail product and image recognition features associated with a collection of reference products, the method including the acts:

receiving plural recognition features derived from imagery associated with the retail product;

using a hardware processor configured with software, distinguishing a subset of the received features that are associated with a logo, said logo being present on plural of said reference products; and

taking an action in response to said distinguishing, wherein said action comprises at least one of;

(a) providing, via a user interface of a camera-equipped portable device that captured said imagery of the retail product, information indicating that the camera should be moved further away from the product, so as to capture a wider field of view;

(b) weighting said subset of received features less than other of the received features, in determining a match between the retail product and one of said reference products;

(c) disregarding said subset of received features in determining a match between the retail product and one of said reference products;

(d) disregarding features other than said subset of features, and identifying plural reference products having said logo;

or

(e) treating said determined subset of received features differently than others of the received features, in identifying one or more reference products.

2. The method of claim 1 in which the action includes providing, via a user interface of a camera-equipped portable device that captured said imagery of the retail product, information indicating that the camera should be moved further away from the product, so as to capture a wider field of view.

3. The method of claim 1 in which the action includes weighting said subset of received features less than other of the received features, in determining a match between the retail product and one of said reference products.

4. The method of claim 1 in which the action includes disregarding said subset of received features in determining a match between the retail product and one of said reference products.

5. The method of claim 1 in which the action includes disregarding features other than said subset of features, and identifying plural reference products having said logo.

6. The method of claim 1 in which the action includes treating said determined subset of received features differently than others of the received features, in identifying one or more reference products.

7. A method of enrolling a retail product in a reference product database, the method including the acts:

receiving plural recognition features derived from imagery associated with the retail product;

using a hardware processor configured with software, distinguishing a subset of the received features that are associated with a logo, said logo being present on plural of said reference products; and

treating said determined subset of features differently in enrolling said received recognition features in the reference product database, wherein said treating includes flagging said determined subset of features in the database, so that other processes accessing information in the database can identify the flagged features as associated with a logo.

8. A method of enrolling a retail product in a reference product database, the method including the acts:

receiving plural recognition features derived from imagery associated with the retail product;

using a hardware processor configured with software, distinguishing a subset of the received features that are associated with a logo, said logo being present on plural of said reference products; and

treating said determined subset of features differently in enrolling said received recognition features in the reference product database, wherein said treating includes storing weighting values in the database, in association both with said determined subset of features, and others of said received recognition features, wherein the weighting values associated with the determined subset of features have a statistical distribution different than the weighting values association with said others of said received recognition features.

9. A method comprising:

receiving plural recognition features derived from imagery associated with a retail product;

identifying recognition features in a reference data structure that correspond to certain of the received features;

using a hardware processor configured with software, scoring a match between the retail product and a reference product based on said correspondence;

wherein said scoring includes weighting, based on auxiliary data stored in the data structure, correspondence between one recognition feature in the reference data structure and one recognition feature among the received recognition features.

10. The method of claim 9 in which the auxiliary data comprises data indicating that the reference recognition feature is due to a visible feature on a reference product that is also found on other reference products.

11. The method of claim 9 in which the auxiliary data comprises data indicating that the reference recognition feature is due to a logo on a reference product that is also found on other reference products.