IMAGE BLOCK SELECTION FOR EFFICIENT TIME-LIMITED DECODING

• US 20160364623A1
• Filed: 06/08/2016
• Published: 12/15/2016
• Est. Priority Date: 06/11/2015
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Evans, Eric D., Filler, Tomas
• Granted Patent: US 9,922,220 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06K 7/10722 Photodetector array or CCD scanning
• CPC Class Codes: G06K 7/146 the method including quality enhancement steps
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 30/1916 Validation; Performance evaluation
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

Object recognition by point-of-sale camera systems is aided by first removing perspective distortion. Yet pose of the object—relative to the system—depends on actions of the operator, and is usually unknown. Multiple trial counter-distortions to remove perspective distortion can be attempted, but the number of such trials is limited by the frame rate of the camera system—which limits the available processing interval. One embodiment of the present technology examines historical image data to determine counter-distortions that statistically yield best object recognition results. Similarly, the system can analyze historical data to learn what sub-parts of captured imagery most likely enable object recognition. A set-cover strategy is desirably used. In some arrangements, the system identifies different counter-distortions, and image sub-parts, that work best with different clerk- and customer-operators of the system, and processes captured imagery accordingly. A great variety of other features and arrangements are also detailed.

Claims

1. A method of enhancing operation of a visual recognition system that is allotted a limited time interval to produce an identification result, which system includes a camera that provides image frames for extraction of identification information therefrom, the method comprising the acts:

(a) for one sub-part of one image frame, applying at least M different values of tilt-angle correction in attempting to extract identification information from M resultant tilt angle-corrected counterparts of said one sub-part of said one image frame, where M>

1;

(b) performing act (a) at least N times, each with a differently-located sub-part of said one image frame, where N>

1, wherein extraction of identification information from said one image frame is attempted for at least M*N combinations of tilt angle correction values and sub-part locations;

(c) performing act (b) plural times, each with a different image frame drawn from a set P of image frames;

(d) determining which single combination of tilt angle correction value and sub-part location resulted in successful extraction of identification information from a greatest number of said image frames in said set P;

adding this combination of tilt angle correction value and sub-part location to a listing of decoding parameters; and

removing, from the set P, those image frames from which identification information was successfully extracted using said combination of tilt angle correction value and sub-part location; and

(e) repeating act (d) one or more times with successively smaller sets P;

wherein the method yields a listing of decoding parameters that is ranked by likelihood of successfully extracting unique identification information from an image frame, given the previous application of other decoding parameters in said listing, wherein attempts to extract identification information from a new image can successively apply parameters from said ranked listing until said limited time interval elapses.

2. The method of claim 1 that further includes the acts:

capturing an image frame with said camera;

processing the captured image frame with a first set of parameters from said ranked listing, to yield a first processed image excerpt;

attempting to extract identification information from said first processed image excerpt; and

upon failure to successfully extract identification information from said first processed image excerpt;

processing the captured image frame with a next set of parameters from said ranked listing, to yield a second processed image excerpt; and

attempting to extract identification information from said second processed image excerpt.

3. The method of claim 1 in which:

act (a) includes, for one sub-part of one image frame, applying at least M different values of tilt angle-correction, and for each such value, applying R different values of bearing-angle correction, in attempting to extract identification information from M*R resultant tilt angle- and bearing angle-corrected counterparts of said one sub-part of said one image frame, where M>

1 and R>

1;

act (b) includes performing act (a) at least N times, each with a differently-located sub-part of said one image frame, where N>

1, wherein extraction of identification information from said one image frame is attempted for at least M*R*N combinations of tilt angle correction values, bearing angle correction values, and sub-part locations; and

act (d) includes determining which single combination of tilt angle correction value, bearing angle correction value, and sub-part location resulted in successful extraction of identification information from a greatest number of said image frames in said set P;

adding this combination of tilt angle correction value, bearing angle correction value and sub-part location to a listing of decoding parameters; and

removing, from the set P, those image frames from which identification information was successfully extracted using said combination of tilt angle correction value, bearing angle correction value and sub-part location.

4. The method of claim 3 in which:

act (a) includes, for one sub-part of one image frame, applying at least M different values of tilt angle-correction, and for each such value of tilt angle-correction, applying R different values of bearing angle-correction, and for each such value of bearing angle-correction, applying S different values of camera model-correction, in attempting to extract identification information from M*R*S resultant tilt angle-, bearing angle- and camera model-corrected counterparts of said one sub-part of said one image frame, where M>

1, R>

1 and S>

1; and

act (b) includes performing act (a) at least N times, each with a differently-located sub-part of said one image frame, where N>

1, wherein extraction of identification information from said one image frame is attempted for at least M*R*S*N combinations of tilt angle correction values, bearing angle correction values, and sub-part locations; and

act (d) includes determining which single combination of tilt angle correction value, camera model values, and sub-part location resulted in successful extraction of identification information from a greatest number of said image frames in said set P;

adding this combination of tilt angle correction value, camera model values and sub-part location to a listing of decoding parameters; and

removing, from the set P, those image frames from which identification information was successfully extracted using said combination of tilt angle correction value, camera model values and sub-part location.

5. The method of claim 1 in which:

act (a) includes, for one sub-part of one image frame, applying at least M different values of tilt angle-correction, and for each such value of tilt angle-correction, applying S different values of camera model-correction, in attempting to extract identification information from M*S resultant tilt angle- and camera model-corrected counterparts of said one sub-part of said one image frame, where M>

1 and S>

1;

act (b) includes performing act (a) at least N times, each with a differently-located sub-part of said one image frame, where N>

1, wherein extraction of identification information from said one image frame is attempted for at least M*S*N combinations of tilt angle correction values, bearing angle correction values, and sub-part locations.

6. The method of claim 1 in which the extracting comprises extracting an image fingerprint, and identifying corresponding information in a fingerprint database.

7. The method of claim 1 in which the extracting comprises barcode decoding.

8. The method of claim 1 in which the extracting comprises digital watermark decoding.

9. A method employing a camera scanning system at a retail checkout, comprising the acts:

capturing a frame of imagery depicting a retail product presented for checkout, using said camera scanning system;

identifying a barcode region in the frame, imaged from a portion of the product;

discerning pose information for said portion of the product relative to the camera scanning system, by reference to presentation of said barcode region in the frame; and

employing said discerned pose information in attempting to decode a digital watermark payload from the frame of imagery.

10. A method employing a camera scanning system at a retail checkout, comprising the acts:

storing reference images captured by the camera scanning system, as a person operates the system by moving products past a window thereof;

determining, from analysis of plural stored reference images, a set cover-based ordering of decoding parameters useful in extracting product identifying information from said reference images;

storing said ordering of decoding parameters in association with an identification of said person; and

when, after a period of use by one or more other people, said person again operates a scanning system, recalling said stored ordering of decoding parameters, and using said ordering of parameters in attempting extraction of product identifying information from scanning system imagery.