System And Method For Recovering Image Detail From Multiple Image Frames In Real-Time

• US 20070242883A1
• Filed: 04/11/2007
• Published: 10/18/2007
• Est. Priority Date: 04/12/2006
• Status: Active Grant

• Current Assignee: Scanbuy Incorporated
• Original Assignee: Scanbuy Incorporated
• Inventors: Kruppa, Hannes Martin
• Granted Patent: US 8,150,163 B2
• Time in Patent Office: Days
• US Class Current: 382/181
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

This invention describes an efficient super-resolution method for image enhancement that leverages prior knowledge about the specific task for which detail information is recovered. The particular case of barcode scanning is considered. The barcode localization and decoding algorithm of the present invention employs a subpixel-accurate search algorithm which allows for estimating barcode bit values at a higher resolution than that available in the image data itself. It thus allows for the synthesis of image detail from multiple frames with the result containing more detail than any of the input frames. For efficiency, it leverages special properties of the quadrilateral target object as well as prior knowledge about constant patterns in the barcodes of interest. This allows for real-time software implementations on portable devices such as camera-equipped cell phones where super-resolution helps to overcome some of the typical camera resolution and processing power constraints.

Claims

1. ) A process for recovering image detail from multiple image frames in real-time, comprising the steps of:

Detecting and locating a four-sided object or region of interest in the image frame

Efficient subpixel-accurate refinement of the four corner coordinates

Computing a set of features over the region of interest after projection onto an orthoframe (rectification, projective mapping, warping)

Estimating the noise level within the considered region of interest and rejecting noisy frames

Collecting statistical information of the extracted features and predicting detail information about the object based on those statistics.

Building a representation of the object from multiple frames that contains more detail information about the object than any of the individual frames

2. ) A process for recovering image detail as defined in claim 1, where the registration process (i.e. subpixel accurate corner refinement) uses prior knowledge about the object'"'"'s appearance for computational efficiency.

3. ) A process for recovering image detail as defined in claim 1, where the noise estimation process uses prior knowledge about the object'"'"'s appearance for computational efficiency.

4. ) A process for recovering image detail as defined in claim 1, where the extracted features are obtained using image sharpening.

5. ) A process for recovering image detail as defined in claim 1, where the registration process uses prior knowledge about the target object'"'"'s appearance for computational efficiency.

6. ) A process for recovering image detail as defined in claim 1, where feature statistics are represented in non-parametric form as histograms.

7. ) A process for recovering image detail as defined in claim 1, where the object of interest is a four-sided barcode symbol in one or two dimensions.

8. ) A process for recovering image detail as defined in claim 7, where a barcode image is warped onto an orthoframe of the barcode'"'"'s dimension (rows and columns in the 2D case) and where all consequent computations are carried out on said orthoframe independent of the underlying image resolution. In other words, the algorithm scales with the complexity of the barcode and is independent of the complexity of the (image) data.

9. ) A process for recovering image detail as defined in claim 7, where barcode bits are predicted based on histogram peaks in the feature statistics.

10. ) A process for recovering image detail as defined in claim 2, where the barcode'"'"'s dimension in terms of rows and columns (“

version number”

) is recovered first.

11. ) A process for recovering image detail as defined in claim 9, where the barcode dimension is “

locked”

(not further examined in subsequent frames) once it has been recovered with a high level of confidence.