Apparatus for enhancing images using flow estimation

• US 6,490,364 B2
• Filed: 06/25/2001
• Issued: 12/03/2002
• Est. Priority Date: 08/28/1998
• Status: Expired due to Term

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Kumar, Rakesh, Sawhney, Harpreet Singh, Bergen, James Russell, Lubin, Jeffrey, Hanna, Keith James
• Primary Examiner(s): Johns, Andrew W.
• Application Number:
  US09/888,693
• Publication Number:
  US 20010036307A1
• Time in Patent Office: 526 Days
• Field of Search: 382/107, 382/154, 382/254, 382/275, 382/284, 382/294, 382/299, 382/318
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method and apparatus for accurately computing parallax information as captured by imagery of a scene. The method computes the parallax information of each point in an image by computing the parallax within windows that are offset with respect to the point for which the parallax is being computed. Additionally, parallax computations are performed over multiple frames of imagery to ensure accuracy of the parallax computation and to facilitate correction of occluded imagery.

Claims

1. Apparatus for enhancing an image comprising:

a first imaging device for producing images at a first resolution;

a second imaging device for producing images at a second resolution; and

an image processor for aligning said images from said first and second imaging devices to form a plurality of aligned images, computing a flow estimation for each of the aligned images, and using the flow estimations to enhance regions in a first aligned image with information from at least one other aligned image.

2. The apparatus of claim 1 wherein said image processor comprises:

an image flow generator.

3. The apparatus of claim 2 wherein said image flow generator is a parallax computer.

4. The apparatus of claim 3 wherein said parallax computer further comprises one or more augmentation modules selected from the group consisting of:

a module for dividing the images into tiles, a depth correlator, a module which performs Just Noticeable Differences, a correspondence checker, and a blank area avoidance module.