System and method for determining optical flow

• US 5,680,487 A
• Filed: 05/02/1994
• Issued: 10/21/1997
• Est. Priority Date: 12/23/1991
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Markandey, Vishal
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US08/237,946
• Time in Patent Office: 1,268 Days
• Field of Search: 382/216, 382/212, 382/302, 382/312, 382/103, 382/107, 382/226, 382/227, 382/291, 382/294, 382/299, 382/300, 348/153, 348/154, 348/155
• US Class Current: 382/291
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53

Abstract

A method and system are provided for determining optical flow between first and second images. First and second multiresolution images are generated (108, 110) from the first and second images, respectively, such that each multiresolution image has a plurality of levels of resolution. A multiresolution optical flow field is initialized (112) at a first one of the resolution levels. At each resolution level higher than the first resolution level, a residual optical flow field is determined (120) at the higher resolution level, and the multiresolution optical flow field is updated (122) by adding the residual optical flow field.

Claims

1. A computer implemented method of determining optical flow between first and second images, comprising the steps of:

sensing the first image and the second image by a sensor;

generating first and second multi-resolution images from the first and second images, respectively, such that each of said first and second multi-resolution images includes a plurality of levels of resolution;

initializing a multi-resolution optical flow field at a first resolution level of said levels of resolution;

at each of said resolution levels higher than said first resolution level, determining a residual optical flow field at each of said higher resolution levels;

updating said multi-resolution optical flow field by adding said residual optical flow field; and

wherein said step of determining said residual optical flow field comprises the steps of;

expanding said multi-resolution optical flow field from a lower one of said resolution levels to said higher resolution level;

generating a registered image at said higher resolution level by registering said first multi-resolution image relative to said second multi-resolution image at said higher resolution level in response to said multi-resolution optical flow field after said expanding step; and

determining said residual optical flow field by determining an optical flow field between said registered image and said first multi-resolution image at said higher resolution level.

2. The computer implemented method of claim 1, wherein said step of determining said residual optical flow field comprises the step of determining said residual optical flow field according to brightness.

3. The computer implemented method of claim 1, wherein said step of determining said residual optical flow field comprises the step of determining said residual optical flow field according to gradient constancy assumptions.

4. The computer implemented method of claim 1, wherein said step of determining said residual optical flow field comprises the step of determining said residual optical flow field according to correlation techniques.

5. The computer implemented method of claim 1, wherein said step of determining said residual optical flow field comprises the step of determining said residual optical flow field according to Fourier transform techniques.

6. The computer implemented method of claim 1, wherein the method comprises the step of providing a number of pixels of said lower resolution level being half a number of pixels of said higher resolution level.

7. A computer implemented method of determining optical flow between first and second images, comprising the steps of:

sensing the first image and the second image by a sensor;

generating first and second multi-resolution images from the first and second images, respectively, such that each of said first and second multi-resolution images includes a plurality of levels of resolution;

initializing a multi-resolution optical flow field at a first resolution level of said levels of resolution;

at each of said resolution levels higher than said first resolution level, determining a residual optical flow field at each of said higher resolution levels; and

updating said multi-resolution optical flow field by adding said residual optical flow field; and

wherein the method comprises the step of providing a number of pixels of a highest one of said resolution levels being substantially equal to a number of pixels of the first image; and

wherein said step of generating said first multi-resolution image comprises the steps of;

for each pixel of said highest resolution level, setting a value of said pixel equal to a value of a corresponding pixel of the first image; and

for each pixel of a resolution level lower than said highest resolution level, setting a value of each said pixel according to a weighted averaging function of values of corresponding pixels of said highest resolution level.

8. The method of claim 7, wherein said weighted averaging function approximates Gaussian filtering.