Method and apparatus for producing optical flow using multi-spectral images

• US 5,265,172 A
• Filed: 03/03/1992
• Issued: 11/23/1993
• Est. Priority Date: 10/13/1989
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Flinchbaugh, Bruce E., Markandey, Vishal
• Primary Examiner(s): Couso, Jose L.
• Application Number:
  US07/845,758
• Time in Patent Office: 630 Days
• Field of Search: 324/306, 324/309, 358/103, 358/105, 358/107, 358/133, 382/1, 382/6, 382/17, 382/41, 382/42, 382/43, 382/44, 356/39, 356/311, 250/461.2, 250/208.1
• US Class Current: 382/107
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for producing optical flow obtains a first image of a first spectral region of a field of view, obtains a second image of a second spectral region of the field of view, and processes the images to obtain the optical flow of the field of view. The first and second spectral regions are non-identical, as, for example, the visible and infrared bandwidths.

Claims

1. A method of determining optical flow comprising the steps of:

obtaining a first image from a first spectral region of a field of view at a first time by a first image sensor;

obtaining a second image from a second spectral region of the field of view at said first time by a second image sensor; and

processing said first and second images to obtain the optical flow of the field of view by a processor.

2. The method of determining optical flow as defined in claim 1, wherein said processing step comprises the step of producing first order partial derivatives of intensities at one or more points within the first and second images for said first and second spectral regions.

3. the method of determining optical flow as defined in claim 2, wherein the first order partial derivatives have the following relationships:

space="preserve" listing-type="equation">E.sub.xs1.sup.u +E.sub.ys1.sup.v +E.sub.ts1 =0

space="preserve" listing-type="equation">E.sub.xs2.sup.u +E.sub.ys2.sup.v +E.sub.ts2 =0

, and wherein E_xs1, E_ys1 and E_ts1 are the first order partial derivatives of the intensities in said first spectral region with respect to image coordinates (x,y) and time (t) respectively, and wherein E_xs2, E_ys2 and E_ts2 are the first order partial derivatives of the intensities in said second spectral region with respect to the image coordinates (x,y) and the time (t) respectively and, and wherein (u, v) is an associated optical flow vector.

4. The method of determining optical flow as defined in claim 1, wherein said processing step comprises smoothing said first and second images.

5. The method of determining optical flow as defined in claim 1, wherein said processing step further comprises the step of smoothing said optical flow.

6. The method of determining optical flow as defined in claim 5, wherein said processing step comprises the step of median-filtering said optical flow.

7. The method of determining optical flow as defined in claim 1, wherein said method further comprises the step of substantially spatially registering said first and second images of said first and second spectral regions.

8. The method of determining optical flow as defined in claim 1, wherein said processing step comprises the step of substantially spatially registering said first and second images.

9. The method of determining optical flow as defined in claim 1, wherein said processing step comprises the step of producing second order partial derivatives of the intensities of the first and second images for said first and second spectral regions.

10. The method of determining optical flow as defined in claim 9, wherein the method comprises the step of providing the second order partial derivatives with the following relationships:

space="preserve" listing-type="equation">E.sub.xxs1.sup.u +E.sub.xys1.sup.v +E.sub.xts1 =0

space="preserve" listing-type="equation">E.sub.xys1.sup.u +E.sub.yys1.sup.v +E.sub.yts1 =0

space="preserve" listing-type="equation">E.sub.xxs2.sup.u +E.sub.xys2.sup.v +E.sub.xts2 =0

space="preserve" listing-type="equation">E.sub.xys2.sup.u +E.sub.yys2.sup.v +E.sub.yts2 =0

, and wherein E_xxa1, E_xys1 and E_yys1 are the second order partial derivatives of the intensities of said first spectral region with respect to image coordinates (x,y), and wherein E_xxs2, E_xys2 and E_yys2 are additional second order partial derivatives of additional intensities of said second spectral region with respect to said image coordinates (x,y), and wherein E_xts1 and E_yts1 are the second order partial derivatives of said intensities of said first spectral region, and wherein and E_xts2 and E_yts2 are further second order partial derivatives of further intensities of said second spectral region with respect to said image coordinates (x,y) and time (t), and wherein (u,v) is an associated optical flow vector.

11. The method of determining optical flow as defined in claim 9, wherein said processing step further comprises the step of smoothing said first and second images.

12. The method of determining optical flow as defined in claim 9, wherein said processing step further comprises the step of smoothing said optical flow.

13. The method of determining optical flow as defined in claim 12, wherein said processing step comprises the step of median-filtering said optical flow.

14. The method of determining optical flow as defined in claim 9, wherein said method further comprises the step of substantially spatially registering said first and second images of said first and second spectral regions.

15. The method of determining optical flow as defined in claim 9, wherein said processing step further comprises the step of substantially spatially registering said images.

16. A method of determining optical flow comprising the steps of:

obtaining a first image from a first spectral region of a field of view at a first time by a first image sensor;

obtaining a second image from a second spectral region of the field of view at said first time by a second image sensor; and

processing said first and second images to obtain first and second order partial derivatives of intensities of said first and second images for said first and second spectral regions by a processor to thereby obtain the optical flow of said first and second images.

17. The method of determining optical flow as defined in claim 16, wherein said processing step comprises the step of smoothing said first and second images.

18. The method of determining optical flow as defined in claim 16, wherein said processing step comprises the step of smoothing said optical flow.

19. The method of determining optical flow as defined in claim 18, wherein said processing step further comprises the step of median-filtering said optical flow.

20. Apparatus for determining optical flow comprising:

a first image sensor for obtaining a first image from a first spectral region of a field of view;

a second image sensor for obtaining a second image from a second spectral region of the field of view, wherein said first image sensor and said second image sensor are operable to obtain said first and second images at the same instance of time; and

a processor for processing said first and second images to obtain the optical flow of the field of view.

21. Apparatus for determining optical flow as defined in claim 20, wherein said processor comprises means for obtaining first order partial derivatives of intensities of the first and second images for said first and second spectral regions.

22. Apparatus for determining optical flow as defined in claim 20, wherein said processor comprises means for obtaining second order partial derivatives of intensities of the first and second images for said first and second spectral regions.