Method and apparatus to facilitate correcting rolling shutter images

• US 20070154202A1
• Filed: 01/04/2006
• Published: 07/05/2007
• Est. Priority Date: 01/04/2006
• Status: Abandoned Application

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Tang, Bei, Kujawa, Gregory, Lee, King
• Application Number:
  US11/324,810
• Publication Number:
  US 20070154202A1
• Time in Patent Office: Days
• US Class Current: 396/153
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode
• CPC Class Codes: H04N 23/73 by influencing the exposure time

Abstract

A first and a second image (401 and 501) containing a shared region of interest (400) are captured (201 and 202) using a rolling shutter mode of image capture. These two images are captured at different times, such that the second image follows the first image by a given amount of time. A corrected image is then formed (203) of the region of interest by modifying at least one of the first and second images as a function, at least in part, of the given amount of time that separates capture of the two images along with pixel capture delay times as correspond to the rolling shutter mode of image capture (the latter referring, by approach, to pixel row capture delay times as characterize the rolling shutter image capture process itself)

Claims

1. A method comprising:

capturing a first image containing a region of interest using a rolling shutter mode of image capture;

capturing a second image that also contains the region of interest using a rolling shutter mode of image capture, wherein capturing the second image begins a given amount of time subsequent to capturing the first image;

forming a corrected image of the region of interest by modifying at least one of the first image and the second image as a function, at least in part, of;

the given amount of time; and

pixel capture delay times as correspond to the rolling shutter mode of image capture.

2. The method of claim 1 wherein capturing a first image and capturing a second image comprises capturing the first image and the second image using a complimentary metal oxide semiconductor-based image sensor.

3. The method of claim 1 wherein the rolling shutter mode of image capture comprises a row-by-row rolling shutter mode of image capture.

4. The method of claim 1 wherein the pixel capture delay times comprise pixel row capture delay times.

5. The method of claim 1 wherein forming a corrected image of the region of interest by modifying at least one of the first image and the second image as a function, at least in part, of the given amount of time and pixel capture delay times as correspond to the rolling shutter mode of image capture further comprises determining a motion metric as corresponds to apparent movement of the region of interest during the given amount of time, wherein the motion metric comprises a motion vector.

6. The method of claim 5 wherein forming a corrected image of the region of interest by modifying at least one of the first image and the second image as a function, at least in part, of the given amount of time and pixel capture delay times as correspond to the rolling shutter mode of image capture further comprises modifying pixels as comprise the image to be modified in accordance with the expression:

$I<br><br>\left(x,y\right)=I<br><br>\left(x+\lfloor <br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>x<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{T_r}{T_F}<br><br>y\rfloor <br><br>,y+\lfloor <br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>y<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{T_r}{T_F}<br><br>y\rfloor <br><br>\right)$

for all x, y within ROI

where;

I refers to an image pixel

T_r refers to pixel capture delay time

T_F refers to given amount of time between image capture

x refers to the column index value

y refers to the row index value

□

x refers to the x component of a motion vector

□

y refers to the y component of a motion vector

denotes rounding towards a zero operator

ROI denotes the region of interest.

7. The method of claim 1 wherein forming a corrected image of the region of interest by modifying at least one of the first image and the second image as a function, at least in part, of the given amount of time and pixel capture delay times as correspond to the rolling shutter mode of image capture further comprises determining a motion metric as corresponds to apparent movement of the region of interest during the given amount of time.

8. The method of claim 7 further comprising:

comparing the motion metric with a predetermined threshold to obtain a result;

when the result corresponds to a first category of result, responsively then forming the corrected image of the region of interest by modifying at least one of the first image and the second image;

when the result corresponds to a second category of result, responsively not then forming a corrected image of the region of interest.

9. The method of claim 1 further comprising:

using the corrected image of the region of interest to facilitate an object recognition process.

10. The method of claim 9 wherein the object recognition process comprises a vehicular license plate recognition process.

11. A method of facilitating compensation of a distorted image that has been captured using a rolling shutter mode of image capture that introduced rolling shutter distortion, comprising:

capturing a second image using a rolling shutter mode of image capture, wherein capturing the second image begins a given amount of time subsequent to capturing the distorted image;

determining a motion metric as corresponds to content of the distorted image and the second image;

determining to form a corrected image;

using a mapping approach to at least partially correct the rolling shutter distortion to provide a corrected image, wherein the mapping approach processes image information as a function, at least in part, of;

the given amount of time;

pixel capture delay times as correspond to the rolling shutter mode of image capture; and

the motion metric.

12. The method of claim 11 wherein the given amount of time substantially comprises a fraction of a second.

13. The method of claim 11 wherein determining to form a corrected image comprises determining whether sufficient distortion due to rolling shutter distortion has occurred to warrant providing a corrected image.

14. The method of claim 11 wherein the mapping approach implements a process represented by:

$I<br><br>\left(x,y\right)=I<br><br>\left(x+\lfloor <br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>x<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{T_r}{T_F}<br><br>y\rfloor <br><br>,y+\lfloor <br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>y<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{T_r}{T_F}<br><br>y\rfloor <br><br>\right)$

for all x, y within ROI

where;

I refers to an image pixel

T_r refers to pixel capture delay time

T_F refers to given amount of time between image capture

x refers to the column index value

y refers to the row index value

□

x refers to the x component of a motion vector

□

y refers to the y component of a motion vector

denotes rounding towards a zero operator

ROI denotes the region of interest.

15. An apparatus comprising:

an image memory operably coupled to receive captured images and having stored therein;

a first image containing a region of interest, which first image was captured via a rolling shutter mode of image capture and may therefore exhibit rolling shutter distortion due to relative movement of the region of interest;

a second image that also contains the region of interest, which second image was captured via a rolling shutter mode of image capture at a given amount of time subsequent to when the first image was captured;

a motion metric memory having stored therein a motion metric that corresponds to an amount of apparent motion as corresponds to the region of interest as between the first image and the second image;

a pixel mapping processor that is operably coupled to the image memory and the motion metric memory and that is configured and arranged to form a corrected image of the region of interest by modifying at least one of the first image and the second image as a function, at least in part, of;

the motion metric; and

pixel capture delay times as correspond to the rolling shutter mode of image capture.

16. The apparatus of claim 15 wherein the motion metric comprises a motion vector value.

17. The apparatus of claim 15 wherein the pixel capture delay times comprise pixel row capture delay times.

18. The apparatus of claim 15 wherein the pixel mapping processor comprises means for forming a corrected image of the region of interest by modifying at least one of the first image and the second image as a function, at least in part, of:

the motion metric; and

pixel capture delay times as correspond to the rolling shutter mode of image capture.

19. The apparatus of claim 15 wherein the pixel mapping processor comprises means for shifting pixels as comprise an image to be modified in accordance with the expression:

$I<br><br>\left(x,y\right)=I<br><br>\left(x+\lfloor <br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>x<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{T_r}{T_F}<br><br>y\rfloor <br><br>,y+\lfloor <br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>y<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{T_r}{T_F}<br><br>y\rfloor <br><br>\right)$

for all x, y within ROI

where;

I refers to an image pixel

T_r refers to pixel capture delay time

T_F refers to given amount of time between image capture

x refers to the column index value

y refers to the row index value

□

x refers to the x component of a motion vector

□

y refers to the y component of a motion vector

denotes rounding towards a zero operator

ROI denotes the region of interest.