Image correction apparatus and image correction method for correcting image blur using a mobile vector

• US 7,982,772 B2
• Filed: 03/29/2005
• Issued: 07/19/2011
• Est. Priority Date: 03/30/2004
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kaku, Toshihiko
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): Quiett; Carramah J
• Application Number:
  US11/092,577
• Publication Number:
  US 20050225645A1
• Time in Patent Office: 2,303 Days
• Field of Search: 348/148, 348/149, 348/208.99, 348/208.1, 348/208.4, 348/208.5, 348/248, 348/352, 340/937
• US Class Current: 348/208.4
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

An image correction apparatus which corrects blurring generated by the traveling of a vehicle in the captured image, including: a camera which captures an image of front of a vehicle continuously; mobile vector calculation means of calculating a mobile vector in an image of each object based on a position of an object in each image continuously captured by the camera; and blurring correction means of correcting blurring of the image captured by the camera using the mobile vector calculated by the mobile vector calculation means.

Claims

1. An image correction apparatus, comprising:

a camera which continuously captures images having image content regarding a view from a vehicle;

mobile vector calculation means of calculating a mobile vector that corresponds to an object in the image content, based on a position of the object in images continuously captured by said camera; and

blurring correction means of defining, using the mobile vector calculated by said mobile vector calculation means, an image region for correcting image blurring and correcting blurring of the defined region of the image using the mobile vector, wherein

the blurring correction means defines the direction of the mobile vector as a direction for correcting the blurring and defines the width for correcting the blurring by calculating the width of the blurring by multiplying the mobile vector and an exposure time divided by a timing interval by which the mobile vector was calculated.

2. The image correction apparatus as claimed in claim 1, wherein said mobile vector calculation means calculates a vanishing point of an optical flow of the image captured by said camera, and corrects direction of the mobile vector assuming that the mobile vector of the object is radially oriented from the vanishing point.

3. The image correction apparatus as claimed in claim 2, wherein said mobile vector calculation means corrects length of the mobile vector based on a direction to which the object in images continuously captured by said camera has been corrected.

4. The image correction apparatus as claimed in claim 1, further comprising:

distance measurement means of measuring distance from the vehicle to the object;

speed measurement means of measuring speed of the vehicle; and

optical flow calculation means of calculating a calculatory optical flow of the object based on speed measured by said speed measurement means and distance to the object measured by said distance measurement means, wherein

said mobile vector calculation means calculates a position of the object in a second image by calculating a position of the object, which is included in a first image, in a second image based on the optical flow calculated by said optical flow calculation means, and searching the same object in a region centering on the position.

5. The image correction apparatus as claimed in claim 4, further comprising:

steering angle measurement means of measuring steering angle of the vehicle; and

swing speed calculation means of calculating swing speed of the vehicle based on speed measured by said speed measurement means and steering angle measured by said steering angle measurement means, wherein

said optical flow calculation means calculates a calculatory optical flow of the object further based on the swing speed calculated by said swing speed calculation means.

6. The image correction apparatus as claimed in claim 1, wherein said blurring correction means corrects blurring by adding an image, which is obtained by differentiating an image in each object to a direction of the mobile vector, to an original image.

7. The image correction apparatus as claimed in claim 6, further comprising a display unit which displays an image, of which blurring has been corrected by said blurring correction means.

8. The image correction apparatus as claimed in claim 1, further comprising:

exposure time control means of controlling exposure time of said camera; and

acceleration measurement means of measuring acceleration of the vehicle;

wherein

the larger the acceleration of the vehicle measured by said acceleration measurement means is, the shorter said exposure time control means controls exposure time of said camera.

9. The image correction apparatus as claimed in claim 8, wherein

said acceleration measurement means further measures change of swing speed of the vehicle, and

the larger the change of swing speed of the vehicle measured by said acceleration measurement means is, the shorter said exposure time control means controls exposure time of said camera.

10. The image correction apparatus as claimed in claim 8, wherein

said acceleration measurement means measures at least one of an amount of stepping on an accelerator of the vehicle, an amount of stepping on a brake, and a steering angle of a steering, and

the larger at least one variation of an amount of stepping on the accelerator, an amount of stepping on the brake, and a steering angle of the steering wheel is, the shorter said exposure time control means controls the exposure time of said camera.

11. The image correction apparatus as claimed in claim 1, wherein said blurring correction means corrects blurring in the region near right side and left side of each of the images continuously captured by said camera more accurately than correction of blurring in a region near upper side and lower side of each of the images.

12. The image correction apparatus as claimed in claim 11, wherein said blurring correction means omits to correct blurring in a region near upper side and lower side of each of the images continuously captured by said camera.

13. The image correction apparatus as claimed in claim 1, wherein

said mobile vector calculation means calculates a vanishing point of an optical flow of the image captured by said camera, and

said blurring correction means corrects blurring in a region near the vanishing point of the optical flow less accurately than the other regions.

14. The image correction apparatus as claimed in claim 1, wherein

said blurring correction means corrects blurring in a central region of each of the images continuously captured by said camera less accurately than the other regions.

27. The image correction apparatus as claimed in claim 1, wherein the blurring correction means defines the image region for correcting an image for each of all objects in the image content, thereby correcting blurring generated in each of the objects.

15. An image correction method, comprising:

a step of continuously capturing by a camera images having image content regarding a view from a vehicle;

a mobile vector calculation step of calculating a mobile vector that corresponds to an object in the image content based on a position of the object in images continuously captured by the camera;

a blurring correction step of defining, using the mobile vector calculated by said mobile vector calculation step, an image region for correcting image blurring and correcting blurring of the defined region of the image using the mobile vector, wherein

the blurring correction means defines the direction of the mobile vector as a direction for correcting the blurring and defines the width for correcting the blurring by calculating the width of the blurring by multiplying the mobile vector and an exposure time divided by a timing interval by which the mobile vector was calculated.

16. The image correction method as claimed in claim 15, wherein a vanishing point of an optical flow of the image captured by said camera is calculated in said mobile vector calculation step, and direction of the mobile vector is corrected assuming that the mobile vector of each object is radially oriented from the vanishing point.

17. The image correction method as claimed in claim 16, wherein length of the mobile vector is corrected in said mobile vector calculation step based on a direction to which the object in images continuously captured by said camera has been corrected.

18. The image correction method as claimed in claim 15, further comprising:

a distance measurement step of measuring distance from the vehicle to the object;

a speed measurement step of measuring speed of the vehicle; and

an optical flow calculation step of calculating a calculatory optical flow of the object based on speed measured in said speed measurement step and distance to the object measured in said distance measurement step, wherein

a position of the object in a second image is calculated in said mobile vector calculation step by calculating a position of the object, which is included in a first image, in a second image based on the optical flow calculated in said optical flow calculation step, and searching the same object in a region centering on the position.

19. The image correction method as claimed in claim 18, further comprising:

a steering angle measurement step of measuring steering angle of the vehicle; and

a swing speed calculation step of calculating swing speed of the vehicle based on speed measured in said speed measurement step and steering angle measured in said steering angle measurement step, wherein

a calculatory optical flow of the object is calculated in said optical flow calculation step further based on the swing speed calculated in said swing speed calculation step. corrected in said blurring correction step by adding an image, which is obtained by

20. The image correction method as claimed in claim 15, wherein blurring is corrected in said blurring correction step by adding an image, which is obtained by differentiating an image in each object to a direction of the mobile vector, to an original image.

21. The image correction method as claimed in claim 20, further comprising a displaying step of displaying an image, of which blurring has been corrected in said blurring correction step.

22. The image correction method as claimed in claim 15, wherein blurring is corrected in said blurring correction step in the region near right side and left side of each of the images continuously captured by said camera more accurately than correction of blurring in a region near upper side and lower side of each of the images.

23. The image correction method as claimed in claim 22, wherein blurring correction is omitted in said blurring correction step in a region near upper side and lower side of the images continuously captured by said camera.

24. The image correction method as claimed in claim 15, wherein

a vanishing point of an optical flow of the image captured by said camera is calculated in said mobile vector calculation step, and

blurring is corrected in said blurring correction step in a region near the vanishing point of the optical flow less accurately than the other regions.

25. The image correction method as claimed in claim 15, wherein

blurring is corrected in said blurring correction steps in a central region of the images continuously captured by the camera less accurately than the other regions.

26. The image correction method as claimed in claim 15, further comprising a steering angle measurement step of measuring steering angle of the vehicle.

28. The image correction apparatus as claimed in claim 15, wherein the blurring correction step of defining the image region for correcting an image is performed for each of all objects in the image content, thereby correcting blurring generated in each of the objects.