Moving object periphery image correction apparatus

• US 10,178,314 B2
• Filed: 03/08/2011
• Issued: 01/08/2019
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Morimoto, Takuji, Haga, Tetsuji
• Primary Examiner(s): Chio, Tat C
• Assistant Examiner(s): Lima, Fabio S
• Application Number:
  US13/990,944
• Publication Number:
  US 20130258047A1
• Time in Patent Office: 2,863 Days
• Field of Search: 348148
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

A reference line detection unit 12 extracts an edge from a distorted image taken by a fisheye camera to thereby determine a curved reference line. A distortion parameter calculation unit 13 calculates a distortion parameter such that the reference line is made linear. A distortion correction unit 14 corrects distortion of the distorted image by a distortion correction equation by using the distortion parameter.

Claims

1. A moving object periphery image correction apparatus comprising:

an interface to an imaging device from which a plurality of periphery images of a moving object are acquired from said imaging device;

a processor; and

a memory coupled to the processor, said memory storing instructions which, when executed by the processor, result in performance of a process including,

extracting an edge from each of the acquired periphery images, and determining a curved reference line from each extracted edge;

calculating a priority for each of the acquired periphery images;

selecting at least one of the acquired periphery images with the highest priority as a reference image;

calculating a distortion parameter for distortion correction of the reference image such that the determined reference line is made linear;

correcting a distortion of one or more acquired periphery images acquired with the use of the calculated distortion parameter, and outputting each corrected periphery image as a periphery image to be displayed; and

acquiring topographical information of the periphery of the moving object,

wherein the acquired topographical information is used by the processor to determine whether the periphery of the moving object is at an expressway, and to give a higher priority to each of the periphery images acquired from the imaging device when the periphery of the moving object is determined to be at an expressway thereby increasing the likelihood of selecting an acquired peripheral image with a limited number of potential edges as the reference image.

2. The moving object periphery image correction apparatus of claim 1, wherein the process further comprises calculating an optimum distortion parameter by using a plurality of distortion parameters calculated for each of a plurality of reference lines when a plurality of reference lines are determined from respective edges extracted from the periphery image,

wherein the distortion of the periphery image is corrected by using the calculated optimum distortion parameter.

3. The moving object periphery image correction apparatus of claim 2, wherein an average of the plurality of distortion parameters is calculated to determine the optimum distortion parameter.

4. The moving object periphery image correction apparatus of claim 2, wherein the process further comprises distortion-correcting each of reference images by using the plurality of distortion parameters calculated for each of the plurality of reference lines when the plurality of reference lines are detected, and calculating an evaluation value indicative of adequacy of the distortion correction on the basis of a result of the said correction,

wherein an optimum distortion parameter is calculated from among the plurality of distortion parameters based on the calculated evaluation value.

5. The moving object periphery image correction apparatus of claim 1, wherein the process determines a plurality of reference lines from respective edges extracted from the periphery image, the process further comprising causing a user to select a desired reference line from among the plurality of reference lines,

wherein the distortion parameter is calculated by using the reference line selected by the user.

6. The moving object periphery image correction apparatus of claim 1, wherein the process further comprises receiving a reference line drawn by a user,

wherein the distortion parameter is calculated by using the reference line drawn by the user.

7. The moving object periphery image correction apparatus of claim 1, wherein the process further comprises detecting a given characteristic object from the periphery image, and causing a user to select a desired characteristic object from among a plurality of characteristic objects,

wherein centering and distortion correction of the periphery image is performed such that the characteristic object selected by the user is displayed at the center of a screen of a display.

8. The moving object periphery image correction apparatus of claim 1, wherein the process further comprises receiving input from a user of a position of the periphery image to be displayed at the center of a screen of a display,

wherein centering and distortion correction of the periphery image is performed such that the position inputted from the user is positioned at the center of the screen of the display.

9. The moving object periphery image correction apparatus of claim 1, wherein the determined curved reference line extends horizontally.

10. The moving object periphery image correction apparatus of claim 1, wherein the imaging device is attachable and detachable to and from the moving object.

11. The moving object periphery image correction apparatus of claim 1, wherein the imaging device is a wide-angle imaging device.

12. The moving object periphery image correction apparatus of claim 1, wherein the moving object is a vehicle.

13. A moving object periphery image correction apparatus comprising:

an interface to an imaging device from which a plurality of periphery images of a moving object are acquired from said imaging device;

a processor; and

a memory coupled to the processor, said memory storing instructions which, when executed by the processor, result in performance of a process including,

extracting an edge from each of the acquired periphery images, and determining a curved reference line from each extracted edge;

calculating a priority for each of the acquired periphery images;

selecting at least one of the acquired periphery images with the highest priority as a reference image;

calculating a distortion parameter for distortion correction of the reference image such that the determined reference line is made linear;

correcting a distortion of one or more acquired periphery images acquired with the use of the calculated distortion parameter, and outputting each corrected periphery image as a periphery image to be displayed; and

acquiring topographical information of the periphery of the moving object, wherein the acquired topographical information is used by the processor to determine whether the periphery of the moving object is at an expressway, and to give a higher priority to each of the periphery images acquired from the imaging device when the periphery of the moving object is determined to be at an expressway thereby increasing the likelihood of selecting an acquired peripheral image with a limited number of potential edges as the reference image, and

wherein the distortion is corrected for the one or more acquired periphery images using the following equations;

Xu=Xd+k×

(Xd²+Yd²) and

Yu=Yd+k×

(Xd²+Yd²),

where (Xu, Yu) is a coordinate after a distortion correction, (Xd, Yd) is a coordinate before the distortion correction, and k is the distortion parameter.