Monitoring apparatus and method of displaying bird's-eye view image

• US 20060029255A1
• Filed: 02/15/2005
• Published: 02/09/2006
• Est. Priority Date: 08/05/2004
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Ozaki, Nobuyuki
• Granted Patent: US 7,511,734 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/006 Geometric correction correction of chromatic aberrations adapted to be used in an image pickup device containing an electronic image sensor H04N23/10; detecting, correcting, reducing or removing artefacts resulting only from the lens unit, e.g. flare, shading, vignetting or "cos4" H04N25/61
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

A monitoring apparatus for providing the driver with the image of blind-spot areas of a moving body. The monitoring apparatus includes: at least one of fish-eye camera mounted on a moving body, to image fish-eye image around the moving body; a dewarping unit configured to convert at least part of the fish-eye image obtained by the fish-eye camera into a two-dimensional image using a dewarping parameter; a conversion parameter calculation unit configured to calculate a conversion parameter for obtaining a bird'"'"'s-eye view image from the two-dimensional image, showing an image from a hypothetical position looking down on the moving body; a bird'"'"'s-eye view image creation unit configured to create a bird'"'"'s-eye view image by sequentially fetching position data corresponding to each pixel of the bird'"'"'s-eye view image using the conversion parameter and the dewarping conversion parameter; and a display device to display the bird'"'"'s-eye view image created by the image creation unit.

Claims

1. A monitoring apparatus comprising:

at least one of fish-eye camera mounted on a moving body, to image a fish-eye image around the moving body;

a dewarping unit configured to convert at least part of the fish-eye image obtained by the fish-eye camera into a two-dimensional image using a dewarping parameter;

a conversion parameter calculation unit configured to calculate a conversion parameter for obtaining a bird'"'"'s-eye view image from the two-dimensional image, showing an image from a hypothetical position looking down on the moving body;

a bird'"'"'s-eye view image creation unit configured to create a bird'"'"'s-eye view image by sequentially fetching position data corresponding to each pixel of the bird'"'"'s-eye view image using the conversion parameter and the dewarping conversion parameter; and

a display device to display the bird'"'"'s-eye view image created by the image creation unit.

2. The apparatus according to claim 1,

wherein the conversion parameter calculation unit comprises a specifying unit configured to selectively specify a plurality of pixels in the dewarped two-dimensional image.

3. The apparatus according to claim 1,

wherein a plurality of fish-eye camera are used and a plurality of conversion parameter calculation are provided, each corresponding to each fish-eye camera.

4. The apparatus according to claim 3, further comprising an image-synthesis unit configured to synthesize a plurality of bird'"'"'s-eye view images obtained corresponding to respective fish-eye cameras into one image.

5. The apparatus according to claim 1, further comprising an image-recognition device configured to recognize an obstacle in the bird'"'"'s-eye view image.

6. The apparatus according to claim 5,

wherein the image-recognition device recognizes an obstacle based a moving vector in the bird'"'"'s-eye view image.

7. A surveillance apparatus, comprising:

a plurality of fish-eye cameras mounted on a vehicle imaging fish-eye images around the vehicle;

a dewarping unit configured to dewarp at least parts of the fish-eye images into two-dimensional images using dewarping parameters;

a conversion parameter calculation unit configured to calculate conversion parameters for converting the two-dimensional images into bird'"'"'s-eye view images, each viewing from a hypothetical position looking down on the vehicle;

a bird'"'"'s-eye view image generating unit configured to generate bird'"'"'s-eye view images by sequentially calculating position data of a pixel in the fish-eye image corresponding to each pixel of the bird'"'"'s-eye view image using the conversion parameter obtained by the conversion parameter calculation unit and the dewarping parameter;

an image superposing unit configured to superpose the bird'"'"'s-eye view images into an superposed image; and

a display device to display the superposed image generated by the image superposing unit.

8. The apparatus according to claim 7,

wherein the bird'"'"'s-eye view image generating unit uses a GPU function of a graphics processor unit.

9. A method of displaying a bird'"'"'s-eye view image, comprising steps of:

storing a fish-eye image into a first memory showing surroundings of a movable body obtained using a fish-eye camera attached to the movable body;

performing calibration processing on part of the fish-eye image in advance and obtaining a group of image-conversion parameters to be used for converting the fish-eye image into a bird'"'"'s-eye view image for overlooking the movable body to store the parameters into a second memory; and

sequentially calculating an image-data position of the fish-eye image corresponding to each pixel data of the bird'"'"'s-eye view image using the group of image-conversion parameters of the second memory and sequentially fetching the image data from the first memory as pixel data of the bird'"'"'s-eye view image to supply to a display device.

10. The method according to claim 9,

wherein the performing step comprises:

a converting step for converting part of the first fish-eye image into a dewarped two-dimensional image using a dewarping parameters; and

a calculating step for calculating conversion parameters which is used to convert the dewarped two-dimensional image into the bird'"'"'s-eye view image,

wherein the dewarping parameter and the conversion parameters are used as the group of image-conversion parameters.

11. A method of converting an image for converting a fish-eye image obtained by a fish-eye camera mounted on a moving body into a bird'"'"'s-eye view image viewed from a hypothetically overlooking position of the moving body, comprising:

obtaining a dewarping parameter for converting the fish-eye image into a dewarped image and a conversion parameter for converting the dewarped image into the bird'"'"'s-eye view, thereby storing parameters into a parameter file; and

calculating a pixel position in the fish-eye image corresponding to each pixel in the bird'"'"'s-eye view image using the dewarping parameter and the conversion parameter stored in the parameter file and sequentially fetching the pixel data of the fish-eye image and output as pixel data of the bird'"'"'s-eye view image.

12. The method according-to claim 11,

wherein the images are moving images.