METHOD AND APPARATUS FOR GENERATING A BIRD'S-EYE VIEW IMAGE

• US 20090122140A1
• Filed: 10/29/2008
• Published: 05/14/2009
• Est. Priority Date: 11/09/2007
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Imamura, Kosuke
• Granted Patent: US 8,854,463 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

An obstacle detecting unit detects whether an obstacle exists in an area surrounding a vehicle in the direction a camera installed on the vehicle is pointed. If the obstacle is determined to have a portion at a predetermined height above a road surface with which the vehicle may make initial contact when the vehicle moves toward the obstacle, a virtual projection surface is set to the height of the predicted contact portion and viewpoint conversion processing is performed so as to project pixels of an image captured by the camera onto the virtual projection plane. In this way, the height of the projection plane and the height of the predicted contact portion match each other and the distance between the predicted contact portion and the vehicle is accurately expressed on a bird'"'"'s-eye view image resulting from the viewpoint conversion.

Claims

1. A bird'"'"'s-eye view image generating apparatus comprising:

an image input unit for inputting an image of an area surrounding a vehicle photographed by an image capture device installed on the vehicle;

an obstacle detecting unit for detecting an obstacle that exists in the area surrounding the vehicle in a direction the image-capture is pointed, the obstacle having a predicted contact portion at a predetermined height above a surface of a road that is captured in the image of the surroundings of the vehicle, the predicted contact portion being a portion of the obstacle which will make initial contact with the vehicle when the vehicle moves toward the obstacle; and

a bird'"'"'s-eye view image generating unit for setting, when the obstacle detecting unit detects the obstacle, a virtual projection plane at the height above the surface of the road where the predicted contact portion is detected, and performing viewpoint conversion processing in which pixels of the image input by the image input unit are projected onto the virtual projection plane, thereby generating a bird'"'"'s-eye view image that is an image of the area surrounding the vehicle in the direction the image capture device is pointed as viewed from a virtual viewpoint above the vehicle that accurately reflects the distance between the vehicle and the predicted contact portion of the obstacle.

2. The bird'"'"'s-eye view image generating apparatus according to claim 1, wherein the captured image is modified in the direction toward the vehicle based on a difference between the height of the virtual projection plane and the surface of the road to accurately reflect the distance between the vehicle and the predicted contact portion of the obstacle.

3. The bird'"'"'s-eye view image generating apparatus according to claim 1, wherein the bird'"'"'s-eye view image generating unit includes a plurality of viewpoint-conversion mapping tables associated with a plurality of virtual projection planes, each virtual projection plane having a different height from the road surface, and wherein the bird'"'"'s-eye view image generating unit performs the viewpoint conversion processing using the viewpoint-conversion mapping table associated with the virtual projection plane that most closely corresponds to the height of the predicted contact portion of the obstacle detected by the obstacle detecting unit.

4. The bird'"'"'s-eye view image generating apparatus according to claim 3, wherein the obstacle detecting unit detects a distance from the vehicle to the predicted contact portion of the obstacle, and the bird'"'"'s-eye view image generating unit 1) performs a first viewpoint conversion processing on a first portion of the image captured by the image capture device by using a viewpoint-conversion mapping table that sets the road surface as a projection plane, the first portion of the image having a range between the vehicle and the predicted contact portion of the obstacle, and 2) performs a second viewpoint conversion processing on a second portion of the image captured by the image capture device by using the viewpoint-conversion mapping table associated with the virtual projection plane most closely corresponding to the height of the predicted contact portion of the obstacle detected by the obstacle detecting unit, the second portion of the image having a range between the predicted contact portion of the obstacle and an outer edge of the image capturing the surrounding area disposed farthest from the vehicle.

5. The bird'"'"'s-eye view image generating apparatus according to claim 1, further comprising a mode control unit for switching an operation mode from a road-surface projection mode of the bird'"'"'s-eye view image generating unit to a virtual-plane projection mode when the obstacle detecting unit detects the obstacle having the predicted contact portion at the height above the surface of the road, wherein in the road-surface projection mode, the bird'"'"'s-eye view image generating unit performs the viewpoint conversion processing so as to project the pixels of the image input by the image input unit onto a projection plane set at a height of the road surface in the area surrounding the vehicle, and in the virtual-plane projection mode, the bird'"'"'s-eye view image generating unit performs the viewpoint conversion processing so as to project the pixels of the image input by the image input unit onto the virtual projection plane.

6. The bird'"'"'s-eye view image generating apparatus according to claim 1, further comprising:

a vehicle movement-amount detecting unit for detecting an amount of movement of the vehicle;

a table storage unit that stores data in a distance table in which a height of the predicted contact portion of the obstacle and an allowable-movement distance for the vehicle are associated with each other; and

a mode control unit for switching an operation mode from a road-surface projection mode to a virtual-plane projection mode when the obstacle detecting unit

     1) detects the obstacle and

     2) determines, based on the detected amount of movement of the vehicle, that the vehicle has moved by an amount corresponding to the allowable-movement distance stored by the table storage unit in association with the predicted contact portion detected by the obstacle detecting unit,

wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image input by the image input unit onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image input by the image input unit onto the virtual projection plane.

7. The bird'"'"'s-eye view image generating apparatus according to claim 1, wherein, when the obstacle detecting unit detects a plurality of obstacles, the obstacle detecting unit also detects distances from the vehicle to each of the respective predicted contact portions of the obstacles, and the bird'"'"'s-eye view image generating unit sets the virtual projection plane to a position at a height of the predicted contact portion of the obstacle that is the closest in proximity to the vehicle.

8. The bird'"'"'s-eye view image generating apparatus according to claim 1, wherein the image capture device includes a plurality of image capture devices disposed at different locations on the vehicle such that said image capture devices capture images of the front, rear, left side, and right side areas surrounding the vehicle.

9. The bird'"'"'s-eye view image generating apparatus according to claim 8, wherein the bird'"'"'s-eye view image generating unit performs the viewpoint conversion processing for images captured by each image capture device when the obstacle detecting unit detects an obstacle in the respective images.

11. The bird'"'"'s-eye view image generating method according to claim 7, wherein a distance from the vehicle to the predicted contact portion of the obstacle is detected, and wherein the viewpoint conversion processing is performed by performing a first viewpoint conversion processing on a first portion of the image having a range between the vehicle and the detected distance from the vehicle to the predicted contact portion of the obstacle, the first viewpoint conversion processing using the road surface as a projection plane, and performing a second viewpoint conversion processing on a second portion of the image having a range between the predicted contact portion of the obstacle and the outer edge of the image disposed farthest from the vehicle, the second viewpoint conversion processing using the virtual projection plane as the projection plane.

12. The bird'"'"'s-eye view image generating method according to claim 7, wherein, when the obstacle is detected, an operation mode is switched from a road-surface projection mode to a virtual-plane projection mode, wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto the virtual projection plane.

13. The bird'"'"'s-eye view image generating method according to claim 7, further comprising:

detecting an amount of movement of the vehicle after the obstacle is detected, and if the vehicle has been detected to have moved by an amount corresponding to an allowable-movement distance stored by a table storage unit in association with a height of the predicted contact portion, an operation mode is switched from a road-surface projection mode to a virtual-plane projection mode to perform the viewpoint conversion processing, wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto the virtual projection plane.

10. A bird'"'"'s-eye view image generating method comprising:

inputting an image of an area surrounding a vehicle captured by an image capture device installed on the vehicle;

detecting whether an obstacle exists in the area surrounding the vehicle in a direction of the image-capture-device is pointed, the obstacle having a predicted contact portion at a predetermined height above a road surface, wherein the predicted contact portion is a portion with which the vehicle may make contact when the vehicle moves toward the object; and

setting, when the obstacle is detected, a virtual projection plane to a height where the predicted contact portion of the obstacle exists; and

performing viewpoint conversion processing so as to project pixels of the input image onto the virtual projection plane, thereby generating a bird'"'"'s-eye view image that is an image of the area surrounding the vehicle as viewed from a virtual viewpoint above the vehicle in a direction of the image capture device.

14. A bird'"'"'s-eye view image generating apparatus comprising:

an image input unit for inputting an image of an area surrounding a vehicle photographed by an image capture device installed on the vehicle, the vehicle comprising a protruding portion that extends away from the vehicle as compared to a non-protruding portion of the vehicle;

an obstacle detecting unit for detecting an obstacle that exists in the area surrounding the vehicle in a direction the image-capture is pointed, the obstacle having a predicted contact portion at a predetermined height above a surface of a road captured in the image of the surroundings of the vehicle, the predicted contact portion being a portion of the obstacle which will make initial contact with the vehicle when the vehicle moves toward the obstacle; and

a bird'"'"'s-eye view image generating unit for setting, when the obstacle detecting unit detects the obstacle, a virtual projection plane at the height above the surface of the road where the predicted contact portion is detected, and performing viewpoint conversion processing;

a vehicle movement-amount detecting unit for detecting an amount of movement of the vehicle;

a table storage unit that stores data in a distance table in which a height of the predicted contact portion of the obstacle and an allowable-movement distance for the vehicle are associated with each other, the allowable-movement distance for the vehicle substantially corresponding to the distance the protruding portion extends away from the non-protruding portion of the vehicle at the height of the predicted contact portion of the obstacle; and

a mode control unit for switching an operation mode from a road-surface projection mode to a virtual-plane projection mode when the obstacle detecting unit

     1) detects the obstacle and

     2) determines, based on the detected amount of movement of the vehicle, that the vehicle has moved by an amount corresponding to the allowable-movement distance stored by the table storage unit in association with the predicted contact portion detected by the obstacle detecting unit,

wherein in the road-surface projection mode, the bird'"'"'s-eye view image generating unit performs viewpoint conversion processing so as to project the pixels of the image input by the image input unit onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image input by the image input unit onto the virtual projection plane, thereby altering the image to accurately reflect the distance between the vehicle and the predicted contact portion of the obstacle.

15. The bird'"'"'s-eye view image generating apparatus according to claim 14, wherein the mode control unit switches the operation mode from the road-surface projection mode to the virtual-plane projection mode when the vehicle has moved by an amount based on the allowable-movement distance after the obstacle is detected.

16. The bird'"'"'s-eye view image generating apparatus according to claim 14, wherein the captured image is modified in the direction toward the vehicle based on a difference between the height of the virtual projection plane and the surface of the road to accurately reflect the distance between the vehicle and the predicted contact portion of the obstacle.

17. The bird'"'"'s-eye view image generating apparatus according to claim 14, wherein, when the obstacle detecting unit detects a plurality of obstacles, the obstacle detecting unit also detects distances from the vehicle to each of the respective predicted contact portions of the obstacles, and the bird'"'"'s-eye view image generating unit sets the virtual projection plane to a position at a height of the predicted contact portion of the obstacle that is the closest in proximity to the vehicle.

18. The bird'"'"'s-eye view image generating apparatus according to claim 14, wherein the image capture device includes a plurality of image capture devices disposed at different locations on the vehicle such that the image capture devices capture images of the front, rear, left side, and right side areas surrounding the vehicle, and the image generating unit performs the viewpoint conversion processing for images captured by each image capture device when the obstacle detecting unit detects an obstacle in the images.

19. The bird'"'"'s-eye view image generating apparatus according to claim 14, wherein the bird'"'"'s-eye view image generating unit includes a plurality of viewpoint-conversion mapping tables associated with a plurality of virtual projection planes, each virtual projection plane having a different height from the road surface, and wherein the bird'"'"'s-eye view image generating unit performs the viewpoint conversion processing using the viewpoint-conversion mapping table associated with the virtual projection plane that most closely corresponds to the height of the predicted contact portion of the obstacle detected by the obstacle detecting unit.

20. The bird'"'"'s-eye view image generating apparatus according to claim 19, wherein the obstacle detecting unit detects a distance from the vehicle to the predicted contact portion of the obstacle, and the bird'"'"'s-eye view image generating unit 1) performs a first viewpoint conversion processing on a first portion of the image captured by the image capture device by using a viewpoint-conversion mapping table that sets the road surface as a projection plane, the first portion of the image having a range between the vehicle and the predicted contact portion of the obstacle, and 2) performs a second viewpoint conversion processing on a second portion of the image captured by the image capture device by using the viewpoint-conversion mapping table associated with the virtual projection plane most closely corresponding to the height of the predicted contact portion of the obstacle detected by the obstacle detecting unit, the second portion of the image having a range between the predicted contact portion of the obstacle and an outer edge of the image capturing the surrounding area disposed farthest from the vehicle.