Method and apparatus for generating a bird's-eye view image

• US 8,854,463 B2
• Filed: 10/29/2008
• Issued: 10/07/2014
• Est. Priority Date: 11/09/2007
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Imamura, Kosuke
• Primary Examiner(s): Dharia, Rupal
• Assistant Examiner(s): Ma, Wing
• Application Number:
  US12/260,749
• Publication Number:
  US 20090122140A1
• Time in Patent Office: 2,169 Days
• Field of Search: 348/148, 348/118, 382/106
• US Class Current: 348/148
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

An obstacle detecting unit detects whether an obstacle exists in an area surrounding a vehicle in the direction a camera installed on the vehicle is pointed. If the obstacle is determined to have a portion at a predetermined height above a road surface with which the vehicle may make initial contact when the vehicle moves toward the obstacle, a virtual projection surface is set to the height of the predicted contact portion and viewpoint conversion processing is performed so as to project pixels of an image captured by the camera onto the virtual projection plane. In this way, the height of the projection plane and the height of the predicted contact portion match each other and the distance between the predicted contact portion and the vehicle is accurately expressed on a bird'"'"'s-eye view image resulting from the viewpoint conversion.

Claims

1. A bird'"'"'s-eye view image generating apparatus method comprising:

inputting an image of an area surrounding a vehicle photographed by an image capture device installed on the vehicle;

detecting when an obstacle that exists in the area surrounding the vehicle in a direction the image-capture device is pointed, the obstacle having a predicted contact portion at a predetermined height above a surface of a road that is captured in the image of the surroundings of the vehicle, the predicted contact portion being a portion of the obstacle which will make initial contact with the vehicle when the vehicle moves toward the obstacle;

setting, when the obstacle is detected, a virtual projection plane at the height above the surface of the road where the predicted contact portion is detected, and performing viewpoint conversion processing in which pixels of the image are projected onto the virtual projection plane, thereby generating a bird'"'"'s-eye view image that is an image of the area surrounding the vehicle in the direction the image capture device is pointed as viewed from a virtual viewpoint above the vehicle that accurately reflects the distance between the vehicle and the predicted contact portion of the obstacle;

detecting an amount of movement of the vehicle; and

,

switching an operation mode from a road-surface projection mode to a virtual-plane projection mode when

     1) the obstacle is detected and

     2) it is determined that, based on the detected amount of movement of the vehicle, that the vehicle has moved by an amount corresponding to an allowable-movement distance stored in a distance table in which a height of the predicted contact portion of the obstacle and an allowable-movement distance for the vehicle are associated with each other;

wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image onto the virtual projection plane.

2. The bird'"'"'s-eye view image generating method according to claim 1, wherein the captured image is modified in the direction toward the vehicle based on a difference between the height of the virtual projection plane and the surface of the road to accurately reflect the distance between the vehicle and the predicted contact portion of the obstacle.

3. The bird'"'"'s-eye view image generating method according to claim 1, further comprising a plurality of viewpoint-conversion mapping tables associated with a plurality of virtual projection planes, each virtual projection plane having a different height from the road surface, and wherein the viewpoint conversion processing is performed using the viewpoint-conversion mapping table associated with the virtual projection plane that most closely corresponds to the height of the predicted contact portion of the detected obstacle.

4. The bird'"'"'s-eye view image generating method according to claim 3, further comprising detecting a distance from the vehicle to the predicted contact portion of the obstacle, and 1) performing a first viewpoint conversion processing on a first portion of the image by using a viewpoint-conversion mapping table that sets the road surface as a projection plane, the first portion of the image having a range between the vehicle and the predicted contact portion of the obstacle, and 2) performing a second viewpoint conversion processing on a second portion of the image by using the viewpoint-conversion mapping table associated with the virtual projection plane most closely corresponding to the height of the predicted contact portion of the detected obstacle, the second portion of the image having a range between the predicted contact portion of the obstacle and an outer edge of the image capturing the surrounding area disposed farthest from the vehicle.

5. The bird'"'"'s-eye view image generating method according to claim 1, wherein, when a plurality of obstacles are detected, distances from the vehicle to each of the respective predicted contact portions of the obstacles are detected, and the virtual projection plane is set to a position at a height of the predicted contact portion of the obstacle that is the closest in proximity to the vehicle.

6. The bird'"'"'s-eye view image generating method according to claim 1, including a plurality of image capture devices disposed at different locations on the vehicle such that said image capture devices capture images of the front, rear, left side, and right side areas surrounding the vehicle.

7. The bird'"'"'s-eye view image generating method according to claim 6, wherein the viewpoint conversion processing for images captured by each image capture device is performed when an obstacle is detected in the respective images.

8. A bird'"'"'s-eye view image generating method comprising:

inputting an image of an area surrounding a vehicle captured by an image capture device installed on the vehicle;

detecting when an obstacle exists in the area surrounding the vehicle in a direction of the image-capture-device is pointed, the obstacle having a predicted contact portion at a predetermined height above a road surface, wherein the predicted contact portion is a portion with which the vehicle may make contact when the vehicle moves toward the object;

setting, when the obstacle is detected, a virtual projection plane to a height where the predicted contact portion of the obstacle exists;

performing viewpoint conversion processing so as to project pixels of the input image onto the virtual projection plane, thereby generating a bird'"'"'s-eye view image that is an image of the area surrounding the vehicle as viewed from a virtual viewpoint above the vehicle in a direction of the image capture device; and

,

detecting an amount of movement of the vehicle after the obstacle is detected, and if the vehicle has been detected to have moved by an amount corresponding to an allowable-movement distance stored by a table storage unit in association with a height of the predicted contact portion, an operation mode is switched from a road-surface projection mode to a virtual-plane projection mode to perform the viewpoint conversion processing, wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto the virtual projection plane.

9. The bird'"'"'s-eye view image generating method according to claim 8, wherein a distance from the vehicle to the predicted contact portion of the obstacle is detected, and wherein the viewpoint conversion processing is performed by performing a first viewpoint conversion processing on a first portion of the image having a range between the vehicle and the detected distance from the vehicle to the predicted contact portion of the obstacle, the first viewpoint conversion processing using the road surface as a projection plane, and performing a second viewpoint conversion processing on a second portion of the image having a range between the predicted contact portion of the obstacle and the outer edge of the image disposed farthest from the vehicle, the second viewpoint conversion processing using the virtual projection plane as the projection plane.

10. The bird'"'"'s-eye view image generating method according to claim 8, wherein, when the obstacle is detected, an operation mode is switched from a road-surface projection mode to a virtual-plane projection mode, wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image captured by the image capture device onto the virtual projection plane.

11. A bird'"'"'s-eye view image generating method comprising:

inputting an image of an area surrounding a vehicle photographed by an image capture device installed on the vehicle, the vehicle comprising a protruding portion that extends away from the vehicle as compared to a non-protruding portion of the vehicle;

detecting an obstacle that exists in the area surrounding the vehicle in a direction the image-capture device is pointed, the obstacle having a predicted contact portion at a predetermined height above a surface of a road captured in the image of the surroundings of the vehicle, the predicted contact portion being a portion of the obstacle which will make initial contact with the vehicle when the vehicle moves toward the obstacle;

setting, when the obstacle is detected, a virtual projection plane at the height above the surface of the road where the predicted contact portion is detected, and performing viewpoint conversion processing;

detecting an amount of movement of the vehicle; and

switching an operation mode from a road-surface projection mode to a virtual-plane projection mode

     1) the obstacle is detected and

     2) it is determined that, based on the detected amount of movement of the vehicle, that the vehicle has moved by an amount corresponding to an allowable-movement distance stored by a table storage unit that stores data in a distance table in which a height of the predicted contact portion of the obstacle and the allowable-movement distance for the vehicle are associated with each other, the allowable-movement distance for the vehicle corresponding to the distance the protruding portion extends away from the non-protruding portion of the vehicle at the height of the predicted contact portion of the obstacle,

wherein in the road-surface projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image onto a projection plane set at a height of the road surface in the vehicle surroundings, and in the virtual-plane projection mode, the viewpoint conversion processing is performed so as to project the pixels of the image onto the virtual projection plane, thereby altering the image to accurately reflect the distance between the vehicle and the predicted contact portion of the obstacle.

12. The bird'"'"'s-eye view image generating method according to claim 11, wherein the operation mode switches from the road-surface projection mode to the virtual-plane projection mode when the vehicle has moved by an amount based on the allowable-movement distance after the obstacle is detected.

13. The bird'"'"'s-eye view image generating method according to claim 11, wherein the captured image is modified in the direction toward the vehicle based on a difference between the height of the virtual projection plane and the surface of the road to accurately reflect the distance between the vehicle and the predicted contact portion of the obstacle.

14. The bird'"'"'s-eye view image generating method according to claim 11, wherein, when a plurality of obstacles are detected, distances from the vehicle to each of the respective predicted contact portions of the obstacles are detected, the virtual projection plane is set to a position at a height of the predicted contact portion of the obstacle that is the closest in proximity to the vehicle.

15. The bird'"'"'s-eye view image generating method according to claim 11, wherein the image capture device includes a plurality of image capture devices disposed at different locations on the vehicle such that the image capture devices capture images of the front, rear, left side, and right side areas surrounding the vehicle, and the viewpoint conversion processing is performed for images captured by each image capture device when the obstacle is detected in the images.

16. The bird'"'"'s-eye view image generating method according to claim 11, further comprising a plurality of viewpoint-conversion mapping tables associated with a plurality of virtual projection planes, each virtual projection plane having a different height from the road surface, and wherein the viewpoint conversion processing is performed using the viewpoint-conversion mapping table associated with the virtual projection plane that most closely corresponds to the height of the predicted contact portion of the detected obstacle.

17. The bird'"'"'s-eye view image generating method according to claim 16, wherein a distance is detected from the vehicle to the predicted contact portion of the obstacle, and 1) a first viewpoint conversion processing is performed on a first portion of the image captured by the image capture device by using a viewpoint-conversion mapping table that sets the road surface as a projection plane, the first portion of the image having a range between the vehicle and the predicted contact portion of the obstacle, and 2) a second viewpoint conversion processing is performed on a second portion of the image captured by the image capture device by using the viewpoint-conversion mapping table associated with the virtual projection plane most closely corresponding to the height of the predicted contact portion of the obstacle detected by the obstacle detecting unit, the second portion of the image having a range between the predicted contact portion of the obstacle and an outer edge of the image capturing the surrounding area disposed farthest from the vehicle.