Image display method and image display apparatus

• US 8,130,271 B2
• Filed: 10/30/2008
• Issued: 03/06/2012
• Est. Priority Date: 12/20/2007
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Imamura, Kosuke
• Primary Examiner(s): Eng, David
• Application Number:
  US12/261,418
• Publication Number:
  US 20090160940A1
• Time in Patent Office: 1,223 Days
• Field of Search: 348/148, 348/159, 345/629
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image display method for displaying a combined image by capturing images at areas surrounding a vehicle includes a) calculating a plurality of displacement vectors for a plurality of points taken on images captured by a plurality of cameras upon the application of a load that changes the attitude of a camera on the vehicle, and b) determining each image borderline between two adjacent images such that a difference in the directions of displacement vectors in the neighborhood of each image borderline between two adjacent images does not exceed a predetermined angle.

Claims

1. An image display method for displaying a combined image of a vehicle, the method comprising:

capturing images corresponding to areas surrounding a vehicle using a corresponding plurality of cameras fixedly mounted to the vehicle;

calculating a plurality of displacement vectors for a plurality of points on each of the images captured by the cameras upon the application of a load to the vehicle sufficient to change an attitude of any of the plurality of cameras on the vehicle;

determining image borderlines between two adjacent images such that a difference in the directions of displacement vectors in the neighborhood of each image borderline does not exceed a predetermined angle;

combining the captured images into a virtual top-view image of the vehicle; and

displaying the virtual top-view image on a display device.

2. The image display method according to claim 1, further comprising clipping an image portion that is demarcated by two image borderlines, out of the image captured by each of the plurality of cameras so as to generate the combined image.

3. The image display method according to claim 1, wherein determining each image borderline between two adjacent images includes

determining a point such that a difference in directions of displacement vectors between two adjacent images does not exceed a predetermined angle;

and

determining each image borderline such that the image borderline passes through the point and extends in the same direction as that of the displacement vector.

4. The image display method according to claim 1, wherein determining each image borderline between two adjacent images includes

finding a plurality of points such that differences in the directions of displacement vectors between two adjacent images, which are captured by two adjacent cameras, do not exceed a predetermined angle;

and

determining each image borderline such that the image borderline passes through the plurality of found points.

5. The image display method according to claim 1, further comprising:

pre-storing correspondences between camera attitudes and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

measuring an actual camera attitude; and

clipping an image area portion in accordance with a set of image clip positions that corresponds to the measured actual camera attitude for each camera to generate the combined image.

6. The image display method according to claim 1, further comprising:

pre-storing correspondences between the number of occupants of a vehicle and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

determining an actual number of occupants of the vehicle; and

clipping an image area portion in accordance with a set of image clip positions that corresponds to the actual number of the occupants of the vehicle to generate the combined image.

7. The image display method according to claim 1, further comprising:

pre-storing correspondences between a total load weight of a vehicle and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

measuring an actual total load weight of the vehicle; and

clipping an image area portion in accordance with a set of image clip positions that corresponds to the actual total load weight of the vehicle for each camera to generate the combined image.

8. The image display method according to claim 1, further comprising:

pre-storing correspondences between a height of a vehicle and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

measuring an actual height of the vehicle; and

clipping an image area portion in accordance with a set of image clip positions that corresponds to the actual height of the vehicle for each camera to generate the combined image.

9. The image display method according to claim 8, wherein

each of the plurality of cameras is a fish-eye camera;

and

the set of image clip positions is used to specify an image clip area at which an image portion should be clipped out of a distortion-corrected fish-eye image, which is an image captured by the fish-eye camera and then corrected for distortion.

10. The image display method according to claim 7, wherein the total weight of the vehicle is the aggregate weight of occupants and luggage excluding the weight of the vehicle itself.

11. An image display apparatus for displaying a combined image of a vehicle, the apparatus comprising:

a plurality of cameras fixedly mounted to the vehicle;

a displacement vector calculating section configured to calculate a plurality of displacement vectors for a plurality of points on each of the images captured by the cameras upon the application of a load sufficient to change an attitude of any of the cameras on the vehicle;

an image borderline determining section configured to determine image borderlines between two adjacent images such that a difference in the directions of displacement vectors in the neighborhood of each image borderline does not exceed a predetermined angle;

an image combiner configured to combine the captured images into a virtual top-view image of the vehicle; and

a display device configured to display the virtual top-view image.

12. The image display apparatus according to claim 11, further comprising:

an image clip position determining section configured to determine positions at which an image portion should be clipped out of the image captured by each of the cameras on the basis of two image borderlines; and

a combined-image generating section configured to generate the combined image by combining the clipped images with each other.

13. The image display apparatus according to claim 11, wherein

the image borderline determining section determines a point such that a difference in the directions of displacement vectors between two adjacent images does not exceed a predetermined angle;

and

the image borderline determining section determines each image borderline such that the image borderline passes through the point and extends in the same direction as that of the displacement vector.

14. The image display apparatus according to claim 11, wherein

the image borderline determining section determines a plurality of points such that differences in the directions of displacement vectors between two adjacent images, which are captured by two adjacent cameras, do not exceed a predetermined angle;

and

the image borderline determining section determines each image borderline such that the image borderline passes through the plurality of determined points.

15. The image display apparatus according to claim 11, further comprising:

a table configured to pre-store correspondences between camera attitudes and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

a measuring section configured to measure an actual camera attitude;

an image clipping section configured to clip an image area portion based on a set of image clip positions that corresponds to the actual camera attitude for each camera; and

a combined-image generating section configured to generate the combined image by combining the clipped images with each other.

16. The image display apparatus according to claim 11, further comprising:

a table configured to pre-store correspondences between the number of occupants of a vehicle and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

an occupant-number detecting section configured to determine an actual number of the occupants of the vehicle;

an image clipping section configured to clip an image area portion in accordance with a set of image clip positions that corresponds to the number of the occupants of the vehicle for each camera; and

a combined-image generating section configured to generate the combined image by combining the clipped images with each other.

17. The image display apparatus according to claim 11, further comprising:

a table configured to pre-store correspondences between a total load weight of a vehicle and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

a load weight measuring section configured to measure an actual total load weight of the vehicle;

an image clipping section configured to clip that clips an image area portion in accordance with a set of image clip positions that corresponds to the actual total load weight of the vehicle for each camera; and

a combined-image generating section configured to generate the combined image by combining the clipped images with each other.

18. The image display apparatus according to claim 11, further comprising:

a table configured to pre-store correspondences between a height of a vehicle and sets of image clip positions, which are determined on the basis of the image borderlines, for each camera;

a vehicle-height measuring section configured to measure the actual height of the vehicle;

an image clipping section configured to clip an image area portion in accordance with a set of image clip positions that corresponds to the actual height of the vehicle for each camera; and

a combined-image generating section configured to generate the combined image by combining the clipped images with each other.

19. The image display apparatus according to claim 18, further comprising:

a plurality of fish-eye cameras; and

a distortion correcting section configured to correct distortion of a fish-eye image captured by each of the plurality of fish-eye cameras;

wherein the set of image clip positions among the sets of image clip positions, which are stored in the table, is used to specify an image clip area at which an image portion should be clipped out of a distortion-corrected fish-eye image.

20. The image display apparatus according to claim 17, wherein the total weight of the vehicle is the aggregate weight of occupants and luggage excluding the weight of the vehicle itself.