BLIND SPOT DISPLAY APPARATUS

• US 20100315215A1
• Filed: 03/27/2009
• Published: 12/16/2010
• Est. Priority Date: 03/27/2008
• Status: Active Grant

• Current Assignee: Piece Future Pte., Ltd.
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Yuda, Masato
• Granted Patent: US 8,395,490 B2
• Time in Patent Office: Days
• US Class Current: 340/435
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Provided is a blind spot display apparatus that presents a situation at a blind spot occurring at an intersection from a driver'"'"'s viewpoint, in a method of presenting the situation to a driver. For generating a blind spot situation image without any uncomfortable feeling, the driver'"'"'s viewpoint and a depth of the blind spot are calculated with higher precision and lower cost. The blind spot display apparatus includes: an omni-directional image storage unit (102) storing omni-directional images obtained at different positions; a blind spot model generating unit (109) calculating a position of the object based on each image of the object (in the blind spot) included in the omni-directional images; an own-vehicle model generating unit (110) calculating a position of a vehicle based on images of the vehicle included in the omni-directional images; an own-vehicle viewpoint coordinate transforming unit (107) transforming the calculated position of the object into display coordinates relative to the calculated position of the vehicle; and a display unit (108) that performs display.

Claims

1. A blind spot display apparatus that displays an image of an object in a blind spot of a vehicle, said apparatus comprising:

a storage unit configured to store omni-directional images which have been obtained from cameras at different positions and each of which includes the image of the object in the blind spot and an image of the vehicle;

an object model generating unit configured to calculate a coordinate position of the object seen from a predetermined camera among the cameras, using a difference between coordinate positions of the images of the object that are included in the omni-directional images;

a vehicle model generating unit configured to calculate a coordinate position of the vehicle seen from the predetermined camera, using a difference between coordinate positions of the images of the vehicle that are included in the omni-directional images; and

a display unit configured to display the image of the object as seen from the vehicle on a display, using the calculated coordinate positions of the object and of the vehicle, the image of the object being included in common in each of the omni-directional images.

2. The blind spot display apparatus according to claim 1,

wherein said vehicle model generating unit is configured to calculate the coordinate position of the vehicle seen from the predetermined camera, using a difference between a first point and a second point the first point indicating the vehicle whose image is included in a predetermined one of the omni-directional images, and the second point (i) indicating the vehicle whose image is included in an other one of the omni-directional images that is different from the predetermined one of the omni-directional images and (ii) corresponding to the first point.

3. The blind spot display apparatus according to claim 2,

wherein said object model generating unit is configured to calculate the coordinate position of the object seen from the predetermined camera, using a difference between a first point and a second point, the first point indicating the object whose image is included in a predetermined one of the omni-directional images, and the second point (i) indicating the object whose image is included in an other one of the omni-directional images that is different from the predetermined one of the omni-directional images and (ii) corresponding to the first point,

the omni-directional images are images captured by the cameras installed at different positions at an intersection, the positions being adjacent to each other,

the omni-directional images are omni-directional images with respect to the cameras that obtain the omni-directional images, respectively, and

said blind spot display apparatus further comprises a selecting unit configured to select two omni-directional images each of which is a captured image of the vehicle and the object, from the omni-directional images, and to provide the selected two omni-directional images to be used by said object model generating unit and said vehicle model generating unit.

4. The blind spot display apparatus according to claim 1,

wherein said display unit is configured to display a predetermined pixel color corresponding to a value of a depth, from the coordinate position of the vehicle calculated by said vehicle model generating unit to the coordinate position of the object calculated by said object model generating unit, as a color of the object on the display.

5. The blind spot display apparatus according to claim 3,

wherein said object model generating unit includes:

a left blind spot corresponding point search unit configured to search for two corresponding points on images of an object in a left blind spot to a left of a traveling direction of the vehicle, and to determine the obtained two points as object corresponding points; and

a right blind spot corresponding point search unit configured to search for two corresponding points on images of an object in a right blind spot to a right of the traveling direction of the vehicle, and to determine the obtained two points as object corresponding points.

6. The blind spot display apparatus according to claim 3,

wherein said selecting unit is configured to segment a corresponding one of the omni-directional images to be transferred into two or more predetermined areas that correspond to a shape of the intersection, and to transfer only one of the segmented areas.

7. The blind spot display apparatus according to claim 6,

wherein said selecting unit is configured to transfer corresponding ones of the segmented areas that are different from each other and are in one of the omni-directional images to said object model generating unit and said vehicle model generating unit,

said object model generating unit is configured to use the corresponding ones of the segmented areas transferred by said selecting unit to said object model generating unit, and

said vehicle model generating unit is configured to use the corresponding ones of the segmented areas transferred by said selecting unit to said vehicle model generating unit.

8. The blind spot display apparatus according to claim 5,

wherein the intersection is a cross intersection,

each of the omni-directional images is made up of four quarter-size images corresponding to surrounding four quarter-size sections with respect to a corresponding one of the cameras that obtain the omni-directional images, and

said selecting unit is configured to select, for said right blind spot corresponding point search unit, a corresponding one of the four quarter-size images that includes an image of a right blind spot at the intersection from one of the omni-directional images, and to select, for said left blind spot corresponding point search unit, a corresponding one of the four quarter-size images that includes an image of a left blind spot at the intersection from one of the omni-directional images.

9. The blind spot display apparatus according to claim 3,

wherein the intersection is a cross intersection,

each of the omni-directional images is made up of four quarter-size images corresponding to surrounding four quarter-size sections with respect to a corresponding one of the cameras that obtain the captured images, and

said selecting unit is configured to select, for said vehicle model generating unit, a quarter-size image including the image of the vehicle from each of two of the omni-directional images.

10. The blind spot display apparatus according to claim 3,

wherein said selecting unit is configured to search the omni-directional images for the images of the vehicle, and to select omni-directional images including a largest image and a second largest image of the vehicle that are obtained through the search, from among the omni-directional images including the images of the vehicle and the images of the blind spot.

11. The blind spot display apparatus according to claim 5,

wherein the intersection is a cross intersection at which two roads intersect crosswise,

the omni-directional images are four captured images respectively captured by four of the cameras installed at four corners formed by the intersecting two roads, and

said selecting unit is configured to select, for said left blind spot corresponding point search unit, two of the cameras at two of the corners to a left of one of the roads to the intersection which the vehicle is to enter, when the object is in the left blind spot to a left of the vehicle with respect to the one of the roads to the intersection which the vehicle is to enter, and to select, for said right blind spot corresponding point search unit, two of the cameras at two of the corners to a right of one of the roads to the intersection which the vehicle is to enter, when the object is in the right blind spot to a right of the vehicle with respect to the one of the roads to the intersection which the vehicle is to enter.

12. The blind spot display apparatus according to claim 5,

wherein said left blind spot corresponding point search unit is configured to perform a matching process on respective pixels of one of two perspective projection images from a left direction to a right direction of the pixels, with respect to the other one of the two perspective projection images that corresponds to one of the two omni-directional images captured at a position closer to the left blind spot, the two perspective projection images being transformed from the two omni-directional images selected by said selecting unit, and

said right blind spot corresponding point search unit is configured to perform the matching process on the respective pixels of one of the two perspective projection images from the right direction to the left direction of the pixels, with respect to the other one of the two perspective projection that corresponds to one of the two omni-directional images captured at a position closer to the right blind spot, the two perspective projection images being transformed from the two omni-directional images selected by said selecting unit.

13. The blind spot display apparatus according to claim 1, further comprising;

a vehicle viewpoint image generating unit configured to capture an image seen from the vehicles; and

a vehicle viewpoint-coordinate transforming unit configured to transform the coordinate position of the object calculated by said object model generating unit into display coordinates seen from the vehicle,

wherein said display unit is configured to display the image captured by said vehicle viewpoint image generating unit, and to display the image of the object on the displayed image at a position of the display coordinates.

14. The blind spot display apparatus according to claim 13,

wherein said display unit is configured to display, on the display, each pixel of the object in a pixel color obtained by semi-transparently superimposing a pixel color of each of the pixels of the image of the object on a pixel color of a corresponding pixel in the image captured by said vehicle viewpoint image generating unit.

15. The blind spot display apparatus according to claim 14,

wherein said vehicle viewpoint-coordinate transforming unit is configured to transform each position determined by viewing a predetermined second view angle from the vehicle into display coordinates corresponding to a position in a first view angle at which said vehicle viewpoint image generating unit captures the image, the second view angle being wider than the first view angle, and

said display unit is configured to display the image of the object at the second view angle in a range of the display coordinates corresponding to the position in the first view angle.

16. The blind spot display apparatus according to claim 14,

wherein said display unit is configured to determine a size of the image of the vehicle in each of the omni-directional images, and to increase transparency of the display on which the object is shown as the determined size becomes smaller.

17. The blind spot display apparatus according to claim 3,

wherein said selecting unit is configured to determine a velocity of the object from corresponding ones of the omni-directional images obtained by one of the cameras, and to transfer each of the corresponding ones of the omni-directional images to said object model generating unit and said vehicle model generating unit only when the determined velocity is equal to or higher than a predetermined velocity.

21. The blind spot display apparatus according to claim 1,

wherein the omni-directional images are images captured by the cameras installed at positions at which both the image of the vehicle and an image of the blind spot that cannot be seen from the vehicle are captured, and

each of said object model generating unit and said vehicle model generating unit is configured to obtain the omni-directional images transferred from an outside of said blind spot display apparatus to said blind spot display apparatus, and use the omni-directional images obtained.

18. A blind spot display method of displaying an image of an object in a blind spot of a vehicle, said method comprising:

storing omni-directional images which have been obtained from cameras at different positions and each of which includes the image of the object in the blind spot and an image of the vehicle;

calculating a coordinate position of the object seen from a predetermined camera among the cameras, using a difference between coordinate positions of the images of the object that are included in the omni-directional images;

calculating a coordinate position of the vehicle seen from the predetermined camera, using a difference between coordinate positions of the images of the vehicle that are included in the omni-directional images; and

displaying the image of the object as seen from the vehicle on a display, using the calculated coordinate positions of the object and of the vehicle, the image of the object being included in common in each of the omni-directional images.

19. A computer program for displaying an image of an object in a blind spot of a vehicle using a computer, said computer program being recorded on a computer-readable recording medium and causing the computer to execute:

storing omni-directional images which have been obtained from cameras at different positions and each of which includes the image of the object in the blind spot and an image of the vehicle;

calculating a coordinate position of the object seen from a predetermined camera among the cameras, using a difference between coordinate positions of the images of the object that are included in the omni-directional images;

calculating a coordinate position of the vehicle seen from the predetermined camera, using a difference between coordinate positions of the images of the vehicle that are included in the omni-directional images; and

displaying the image of the object as seen from the vehicle on a display, using the calculated coordinate positions of the object and of the vehicle, the image of the object being included in common in each of the omni-directional images.

20. A computer-readable recording medium on which the computer program according to claim 19 is recorded.