Vehicle-use visual field assistance system in which information dispatch apparatus transmits images of blind spots to vehicles

• US 8,179,241 B2
• Filed: 09/11/2008
• Issued: 05/15/2012
• Est. Priority Date: 09/14/2007
• Status: Active Grant

• Current Assignee: Carnegie Mellon University, DENSO Corporation
• Original Assignee: Carnegie Mellon University, DENSO Corporation
• Inventors: Tamatsu, Yukimasa, Datta, Ankur, Sheikh, Yaser, Sakai, Hiroshi, Kanade, Takeo
• Primary Examiner(s): Swarthout, Brent
• Assistant Examiner(s): Bee, Andrew
• Application Number:
  US12/283,422
• Publication Number:
  US 20090140881A1
• Time in Patent Office: 1,342 Days
• Field of Search: 340/436, 340/903, 340/988, 340/990, 340/995.2, 382/294, 348/148
• US Class Current: 340/436
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

A camera of a ground-based information dispatch apparatus captures a blind-spot image, showing a region that is a blind spot with respect to a vehicle driver. A vehicle-mounted camera captures a forward-view image corresponding to the viewpoint of the driver, and the forward-view image is transmitted to the information dispatch apparatus together with vehicle position and direction information and camera parameters. Based on the received information, the blind-spot image is converted to a corresponding image having the viewpoint of the vehicle driver, and the forward-view image and viewpoint-converted blind-spot image are combined to form a synthesized image, which is transmitted to the vehicle.

Claims

1. A vehicle-use visual field assistance system comprising:

an information dispatch apparatus comprising

a blind spot image acquisition unit comprising a ground-based camera positioned and oriented for capturing a blind-spot image showing a current condition of a region that is a blind spot with respect to a forward field of view of a driver of an object vehicle approaching the vicinity of a street intersection,

means for receiving vehicle information transmitted from said object vehicle, said vehicle information comprising at least information expressing a forward-view image corresponding to said forward field of view of said driver and information expressing a current position of said object vehicle in relation to said ground-based camera,

means for executing viewpoint conversion of said forward-view image and of said blind-spot image to a converted forward-view image and to a converted blind-spot image respectively, said converted forward-view image and said converted blind-spot image having a common viewpoint, and to combine said converted forward-view image and said converted blind-spot image into a synthesized image, and

means for transmitting said synthesized image;

a vehicle-mounted apparatus installed in said object vehicle, comprising

means for receiving said synthesized image transmitted from said information dispatch apparatus,

means for transmitting, said vehicle information to said information dispatch apparatus, and

means for displaying said received synthesized image.

2. A vehicle-use visual field assistance system as claimed in claim 1, wherein a viewpoint of said forward-view image constitutes said common viewpoint.

3. A vehicle-use visual field assistance system as claimed in claim 2, wherein said means for executing is configured to generate said synthesized image in a manner for rendering at least a part of said converted blind spot image semi-transparent when displayed by said displaying means.

4. A vehicle-use visual field assistance system as claimed in claim 2, comprising means for deriving a partial blind-spot image from said blind-spot image, where said partial blind-spot image contains objects of a category which includes vehicles and persons,

wherein said means for executing combines said partial blind-spot image with said forward-view image to obtain said synthesized image.

5. A vehicle-use visual field assistance system as claimed in claim 4, wherein said information dispatch apparatus comprises a memory having data stored therein beforehand expressing a background image of said blind spot, and wherein:

said deriving means is configured to derive said partial blind spot image as a difference image, expressing differences between said background image and said blind-spot image; and

said means for executing is configured to apply said viewpoint conversion to said difference image, and to combine a resultant viewpoint-converted difference image with said forward-view image to obtain said synthesized image.

6. A vehicle-use visual field assistance system as claimed in claim 4, wherein said information dispatch apparatus comprises a memory having data stored therein beforehand expressing a background image of said blind spot, and wherein:

said deriving means is configured to derive a difference image, expressing differences between said background image and said blind-spot image, select a fixed-size section of said blind-spot image such that said section contains any target bodies which appear in said difference image and which are absent from said difference image, and generate said partial blind-spot image as an image that includes said selected section;

said executing means is configured to apply said viewpoint conversion to said partial blind-spot image, and to combine a resultant viewpoint-converted partial blind-spot image with said forward-view image to obtain said synthesized image.

7. A vehicle-use visual field assistance system as claimed in claim 1, wherein:

said vehicle information includes information specifying a current position of said object vehicle;

said common viewpoint is a birds-eye viewpoint; and

said executing means is configured to generate said synthesized image as an overhead view which includes said blind spot and includes a region containing said current position of the object vehicle.

8. A vehicle-use visual field assistance system as claimed in claim 7, wherein said synthesized image includes a marker indicating said current position of the object vehicle.

12. A vehicle-use visual field assistance system as claimed in claim 1, wherein said vehicle-mounted apparatus comprises:

a camera installed on said object vehicle, for capturing said forward-view image;

means for acquiring said forward-view image from the camera; and

means for transmitting said acquired forward-view image as part of said vehicle information.

13. A vehicle-use visual field assistance system as claimed in claim 12, wherein said vehicle information transmitted by the means for transmitting said acquired forward-view image includes captured-image information for use in performing said viewpoint conversion of said blind-spot image and of said forward-view image.

14. A vehicle-use visual field assistance system as claimed in claim 13, wherein said captured-image information includes internal parameters of said camera of the object vehicle.

15. A vehicle-use visual field assistance system as claimed in claim 14, wherein said internal parameters comprise at least a focal length of a lens of said object vehicle camera and effective spatial dimensions of a picture element of said forward-view image.

16. A vehicle-use visual field assistance system as claimed in claim 14, wherein said captured-image information includes external parameters of said camera of the object vehicle.

17. A vehicle-use visual field assistance system as claimed in claim 16, wherein said external parameters of the camera of the object vehicle comprise a height of said camera and an orientation direction of said camera.

18. A vehicle-use visual field assistance system as claimed in claim 16, wherein said external parameters of the camera of the object vehicle are expressed as relative parameters, said relative parameters representing a difference between a height of said camera of the object vehicle and a predetermined average height of the eyes of a vehicle driver, and a difference between a direction in which said camera is oriented with respect to said object vehicle and a direction of travel of said object vehicle.

19. A vehicle-use visual field assistance system as claimed in claim 12, wherein:

said information dispatch apparatus comprises

a dispatch-side radio transmitting and receiving apparatus, and

means for transmitting a predetermined response signal via said dispatch-side radio transmitting and receiving apparatus when a predetermined verification signal is received via said dispatch-side radio transmitting and receiving apparatus;

said vehicle-mounted apparatus comprises a vehicle-side radio transmitting and receiving apparatus; and

said means for transmitting said acquired forward-view image is configured to transmit said vehicle information via said vehicle-side radio transmitting and receiving apparatus when said response signal is received via said vehicle-side radio transmitting and receiving apparatus.

9. A vehicle-use visual field assistance system comprising:

an information dispatch apparatus comprising

means for acquiring a blind-spot image showing a current condition of a region that is a blind spot with respect to a forward field of view of a driver of an object vehicle approaching the vicinity of a street intersection,

means for receiving vehicle information, said vehicle information including a forward-view image corresponding to said forward field of view of said driver,

means for executing viewpoint conversion of said forward-view image and of said blind-spot image to a converted forward-view image and to a converted blind-spot image respectively, said converted forward-view image and said converted blind-spot image having a common viewpoint, and to combine said converted forward-view image and said converted blind-spot image into a synthesized image,

means for transmitting said synthesized image;

a vehicle-mounted apparatus installed in said object vehicle, comprising

means for receiving said synthesized image transmitted from said information dispatch apparatus,

means for transmitting vehicle information relating to said object vehicle, and

means for displaying said received synthesized image; and

means for inhibiting display of said synthesized image by said displaying means of the vehicle-mounted apparatus when a location of said object vehicle is within a predetermined distance from said street intersection, as indicated by contents of said vehicle information.

10. A vehicle-use visual field assistance system as claimed in claim 9, wherein said inhibiting means comprises means configured for judging whether said object vehicle is within said predetermined distance from the street intersection, based upon said contents of said vehicle information received from said object vehicle, and to inhibit generation of said synthesized image by said image generating means when said object vehicle is judged to be within said predetermined distance.

11. A vehicle-use visual field assistance system as claimed in claim 10, wherein:

said transmitting means of the information dispatch apparatus is configured to transmit a warning image to said object vehicle in place of said synthesized image, for prompting said driver to proceed with caution while directly observing said forward field of view, when said inhibiting means inhibits generation of said synthesized image; and

said displaying means of said vehicle-mounted apparatus is configured to display said warning image, when said warning image is received by said receiving means of the vehicle-mounted apparatus.

20. A vehicle-use visual field assistance system comprising:

an information dispatch apparatus comprising

means for acquiring a blind-spot image showing a current condition of a region that is a blind spot with respect to a forward field of view of a driver of an object vehicle approaching the vicinity of a street intersection,

means for receiving vehicle information, said vehicle information including a forward-view image corresponding to said forward field of view of said driver,

means for executing viewpoint conversion of said forward-view image and of said blind-spot image to a converted forward-view image and to a converted blind-spot image respectively, said converted forward-view image and said converted blind-spot image having a common viewpoint, and to combine said converted forward-view image and said converted blind-spot image into a synthesized image,

means for transmitting said synthesized image;

a vehicle-mounted apparatus installed in said object vehicle, comprising

means for receiving said synthesized image transmitted from said information dispatch apparatus,

means for transmitting vehicle information relating to said object vehicle, and

an infrastructure-side apparatus installed adjacent to a street of said street intersection, said infrastructure-side apparatus comprising;

a sensor positioned and configured to detect when said object vehicle attains a predetermined position, and to generate a sensor signal when said attainment is detected;

a camera responsive to said sensor signal for capturing said forward-view image; and

transmitter means configured to transmit said captured forward-view image to said information dispatch apparatus.

21. A vehicle-use visual field assistance system comprising:

an information dispatch apparatus comprising

a first camera, installed at a location in or adjacent to a street intersection, said camera being positioned and oriented to capture a blind-spot image showing a current condition of a region that is a blind spot with respect to the forward field of view of a driver of an object vehicle approaching the vicinity of said street intersection,

circuitry configured to generate first characteristic information, said first characteristic information being specific to said first camera and comprising internal parameters of said first camera, a location of said first camera, and a height and an orientation direction of said first camera;

a radio receiver apparatus for receiving vehicle information relating to said object vehicle, said vehicle information including a forward-view image and second characteristic information,

means for converting said blind-spot image to a converted blind-spot image which has a viewpoint of said forward-view image, said conversion being executed based upon said first characteristic information and said second characteristic information, and to combine said forward-view image and at least a selected part of said converted blind-spot image into a synthesized image, and

a radio transmitter for transmitting said synthesized image; and

a vehicle-mounted apparatus installed in said object vehicle, comprising

a second camera, mounted on said vehicle, for capturing said forward-view image,

means for detecting a current location of said object vehicle,

circuitry configured to generate second characteristic information, said second characteristic information being specific to said second camera and comprising internal parameters of said second camera, said current location, and a height and a current orientation direction of said second camera,

a radio transmitter for transmitting said forward-view image in conjunction with said second characteristic information, as said vehicle information,

a radio receiver for receiving said synthesized image transmitted from said information dispatch apparatus, and

a display unit for displaying said received synthesized image.

22. A vehicle-use visual field assistance system as claimed in claim 21, wherein said vehicle-mounted apparatus comprises:

means for detecting a direction of motion of said object vehicle; and

a memory having relative information stored therein, indicative of a relationship between said orientation direction of said second camera and said direction of motion of the object vehicle;

and wherein said current orientation direction of said second camera is calculated based upon said relative information and said detected direction of motion.