Two-way video and 3D transmission between vehicles and system placed on roadside

• US 8,686,873 B2
• Filed: 02/28/2011
• Issued: 04/01/2014
• Est. Priority Date: 02/28/2011
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Kalik, Steven F., Demirdjian, David
• Primary Examiner(s): Crosland, Donnie
• Application Number:
  US13/037,000
• Publication Number:
  US 20120218125A1
• Time in Patent Office: 1,128 Days
• Field of Search: 340/905, 348/135, 348/148, 348/149
• US Class Current: 340/905
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

A system and method for providing visual information to a driver of a first vehicle, including: at least one camera or sensor which is not on the first vehicle but which captures image data that includes a view of a road within a vicinity of the first vehicle; a decision unit which receives the image data from the camera or sensor and which identifies information in the image data which a driver of the first vehicle needs to be informed of; and a display unit on the first vehicle which displays information transmitted to the first vehicle in a view that displays information determined to be missing in the vehicle'"'"'s current line of sight, so that the otherwise missing information can be observed by a driver of the first vehicle.

Claims

1. A system for providing visual information to a driver of a first vehicle, comprising:

at least one camera or sensor which is not on the first vehicle but which captures image data that includes a view of a road within a vicinity of the first vehicle;

a receiver which receives image data from the at least one camera or sensor;

a decision unit which receives the image data from receiver and which identifies information in the image data which a driver of the first vehicle needs to be informed of; and

a display unit on the first vehicle which displays information transmitted to the first vehicle in a view that displays information determined to be missing in the first vehicle'"'"'s current line of sight, so that the otherwise missing information can be observed by a driver of the first vehicle, wherein

the decision unit includes;

a common model generator configured to generate a common model, the common model being a representation of the area within the vicinity of the first vehicle; and

an informative view determination unit configured to determine the information in the image data which the driver of the first vehicle needs to be informed of based on analyzing the common model, and

the informative view determination unit determines the information in the image data which the driver of the first vehicle needs to be informed of based on a determination that an object contained in the image data is not within a line of unobstructed sight of the first vehicle.

2. The system according to claim 1, wherein the at least one camera or sensor is a fixed camera or sensor unit that captures a view from the location of the fixed camera or sensor unit.

3. The system according to claim 1, wherein the at least one camera or sensor is attached to a second vehicle which is within the vicinity of the first vehicle and which captures image data that includes a view from the second vehicle.

4. The system according to claim 1, wherein the common model is generated by identifying at least one common object with a known location in a plurality of views and determining a respective location of at least one additional object in the plurality of views based on the at least one additional object'"'"'s relative location to the at least one common object.

5. The system according to claim 1, wherein the decision unit further comprises:

a view selection unit configured to select a form of the view for displaying the determined information to the driver of the first vehicle as virtual three-dimensional space.

6. The system according to claim 1, wherein the at least one camera or sensor is a 3D camera.

7. The system according to claim 1, wherein the receiver is co-located with the camera or sensor.

8. The system according to claim 1, wherein the decision unit is installed in the first vehicle.

9. The system according to claim 1, wherein the decision unit is co-located with the receiver.

10. The system according to claim 3, wherein each of the first vehicle and second vehicle has a GPS receiver which provides location information of the first vehicle and second vehicle respectively.

11. The system according to claim 3, wherein the decision unit receives location information of the first vehicle and second vehicle and determines a location of the second vehicle to the first vehicle based on the location information.

12. The system according to claim 4, wherein an object contained in the one of the plurality of views is an available parking spot.

13. The system according to claim 1, wherein the display unit displays the object not within the line of unobstructed sight of the first vehicle as the information determined to be missing in the first vehicle'"'"'s current line of sight.

14. The system according to claim 13, wherein the display unit displays the object not within the line of unobstructed sight of the first vehicle by varying an opacity of an object within the line of unobstructed sight of the first vehicle.

15. A method, incorporated on a system for providing visual information to a driver of a first vehicle, comprising:

capturing from at least one camera or sensor image that is not on the first vehicle, data that includes a view of a road within a vicinity of the first vehicle;

receiving, at a receiver, image data from the at least one camera or sensor;

receiving, at a decision unit, the image data from the receiver, which includes a view of an area within the vicinity of the first vehicle, and determining information in the image data which the driver of the first vehicle needs to be informed of and selecting a view for displaying the determined information to a driver of the first vehicle;

displaying, at a display unit on the first vehicle, a view determined by the decision unit to include information in the image data of which a driver of the first vehicle needs to be informed;

generating, at the decision unit, a common model, the common model being a representation of the area within the vicinity of the first vehicle;

determining, at the decision unit, the information in the image data which the driver of the first vehicle needs to be informed of based on analyzing the common model; and

determining, at the decision unit, the information in the image data which the driver of the first vehicle needs to be informed of based on determining that an object contained in the image data is not within a line of unobstructed sight of the first vehicle.

16. The method of claim 15, wherein the at least one camera or sensor is at a fixed location with a view of a road within a vicinity of the first vehicle.

17. The method of claim 15, wherein the at least one camera or sensor is attached to a second vehicle which is within the vicinity of the first vehicle.

18. The method according to claim 15, further comprising identifying at least one common object with a known location in a plurality of views and determines a respective location of at least one additional object in the plurality of views based on the at least one additional object'"'"'s relative location to the at least one common object.

19. The method according to claim 15, further comprising displaying the object not within the line of unobstructed sight of the first vehicle as the information in the image data of which the driver of the first vehicle needs to be informed.

20. The method of claim 19, further comprising displaying the object not within the line of unobstructed sight of the first vehicle by varying an opacity of an object within the line of unobstructed sight of the first vehicle.