VEHICLE VISION SYSTEM UTILIZING MULTIPLE CAMERAS AND ETHERNET LINKS

• US 20180191928A1
• Filed: 03/05/2018
• Published: 07/05/2018
• Est. Priority Date: 12/05/2012
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Granted Patent: US 10,171,709 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/00 Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicular vision system includes a plurality of cameras disposed at the vehicle and having respective fields of view exterior of the vehicle and being operable to capture frames of image data. Image data captured by each of the cameras is provided to an ECU via an ETHERNET link from the respective camera to the ECU. At least one control signal for controlling operation of each camera is provided from the ECU to the respective camera via the respective ETHERNET link. Image data captured by at least one of the cameras is processed at the ECU to detect an object present exterior of the equipped vehicle, wherein the object is a vehicle that is approaching the equipped vehicle and that is traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling.

Claims

1. A vehicular vision system, said vehicular vision system comprising:

a plurality of cameras disposed at a vehicle equipped with said vision system;

wherein said plurality of cameras comprises at least a first camera, a second camera, a third camera and a fourth camera;

each of said first camera, said second camera, said third camera and said fourth camera having a respective field of view exterior of the equipped vehicle;

each of said first camera, said second camera, said third camera and said fourth camera being operable to capture frames of image data;

an electronic control unit (ECU);

wherein image data captured by said first camera is provided to the ECU via a first ETHERNET link from said first camera to the ECU;

wherein image data captured by said second camera is provided to the ECU via a second ETHERNET link from said second camera to the ECU;

wherein image data captured by said third camera is provided to the ECU via a third ETHERNET link from said third camera to the ECU;

wherein image data captured by said fourth camera is provided to the ECU via a fourth ETHERNET link from said fourth camera to the ECU;

wherein at least one control signal is provided from the ECU to said first camera via said first ETHERNET link;

wherein at least one control signal is provided from the ECU to said second camera via said second ETHERNET link;

wherein at least one control signal is provided from the ECU to said third camera via said third ETHERNET link;

wherein at least one control signal is provided from the ECU to said fourth camera via said fourth ETHERNET link;

wherein first camera calibration data is provided from said first camera to the ECU via said first ETHERNET link;

wherein second camera calibration data is provided from said second camera to the ECU via said second ETHERNET link;

wherein third camera calibration data is provided from said third camera to the ECU via said third ETHERNET link;

wherein fourth camera calibration data is provided from said fourth camera to the ECU via said fourth ETHERNET link;

wherein said first camera is disposed at a front portion of the equipped vehicle, and wherein said second camera is disposed at a driver-side side portion of the equipped vehicle, and wherein said third camera is disposed at a passenger-side side portion of the equipped vehicle, and wherein said fourth camera is disposed at a rear portion of the equipped vehicle;

wherein image data captured by at least one of said first camera, said second camera, said third camera and said fourth camera is processed at the ECU to detect an object present exterior of the equipped vehicle;

wherein the object present exterior of the equipped vehicle is exterior a side of the equipped vehicle;

wherein the object present exterior the side of the equipped vehicle comprises a vehicle that is approaching the equipped vehicle; and

wherein the vehicle that is approaching the equipped vehicle is traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling.

2. The vehicular vision system of claim 1, wherein image data captured respectively by at least one of said first, second, third and fourth cameras is provided to the ECU via a hub.

3. The vehicular vision system of claim 1, wherein said first camera, said second camera, said third camera and said fourth camera are part of a surround view system of the equipped vehicle.

4. The vehicular vision system of claim 3, wherein image data captured by at least one of said first camera, said second camera, said third camera and said fourth camera is processed at the ECU, and wherein, responsive at least in part to said processing, the ECU outputs images for display at a display device of the equipped vehicle for viewing by a driver of the equipped vehicle.

5. The vehicular vision system of claim 4, wherein the displayed images comprise a birds-eye view.

6. The vehicular vision system of claim 3, wherein said first camera comprises a first megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein said second camera comprises a second megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein said third camera comprises a third megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein said fourth camera comprises a fourth megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements.

7. The vehicular vision system of claim 1, wherein the object present exterior of the equipped vehicle is to the rear of the equipped vehicle and is detected during a reversing maneuver of the equipped vehicle.

8. The vehicular vision system of claim 1, wherein said first camera comprises a first megapixel CMOS imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein said second camera comprises a second megapixel CMOS imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein said third camera comprises a third megapixel CMOS imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein said fourth camera comprises a fourth megapixel CMOS imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements.

9. The vehicular vision system of claim 1, wherein, upon powering said first, second, third and fourth cameras, the first, second, third and fourth ETHERNET links are initialized to establish communication with the respective first, second, third and fourth cameras.

10. The vehicular vision system of claim 9, wherein the established communication at least comprises the first, second, third, and fourth camera calibration data for the respective cameras.

11. The vehicular vision system of claim 1, wherein said first camera receives from the ECU via said first ETHERNET link a first camera control signal that regulates timing of said first camera to be synchronous with reference timing of the ECU, and wherein said second camera receives from the ECU via said second ETHERNET link a second camera control signal that regulates timing of said second camera to be synchronous with reference timing of the ECU, and wherein said third camera receives from the ECU via said third ETHERNET link a third camera control signal that regulates timing of said third camera to be synchronous with reference timing of the ECU, and wherein said fourth camera receives from the ECU via said fourth ETHERNET link a fourth camera control signal that regulates timing of said fourth camera to be synchronous with reference timing of the ECU.

12. The vehicular vision system of claim 11, wherein regulation of timing of said first camera comprises starting said first camera synchronous to the ECU reference timing and holding said first camera synchronous to the ECU reference timing, and wherein regulation of timing of said second camera comprises starting said second camera synchronous to the ECU reference timing and holding said second camera synchronous to the ECU reference timing, and wherein regulation of timing of said third camera comprises starting said third camera synchronous to the ECU reference timing and holding said third camera synchronous to the ECU reference timing, and wherein regulation of timing of said fourth camera comprises starting said fourth camera synchronous to the ECU reference timing and holding said fourth camera synchronous to the ECU reference timing.

13. A vehicular vision system, said vehicular vision system comprising:

a plurality of cameras disposed at a vehicle equipped with said vision system;

wherein said plurality of cameras comprises at least a first camera, a second camera, a third camera and a fourth camera;

each of said first camera, said second camera, said third camera and said fourth camera having a respective field of view exterior of the equipped vehicle;

each of said first camera, said second camera, said third camera and said fourth camera being operable to capture frames of image data;

wherein said first camera, said second camera, said third camera and said fourth camera are part of a surround view system of the equipped vehicle;

an electronic control unit (ECU);

wherein image data captured by said first camera is provided to the ECU via a first ETHERNET link from said first camera to the ECU;

wherein image data captured by said second camera is provided to the ECU via a second ETHERNET link from said second camera to the ECU;

wherein image data captured by said third camera is provided to the ECU via a third ETHERNET link from said third camera to the ECU;

wherein image data captured by said fourth camera is provided to the ECU via a fourth ETHERNET link from said fourth camera to the ECU;

wherein at least one control signal is provided from the ECU to said first camera via said first ETHERNET link;

wherein at least one control signal is provided from the ECU to said second camera via said second ETHERNET link;

wherein at least one control signal is provided from the ECU to said third camera via said third ETHERNET link;

wherein at least one control signal is provided from the ECU to said fourth camera via said fourth ETHERNET link;

wherein first camera calibration data is provided from said first camera to the ECU via said first ETHERNET link;

wherein second camera calibration data is provided from said second camera to the ECU via said second ETHERNET link;

wherein third camera calibration data is provided from said third camera to the ECU via said third ETHERNET link;

wherein fourth camera calibration data is provided from said fourth camera to the ECU via said fourth ETHERNET link;

wherein, upon powering said first camera, said first ETHERNET link is initialized to establish communication with said first camera;

wherein, upon powering said second camera, said second ETHERNET link is initialized to establish communication with said second camera;

wherein, upon powering said third camera, said third ETHERNET link is initialized to establish communication with said third camera;

wherein, upon powering said fourth camera, said fourth ETHERNET link is initialized to establish communication with said fourth camera;

wherein said first camera is disposed at a front portion of the equipped vehicle, and wherein said second camera is disposed at a driver-side side portion of the equipped vehicle, and wherein said third camera is disposed at a passenger-side side portion of the equipped vehicle, and wherein said fourth camera is disposed at a rear portion of the equipped vehicle;

wherein image data captured by at least one of said first camera, said second camera, said third camera and said fourth camera is processed at the ECU to detect an object present exterior of the equipped vehicle;

wherein the object present exterior of the equipped vehicle is exterior a side of the equipped vehicle;

wherein the object present exterior the side of the equipped vehicle comprises a vehicle that is approaching the equipped vehicle; and

wherein the vehicle that is approaching the equipped vehicle traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling.

14. The vehicular vision system of claim 13, wherein image data captured respectively by at least one of said first, second, third and fourth cameras is provided to the ECU via a hub.

15. The vehicular vision system of claim 13, wherein image data captured by at least one of said first camera, said second camera, said third camera and said fourth camera is processed at the ECU, and wherein, responsive at least in part to said processing, the ECU outputs images for display at a display device of the equipped vehicle for viewing by a driver of the equipped vehicle.

16. The vehicular vision system of claim 15, wherein the displayed images comprise a birds-eye view.

17. A vehicular vision system, said vehicular vision system comprising:

a plurality of cameras disposed at a vehicle equipped with said vision system;

wherein said plurality of cameras comprises at least a first camera, a second camera, a third camera and a fourth camera;

wherein said first camera comprises a first megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements;

wherein said second camera comprises a second megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements;

wherein said third camera comprises a third megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements;

wherein said fourth camera comprises a fourth megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements;

each of said first camera, said second camera, said third camera and said fourth camera having a respective field of view exterior of the equipped vehicle;

each of said first camera, said second camera, said third camera and said fourth camera being operable to capture frames of image data;

wherein said first camera, said second camera, said third camera and said fourth camera are part of a surround view system of the equipped vehicle;

an electronic control unit (ECU);

wherein image data captured by said first camera is provided to the ECU via a first ETHERNET link from said first camera to the ECU;

wherein image data captured by said second camera is provided to the ECU via a second ETHERNET link from said second camera to the ECU;

wherein image data captured by said third camera is provided to the ECU via a third ETHERNET link from said third camera to the ECU;

wherein image data captured by said fourth camera is provided to the ECU via a fourth ETHERNET link from said fourth camera to the ECU;

wherein at least one control signal is provided from the ECU to said first camera via said first ETHERNET link;

wherein at least one control signal is provided from the ECU to said second camera via said second ETHERNET link;

wherein at least one control signal is provided from the ECU to said third camera via said third ETHERNET link;

wherein at least one control signal is provided from the ECU to said fourth camera via said fourth ETHERNET link;

wherein first camera calibration data is provided from said first camera to the ECU via said first ETHERNET link;

wherein second camera calibration data is provided from said second camera to the ECU via said second ETHERNET link;

wherein third camera calibration data is provided from said third camera to the ECU via said third ETHERNET link;

wherein fourth camera calibration data is provided from said fourth camera to the ECU via said fourth ETHERNET link;

wherein said first camera is disposed at a front portion of the equipped vehicle, and wherein said second camera is disposed at a driver-side side portion of the equipped vehicle, and wherein said third camera is disposed at a passenger-side side portion of the equipped vehicle, and wherein said fourth camera is disposed at a rear portion of the equipped vehicle;

wherein image data captured by at least one of said first camera, said second camera, said third camera and said fourth camera is processed at the ECU to detect an object present exterior of the equipped vehicle;

wherein the object present exterior of the equipped vehicle is exterior a side of the equipped vehicle;

wherein the object present exterior the side of the equipped vehicle comprises a vehicle that is approaching the equipped vehicle;

wherein the vehicle that is approaching the equipped vehicle traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling; and

wherein image data captured by at least one of said first camera, said second camera, said third camera and said fourth camera is processed at the ECU, and wherein, responsive at least in part to said processing, the ECU outputs images for display at a display device of the equipped vehicle for viewing by a driver of the equipped vehicle.

18. The vehicular vision system of claim 17, wherein the displayed images comprise a birds-eye view.

19. The vehicular vision system of claim 17, wherein said first camera receives from the ECU via said first ETHERNET link a first camera control signal that regulates timing of said first camera to be synchronous with reference timing of the ECU, and wherein said second camera receives from the ECU via said second ETHERNET link a second camera control signal that regulates timing of said second camera to be synchronous with reference timing of the ECU, and wherein said third camera receives from the ECU via said third ETHERNET link a third camera control signal that regulates timing of said third camera to be synchronous with reference timing of the ECU, and wherein said fourth camera receives from the ECU via said fourth ETHERNET link a fourth camera control signal that regulates timing of said fourth camera to be synchronous with reference timing of the ECU.

20. The vehicular vision system of claim 19, wherein regulation of timing of said first camera comprises starting said first camera synchronous to the ECU reference timing and holding said first camera synchronous to the ECU reference timing, and wherein regulation of timing of said second camera comprises starting said second camera synchronous to the ECU reference timing and holding said second camera synchronous to the ECU reference timing, and wherein regulation of timing of said third camera comprises starting said third camera synchronous to the ECU reference timing and holding said third camera synchronous to the ECU reference timing, and wherein regulation of timing of said fourth camera comprises starting said fourth camera synchronous to the ECU reference timing and holding said fourth camera synchronous to the ECU reference timing.