METHOD OF SYNCHRONIZING MULTIPLE VEHICULAR CAMERAS WITH AN ECU

• US 20190158706A1
• Filed: 12/28/2018
• Published: 05/23/2019
• Est. Priority Date: 12/05/2012
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Schaffner, Michael
• Granted Patent: US 10,560,610 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/00 Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method of synchronizing cameras with an electronic control unit (ECU) of a vehicular vision system includes providing camera control signals to the cameras from the ECU via respective links from the ECU to the cameras, with the camera control signals regulating timing of the respective camera to be synchronous with reference timing of the ECU. The timing regulation of the cameras includes starting the camera synchronous to the ECU reference timing and holding the camera synchronous to the ECU reference timing. Image data is captured with each camera and provided to the ECU via the respective link. Image data captured by at least one of the cameras may be processed to detect an object present exterior of the vehicle.

Claims

1. A method of synchronizing cameras with an electronic control unit of a vehicular vision system, said method comprising:

providing a plurality of cameras at a vehicle equipped with the vehicular vision system, wherein the plurality of cameras comprises at least a first camera, a second camera, a third camera and a fourth camera;

wherein providing the plurality of cameras comprises providing the first camera at a front portion of the equipped vehicle, providing the second camera at a left-side side portion of the equipped vehicle, providing the third camera at a right-side side portion of the equipped vehicle, and providing the fourth camera at a rear portion of the equipped vehicle;

wherein each of the first camera, the second camera, the third camera and the fourth camera, when provided at the equipped vehicle, has a respective field of view exterior of the equipped vehicle;

providing an electronic control unit (ECU) at the equipped vehicle;

providing a first camera control signal to the first camera from the ECU via a first link from the ECU to the first camera, wherein the first camera control signal regulates timing of the first camera to be synchronous with reference timing of the ECU;

providing a second camera control signal to the second camera from the ECU via a second link from the ECU to the second camera, wherein the second camera control signal regulates timing of the second camera to be synchronous with reference timing of the ECU;

providing a third camera control signal to the third camera from the ECU via a third link from the ECU to the third camera, wherein the third camera control signal regulates timing of the third camera to be synchronous with reference timing of the ECU;

providing a fourth camera control signal to the fourth camera from the ECU via a fourth link from the ECU to the fourth camera, wherein the fourth camera control signal regulates timing of the fourth camera to be synchronous with reference timing of the ECU;

regulating timing of the first camera via starting the first camera synchronous to the ECU reference timing and holding the first camera synchronous to the ECU reference timing;

regulating timing of the second camera via starting the second camera synchronous to the ECU reference timing and holding the second camera synchronous to the ECU reference timing;

regulating timing of the third camera via starting the third camera synchronous to the ECU reference timing and holding the third camera synchronous to the ECU reference timing;

regulating timing of the fourth camera via starting the fourth camera synchronous to the ECU reference timing and holding the fourth camera synchronous to the ECU reference timing;

capturing frames of image data with each of the first camera, the second camera, the third camera and the fourth camera;

providing image data captured by the first camera to the ECU via the first link from the first camera to the ECU;

providing image data captured by the second camera to the ECU via the second link from the second camera to the ECU;

providing image data captured by the third camera to the ECU via the third link from the third camera to the ECU; and

providing image data captured by the fourth camera to the ECU via the fourth link from the fourth camera to the ECU.

2. The method of claim 1, comprising (i) providing at least one other camera control signal from the ECU to the first camera via the first link, (ii) providing at least one other camera control signal from the ECU to the second camera via the second link, (iii) providing at least one other camera control signal from the ECU to the third camera via the third link and (iv) providing at least one other camera control signal from the ECU to the fourth camera via the fourth link.

3. The method of claim 1, comprising processing, at the ECU, image data captured by at least one of the first camera, the second camera, the third camera and the fourth camera to detect an object present exterior of the equipped vehicle.

4. The method of claim 3, wherein the object present exterior of the equipped vehicle is exterior a side of the equipped vehicle.

5. The method of claim 4, wherein the object present exterior the side of the equipped vehicle comprises a vehicle that is approaching the equipped vehicle.

6. The method of claim 5, wherein the vehicle that is approaching the equipped vehicle is traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling.

7. The method of claim 3, wherein the object present exterior of the equipped vehicle is to the rear of the equipped vehicle and is detected during a reversing maneuver of the equipped vehicle.

8. The method of claim 1, wherein the first camera, the second camera, the third camera and the fourth camera are part of a surround vision system of the equipped vehicle.

9. The method of claim 8, comprising (i) processing, at the ECU, image data captured by at least one of the first camera, the second camera, the third camera and the fourth camera, and (ii) outputting, via the ECU, and responsive at least in part to the processing, images for display at a display device of the equipped vehicle for viewing by an occupant of the equipped vehicle.

10. The method of claim 9, wherein the displayed images comprise birds-eye view images generated by said surround vision system.

11. The method of claim 1, wherein the first camera comprises a first megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the second camera comprises a second megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the third camera comprises a third megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the fourth camera comprises a fourth megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements.

12. The method of claim 1, comprising, upon powering the first, second, third and fourth cameras, initializing the first, second, third and fourth links to establish communication with the respective first, second, third and fourth cameras.

13. The method of claim 12, wherein the established communication at least comprises first, second, third, and fourth camera calibration data for the respective first, second, third, and fourth cameras.

14. A method of synchronizing cameras with an electronic control unit of a vehicular vision system, said method comprising:

providing a plurality of cameras at a vehicle equipped with the vehicular vision system, wherein the plurality of cameras comprises at least a first camera, a second camera, a third camera and a fourth camera;

wherein the first camera comprises a first megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the second camera comprises a second megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the third camera comprises a third megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the fourth camera comprises a fourth megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements;

wherein providing the plurality of cameras comprises providing the first camera at a front portion of the equipped vehicle, providing the second camera at a left-side side portion of the equipped vehicle, providing the third camera at a right-side side portion of the equipped vehicle, and providing the fourth camera at a rear portion of the equipped vehicle;

wherein each of the first camera, the second camera, the third camera and the fourth camera, when provided at the equipped vehicle, has a respective field of view exterior of the equipped vehicle;

providing an electronic control unit (ECU) at the equipped vehicle;

providing a first camera control signal to the first camera from the ECU via a first link from the ECU to the first camera, wherein the first camera control signal regulates timing of the first camera to be synchronous with reference timing of the ECU;

providing a second camera control signal to the second camera from the ECU via a second link from the ECU to the second camera, wherein the second camera control signal regulates timing of the second camera to be synchronous with reference timing of the ECU;

providing a third camera control signal to the third camera from the ECU via a third link from the ECU to the third camera, wherein the third camera control signal regulates timing of the third camera to be synchronous with reference timing of the ECU;

providing a fourth camera control signal to the fourth camera from the ECU via a fourth link from the ECU to the fourth camera, wherein the fourth camera control signal regulates timing of the fourth camera to be synchronous with reference timing of the ECU;

regulating timing of the first camera via starting the first camera synchronous to the ECU reference timing and holding the first camera synchronous to the ECU reference timing;

regulating timing of the second camera via starting the second camera synchronous to the ECU reference timing and holding the second camera synchronous to the ECU reference timing;

regulating timing of the third camera via starting the third camera synchronous to the ECU reference timing and holding the third camera synchronous to the ECU reference timing;

regulating timing of the fourth camera via starting the fourth camera synchronous to the ECU reference timing and holding the fourth camera synchronous to the ECU reference timing;

providing at least one other camera control signal from the ECU to the first camera via the first link;

providing at least one other camera control signal from the ECU to the second camera via the second link;

providing at least one other camera control signal from the ECU to the third camera via the third link;

providing at least one other camera control signal from the ECU to the fourth camera via the fourth link;

capturing frames of image data with each of the first camera, the second camera, the third camera and the fourth camera;

providing image data captured by the first camera to the ECU via the first link from the first camera to the ECU;

providing image data captured by the second camera to the ECU via the second link from the second camera to the ECU;

providing image data captured by the third camera to the ECU via the third link from the third camera to the ECU;

providing image data captured by the fourth camera to the ECU via the fourth link from the fourth camera to the ECU; and

processing, at the ECU, image data captured by at least one of the first camera, the second camera, the third camera and the fourth camera to detect an object present exterior of the equipped vehicle.

15. The method of claim 14, wherein the object present exterior of the equipped vehicle is exterior a side of the equipped vehicle.

16. The method of claim 15, wherein the object present exterior the side of the equipped vehicle comprises a vehicle that is approaching the equipped vehicle.

17. The method of claim 16, wherein the vehicle that is approaching the equipped vehicle is traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling.

18. The method of claim 14, wherein the object present exterior of the equipped vehicle is to the rear of the equipped vehicle and is detected during a reversing maneuver of the equipped vehicle.

19. The method of claim 14, wherein the first camera, the second camera, the third camera and the fourth camera are part of a surround vision system of the equipped vehicle.

20. The method of claim 19, comprising (i) processing, at the ECU, image data captured by at least one of the first camera, the second camera, the third camera and the fourth camera, and (ii) outputting, via the ECU, and responsive at least in part to the processing, images for display at a display device of the equipped vehicle for viewing by an occupant of the equipped vehicle.

21. The method of claim 20, wherein the displayed images comprise birds-eye view images generated by said surround vision system.

22. The method of claim 14, comprising, upon powering the first, second, third and fourth cameras, initializing the first, second, third and fourth links to establish communication with the respective first, second, third and fourth cameras.

23. The method of claim 22, wherein the established communication at least comprises first, second, third, and fourth camera calibration data for the respective first, second, third, and fourth cameras.

24. A method of synchronizing cameras with an electronic control unit of a vehicular vision system, said method comprising:

providing a plurality of cameras at a vehicle equipped with the vehicular vision system, wherein the plurality of cameras comprises at least a first camera, a second camera, a third camera and a fourth camera;

wherein the first camera comprises a first megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the second camera comprises a second megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the third camera comprises a third megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements, and wherein the fourth camera comprises a fourth megapixel imaging array having at least one million photosensing elements arranged in columns and rows of photosensing elements;

wherein providing the plurality of cameras comprises providing the first camera at a front portion of the equipped vehicle, providing the second camera at a left-side side portion of the equipped vehicle, providing the third camera at a right-side side portion of the equipped vehicle, and providing the fourth camera at a rear portion of the equipped vehicle;

wherein each of the first camera, the second camera, the third camera and the fourth camera, when provided at the equipped vehicle, has a respective field of view exterior of the equipped vehicle;

providing an electronic control unit (ECU) at the equipped vehicle;

providing a first link between the ECU and the first camera;

providing a second link between the ECU and the second camera;

providing a third link between the ECU and the third camera;

providing a fourth link between the ECU and the fourth camera;

upon powering the first, second, third and fourth cameras, initializing the first, second, third and fourth links to establish communication with the respective first, second, third and fourth cameras;

providing a first camera control signal to the first camera from the ECU via the first link from the ECU to the first camera, wherein the first camera control signal regulates timing of the first camera to be synchronous with reference timing of the ECU;

providing a second camera control signal to the second camera from the ECU via the second link from the ECU to the second camera, wherein the second camera control signal regulates timing of the second camera to be synchronous with reference timing of the ECU;

providing a third camera control signal to the third camera from the ECU via the third link from the ECU to the third camera, wherein the third camera control signal regulates timing of the third camera to be synchronous with reference timing of the ECU;

providing a fourth camera control signal to the fourth camera from the ECU via the fourth link from the ECU to the fourth camera, wherein the fourth camera control signal regulates timing of the fourth camera to be synchronous with reference timing of the ECU;

regulating timing of the first camera via starting the first camera synchronous to the ECU reference timing and holding the first camera synchronous to the ECU reference timing;

regulating timing of the second camera via starting the second camera synchronous to the ECU reference timing and holding the second camera synchronous to the ECU reference timing;

regulating timing of the third camera via starting the third camera synchronous to the ECU reference timing and holding the third camera synchronous to the ECU reference timing;

regulating timing of the fourth camera via starting the fourth camera synchronous to the ECU reference timing and holding the fourth camera synchronous to the ECU reference timing;

capturing frames of image data with each of the first camera, the second camera, the third camera and the fourth camera;

providing image data captured by the first camera to the ECU via the first link from the first camera to the ECU;

providing image data captured by the second camera to the ECU via the second link from the second camera to the ECU;

providing image data captured by the third camera to the ECU via the third link from the third camera to the ECU; and

providing image data captured by the fourth camera to the ECU via the fourth link from the fourth camera to the ECU.

25. The method of claim 24, wherein the established communication at least comprises first, second, third, and fourth camera calibration data for the respective first, second, third, and fourth cameras.

26. The method of claim 24, comprising (i) providing at least one other camera control signal from the ECU to the first camera via the first link, (ii) providing at least one other camera control signal from the ECU to the second camera via the second link, (iii) providing at least one other camera control signal from the ECU to the third camera via the third link and (iv) providing at least one other camera control signal from the ECU to the fourth camera via the fourth link.

27. The method of claim 24, comprising processing, at the ECU, image data captured by at least one of the first camera, the second camera, the third camera and the fourth camera to detect an object present exterior of the equipped vehicle.

28. The method of claim 27, wherein the object present exterior of the equipped vehicle is exterior a side of the equipped vehicle.

29. The method of claim 28, wherein the object present exterior the side of the equipped vehicle comprises a vehicle that is approaching the equipped vehicle.

30. The method of claim 29, wherein the vehicle that is approaching the equipped vehicle is traveling in a traffic lane adjacent to a traffic lane in which the equipped vehicle is traveling.

31. The method of claim 27, wherein the object present exterior of the equipped vehicle is to the rear of the equipped vehicle and is detected during a reversing maneuver of the equipped vehicle.

32. The method of claim 24, wherein the first camera, the second camera, the third camera and the fourth camera are part of a surround vision system of the equipped vehicle.

33. The method of claim 32, comprising (i) processing, at the ECU, image data captured by at least one of the first camera, the second camera, the third camera and the fourth camera, and (ii) outputting, via the ECU, and responsive at least in part to the processing, images for display at a display device of the equipped vehicle for viewing by an occupant of the equipped vehicle.

34. The method of claim 33, wherein the displayed images comprise birds-eye view images generated by said surround vision system.