Vehicle vision system with adjustable computation and data compression

• US 10,452,076 B2
• Filed: 12/19/2017
• Issued: 10/22/2019
• Est. Priority Date: 01/04/2017
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: May, Benjamin, Ihlenburg, Joern, Shulkin, Boris
• Primary Examiner(s): Jia, Xin
• Application Number:
  US15/846,260
• Publication Number:
  US 20180188740A1
• Time in Patent Office: 672 Days
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors

Abstract

A vision system for a vehicle includes a camera having a camera processor operable to process image data captured by the camera. A central processor is disposed remote from the camera and is operable to process image data captured by the camera. The camera processor processes compressed image data and uncompressed image data and compares ROC curves for a given frame of image data to determine if the compression of that frame of image data has impact on processing of the image data. Responsive to determination that the compression of the frame has impact, the camera processor processes uncompressed data of that frame for a driver assist function and communicates a signal for the driver assist function. Responsive to determination that compression of the frame does not have impact, the vision system communicates compressed data of that frame for processing by the central processor for the driver assist function.

Claims

1. A vision system for a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have, when disposed at the vehicle, a field of view exterior of the vehicle, wherein said camera comprises a camera processor operable to process image data captured by said camera;

a central processor of the vehicle that, with said camera disposed at the vehicle, is disposed in the vehicle at a location separate from and remote from said camera and that is operable to process frames of image data captured by said camera;

wherein image data captured by said camera and processed by said camera processor comprises compressed image data and uncompressed image data;

wherein said camera processor processes compressed image data and uncompressed image data and generates respective receiver operating characteristic (ROC) curves;

wherein said camera processor compares the ROC curves of the compressed image data and the uncompressed image data for a given frame of captured image data to determine if compression of image data of that frame of captured image data has impact on processing of captured image data, and wherein the impact is determined when a difference between the compared ROC curves of the compressed image data and uncompressed image data for the given frame of image data is greater than a threshold level;

wherein, responsive to determination that compression of image data of the frame of captured image data has impact on the processing of image data, said camera processor processes the uncompressed image data of that frame of image data for a driver assist function and communicates a signal to said central processor for the driver assist function; and

wherein, responsive to determination that compression of image data of the frame of captured image data does not have impact on the processing of image data, said vision system communicates compressed image data of that frame of image data to said central processor for processing by said central processor for the driver assist function.

2. The vision system of claim 1, wherein said vision system comprises multiple cameras.

3. The vision system of claim 2, wherein each of said cameras of said multiple cameras comprises a respective camera processor, and wherein each of said camera processors compares ROC curves of the compressed image data and uncompressed image data for a given frame of image data captured by that camera to determine if the compression of image data of that frame of image data has impact on processing of captured image data.

4. The vision system of claim 3, wherein, responsive to determination that compression of image data of the frame of image data captured by that respective camera does not have impact on processing of captured image data, said vision system communicates compressed image data of that frame of image data captured by that camera of said multiple cameras to said central processor for processing by said central processor for the driver assist function.

5. The vision system of claim 1, wherein said processors process captured image data via one or more algorithms.

6. The vision system of claim 5, wherein multiple algorithms are applied to image data captured by said camera by said camera processor and said central processor.

7. The vision system of claim 6, wherein said vision system comprises dynamic balancing of processing load when applying the algorithms.

8. The vision system of claim 1, wherein said vision system comprises dynamic balancing of the processing loads of said camera processor and said central processor.

9. The vision system of claim 1, wherein, responsive to determination that compression of image data of the frame of captured image data has impact on processing of captured image data, said vision system communicates compressed image data of that frame of image data to said central processor for further processing by said central processor for the driver assist function.

10. The vision system of claim 9, wherein the further processing by said central processor is done after processing of uncompressed image data of that frame of image data by said camera processor.

11. A vision system for a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have, when disposed at the vehicle, a field of view exterior of the vehicle, wherein said camera comprises a camera processor operable to process image data captured by said camera;

a central processor of the vehicle that, with said camera disposed at the vehicle, is disposed in the vehicle at a location separate from and remote from said camera and that is operable to process frames of image data captured by said camera;

wherein said vision system comprises dynamic balancing of processing load of said camera processor and said central processor;

wherein each of said camera processor and said central processor processes captured image data via one or more algorithms;

wherein image data captured by said camera and processed by said camera processor comprises compressed image data and uncompressed image data;

wherein said camera processor processes compressed image data and uncompressed image data and generates respective receiver operating characteristic (ROC) curves;

wherein said camera processor compares the ROC curves of the compressed image data and the uncompressed image data for a given frame of captured image data to determine if compression of image data of that frame of captured image data has impact on processing of captured image data, and wherein the impact is determined when a difference between the compared ROC curves of the compressed image data and uncompressed image data for the given frame of image data is greater than a threshold level;

wherein, responsive to determination that compression of image data of the frame of captured image data has impact on the processing of image data, said camera processor processes the uncompressed image data of that frame of image data for a driver assist function and communicates a signal to said central processor for the driver assist function; and

wherein, responsive to determination that compression of image data of the frame of captured image data does not have impact on the processing of image data, said vision system communicates compressed image data of that frame of image data to said central processor for processing by said central processor for the driver assist function.

12. The vision system of claim 11, wherein said vision system comprises multiple cameras.

13. The vision system of claim 12, wherein each of said cameras of said multiple cameras comprises a respective camera processor, and wherein each of said camera processors compares ROC curves of the compressed image data and uncompressed image data for a given frame of image data captured by that camera to determine if the compression of image data of that frame of image data has impact on processing of captured image data.

14. The vision system of claim 13, wherein, responsive to determination that compression of image data of the frame of image data captured by that respective camera does not have impact on processing of captured image data, said vision system communicates compressed image data of that frame of image data captured by that camera of said multiple cameras to said central processor for processing by said central processor for the driver assist function.

15. A vision system for a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have, when disposed at the vehicle, a field of view exterior of the vehicle, wherein said camera comprises a camera processor operable to process image data captured by said camera;

a central processor of the vehicle that, with said camera disposed at the vehicle, is disposed in the vehicle at a location separate from and remote from said camera and that is operable to process frames of image data captured by said camera;

wherein said vision system comprises dynamic balancing of the processing loads of said camera processor and said central processor;

wherein image data captured by said camera and processed by said camera processor comprises compressed image data and uncompressed image data;

wherein said camera processor processes compressed image data and uncompressed image data and generates respective receiver operating characteristic (ROC) curves;

wherein said camera processor compares the ROC curves of the compressed image data and the uncompressed image data for a given frame of captured image data to determine if compression of image data of that frame of captured image data has impact on processing of captured image data, and wherein the impact is determined when a difference between the compared ROC curves of the compressed image data and uncompressed image data for the given frame of image data is greater than a threshold level;

wherein, responsive to determination that compression of image data of the frame of captured image data has impact on the processing of image data, said camera processor (i) processes the uncompressed image data of that frame of image data for a driver assist function, (ii) communicates a signal to said central processor for the driver assist function, and (iii) communicates the compressed image data of that frame of image data to said central processor for further processing by said central processor for the driver assist function; and

wherein, responsive to determination that compression of image data of the frame of captured image data does not have impact on the processing of image data, said vision system communicates compressed image data of that frame of image data to said central processor for processing by said central processor for the driver assist function.

16. The vision system of claim 15, wherein the further processing by said central processor is done after processing of uncompressed image data of that frame of image data by said camera processor.

17. The vision system of claim 16, wherein said processors process captured image data via one or more algorithms.

18. The vision system of claim 15, wherein said vision system comprises multiple cameras.

19. The vision system of claim 18, wherein each of said cameras of said multiple cameras comprises a respective camera processor, and wherein each of said camera processors compares ROC curves of the compressed image data and uncompressed image data for a given frame of image data captured by that camera to determine if the compression of image data of that frame of image data has impact on processing of captured image data.

20. The vision system of claim 19, wherein, responsive to determination that compression of image data of the frame of image data captured by that respective camera does not have impact on processing of captured image data, said vision system communicates compressed image data of that frame of image data captured by that camera of said multiple cameras to said central processor for processing by said central processor for the driver assist function.