Bundling night vision and other driver assistance systems (DAS) using near infra-red (NIR) illumination and a rolling shutter

• US 9,800,779 B2
• Filed: 10/12/2015
• Issued: 10/24/2017
• Est. Priority Date: 10/31/2010
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Belman, Efim
• Primary Examiner(s): Huang, Frank
• Application Number:
  US14/880,553
• Publication Number:
  US 20160037064A1
• Time in Patent Office: 743 Days
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A system mountable in a motor vehicle. The system includes a camera and a processor configured to receive image data from the camera. The camera includes a rolling shutter configured to capture the image data during a frame period and to scan and to read the image data into multiple image frames. A near infra-red illuminator may be configured to provide a near infra-red illumination cone in the field of view of the camera. The near infra-red illumination oscillates with an illumination period. A synchronization mechanism may be configured to synchronize the illumination period to the frame period of the rolling shutter. The frame period may be selected so that the synchronization mechanism provides a spatial profile of the near infra-red illumination cone which may be substantially aligned vertically to a specific region, e.g. near the center of the image frame.

Claims

1. A system mountable in a motor vehicle, the system comprising:

a camera configured to scan image data alternately line-by-line into a plurality of image frames;

a processor configured to receive the image data from the camera;

a near infra-red illuminator configured to provide a near infra-red illumination in a field of view of the camera, wherein the near infra-red illumination oscillates with an with an oscillation frequency and an illumination duty cycle; and

a synchronization mechanism configured synchronize the oscillation frequency and the illumination duty cycle line-by-line with the exposure of the images image frames; and

vary the intensity of the near infra-red illumination between at least two lines during line-by-line scanning of image data by the camera, wherein the illuminator provides a first intensity of near infra-red illumination while a first line of a first image frame of the plurality of image frames is scanned, wherein the illuminator provides a second intensity of near infra-red illumination while a second line of the first image frame is scanned, wherein the first and second lines are different, and wherein the first and second intensities of near infra-red illumination are different.

2. The system according to claim 1, wherein the image frames include a long exposure frame and a short exposure frame, wherein said synchronization mechanism is configured to turn on the near infra-red illumination during exposure of at least one middle row but not all rows of said long exposure frame.

3. The system according to claim 1, wherein the processor includes a plurality of driver assistance modules configured to perform respectively a plurality of driver assistance functions;

wherein a first driver assistance module is configured to use the near infra-red illumination.

4. The system according to claim 1, wherein the camera includes a rolling shutter configured to capture the image data during a frame period, wherein the frame period of the camera is selected so that said synchronization mechanism provides a spatial profile of a near infra-red illumination cone substantially aligned vertically to a specific region of at least a portion of the image frames.

5. The system according to claim 1 wherein the processor includes a plurality of driver assistance modules configured to perform respectively a plurality of driver assistance functions, wherein a first driver assistance module is configured to use the near infra-red illumination, wherein a second driver assistance module is configured to use parts of an image that do not have said near infra-red illumination.

6. The system according to claim 5, wherein the image frames are partitioned into first image frames for use by said first driver assistance module and into second image frames for use by a second driver assistance module.

7. The system according to claim 6, wherein said synchronization module is configured to time said near infra-red illumination to be in said first image frames and to avoid said near infra-red illumination in said second image frames.

8. The system according to claim 6, wherein a parameter of said camera is configurable to alternate respective values between the first and second image frames.

9. The system according to claim 8, wherein said parameter is a frame period of said camera.

10. The system according to claim 6, wherein said first driver assistance module is configured to perform pedestrian detection responsive to said near infra-red illumination in said first image frames.

11. The system according to claim 6, wherein said second driver assistance module uses color information from said second image frames to detect headlights of an oncoming vehicle in the field of view of the camera.

12. The system according to claim 6, wherein said second driver assistance module uses color information from said second image frames to detect taillights of a leading vehicle in the field of view of the camera.

13. The system according to claim 6, wherein said second driver assistance module uses color information from said second image frames to detect a traffic sign in the field of view of the camera.

20. The system according to claim 1, wherein the field of view of the camera includes an environment external to the vehicle, and wherein the scanned image data includes depictions of one or more features in the environment external to the vehicle.

22. The system according to claim 1, wherein the first intensity, provided while the first line of the first image frame of the plurality of image frames is scanned, is substantially zero.

14. A method performable by a system mountable in a vehicle, the system including a camera and a processor, the method comprising:

scanning by the camera to read image data alternately line-by-line into a plurality of image frames;

receiving the image data by the processor;

illuminating to provide near infra-red illumination in a field of view of the camera while oscillating the near infra-red illumination with an oscillation frequency and an illumination duty cycle and while synchronizing the oscillation frequency and the illumination duty cycle line-by-line with the exposure of the image frames; and

varying the intensity of a near infra-red illumination between at least two lines during line-by-line scanning of image data by the camera, comprising;

providing a first intensity of near infra-red illumination while a first line of a first image frame of the plurality of image frames is scanned, and providing a second intensity of near infra-red illumination while a second line of the first image frame is scanned, wherein the first and second lines are different, and wherein the first and second intensities of near infra-red illumination are different.

15. The method of claim 14, wherein the image frames include a long exposure frame and a short exposure frame, wherein said synchronizing turns on the near infra-red illumination during exposure of at least one middle row of said long exposure frame.

16. The method of claim 14 further comprising:

configuring said camera to provide to the processor a plurality of image frames;

wherein the image frames are partitioned into first image frames for use by a first driver assistance module and into second image frames for use by a second driver assistance module.

17. The method of claim 16, wherein said synchronizing provides said near infra-red illumination in said first image frames and avoids said near infra-red illumination in said second image frames.

18. The method of claim 16, further comprising:

alternating values of a camera parameter respectively in alternating first and second image frames.

19. The method of claim 18, wherein said camera parameter is said frame period of a rolling shutter.

21. The method according to claim 14, wherein the field of view of the camera includes an environment external to the vehicle, and wherein the scanned image data includes depictions of one or more features in the environment external to the vehicle.

23. The method according to claim 14, wherein the first intensity, provided while the first line of the first image frame of the plurality of image frames is scanned, is substantially zero.