Time to collision using a camera

• US 10,860,869 B2
• Filed: 02/13/2013
• Issued: 12/08/2020
• Est. Priority Date: 02/15/2012
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Dagan, Erez, Kahlon, Noam, Latowicki, Michael, Stein, Gideon
• Primary Examiner(s): Wendmagegn, Girumsew
• Application Number:
  US14/378,798
• Publication Number:
  US 20150085119A1
• Time in Patent Office: 2,855 Days
• Field of Search: 348148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Computing a time-to-contact (TTC) of a vehicle with an object. The object includes a light source. The system mountable in the vehicle includes a camera and a processor configured to capture images of the object. The processor is operable to track a spot between the images to produce a tracked spot. The spot includes an image of the light source. The processor is operable to compute time-to-contact (TTC) responsive to changes in brightness of the tracked spot.

Claims

1. A method of computing a time-to-contact (TTC) of a vehicle with an object in an environment of the vehicle, wherein the object includes a light source, the method performable by a camera connectible to a processor on-board the vehicle, the method comprising:

capturing, via the camera, a plurality of images of the object;

tracking, by the processor, a spot identified in at least two of the plurality of images, wherein the tracked spot includes light emanating from a light source of another vehicle;

determining, by the processor, a change in brightness of the tracked spot based on a change in respective pixel values of the tracked spot in the at least two of the plurality of images;

determining, by the processor, a change in computed energy of the tracked spot based on a change in aggregate pixel values in at least one image of the at least two of the plurality of images; and

computing, by the processor, the time-to-contact based on the change in brightness of the tracked spot and the change in computed energy of the tracked spot.

2. The method according to claim 1, wherein the energy is computed by summing pixel values from a plurality of pixels of the tracked spot.

3. The method according to claim 1, further comprising:

fitting a function of the energy to a function of the time-to-contact.

4. The method of claim 3, wherein a reciprocal of a square root of the energy is fit to a linear function of the time-to-contact.

5. The method according to claim 1, further comprising:

determining whether the vehicle and the object are on a collision course responsive to detecting motion of the tracked spot in the plurality of images.

6. The method of claim 5, wherein determining whether the vehicle and the object are on a collision course includes a determination with respect to an amount of time before the spot moves a predetermined vertical distance in the plurality of images as compared to an amount of time before the spot moves a predetermined horizontal distance in the plurality of images.

7. The method of claim 1, wherein the energy is computed from a predicted spot intensity based on non-saturated pixels at an edge of the tracked spot.

8. The method of claim 1, wherein the change in the energy over time is determined based on a single frame value, wherein the single frame value is calculated from a histogram of energy changes between a current compared image frame and multiple previous image frames, and wherein the energy changes are weighted in the histogram based on a relative lapse in time between the current compared image frame and the multiple previous image frames.

9. The method of claim 8, wherein the change in the energy over time is determined based on a weighted sum of a current single frame value and a plurality of prior single frame values calculated from a plurality of prior current compared image frames.

10. The method of claim 1, wherein tracking the spot identified in at least two of the plurality of images includes determining whether fluctuations in energy for the tracked spot between images correspond to known fluctuations associated with a known type of light source.

11. A system for computing a time-to-contact (TTC) of a vehicle with an object in an environment of the vehicle, wherein the object includes a light source, the system mountable in the vehicle, the system comprising:

a camera; and

a processor configured to;

capture from the camera a plurality of images of the object;

track a spot identified in at least two of the plurality of images, wherein the tracked spot includes light emanating from a light source of another vehicle;

determine a change in brightness of the tracked spot based on a change in respective pixel values of the tracked spot in the at least two of the plurality of images;

determine a change in computed energy of the tracked spot based on a change in aggregate pixel values in at least one image of the at least two of the plurality of images; and

compute the time-to-contact (TTC) based on the change in brightness of the tracked spot and the change in computed energy of the tracked spot.

12. The system according to claim 11, wherein the energy is computed by summing pixel values from a plurality of pixels of the tracked spot.

13. The system according to claim 11, wherein the processor is further configured to:

fit a function of the energy to a function of the time.

14. The system according to claim 11, wherein the camera is a camera from a stereo pair of cameras.

15. The system according to claim 11, wherein the time-to-contact is computed from a sensor input to the processor from a combination of the camera with a radar system.

16. The system according to claim 11, wherein the time-to-contact is computed from a sensor input to the processor from a combination of the camera with a lidar system.