VISION-BASED OBJECT SENSING AND HIGHLIGHTING IN VEHICLE IMAGE DISPLAY SYSTEMS

• US 20150109444A1
• Filed: 11/05/2013
• Published: 04/23/2015
• Est. Priority Date: 10/22/2013
• Status: Abandoned Application

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Wang, Jinsong, Litkouhi, Bakhtiar B., Kazensky, Dennis B., Piasecki, Jeffrey S., Green, Charles A., Frakes, Ryan M., Kiefer, Raymond J.
• Application Number:
  US14/071,982
• Publication Number:
  US 20150109444A1
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A method of displaying a captured image on a display device of a driven vehicle. A scene exterior of the driven vehicle is captured by an at least one vision-based imaging and at least one sensing device. A time-to-collision is determined for each object detected. A comprehensive time-to-collision is determined for each object as a function of each of the determined time-to-collisions for each object. An image of the captured scene is generated by a processor. The image is dynamically expanded to include sensed objects in the image. Sensed objects are highlighted in the dynamically expanded image. The highlighted objects identifies objects proximate to the driven vehicle that are potential collisions to the driven vehicle. The dynamically expanded image with highlighted objects and associated collective time-to-collisions are displayed for each highlighted object in the display device that is determined as a potential collision.

Claims

1. A method of displaying a captured image on a display device of a driven vehicle comprising the steps of:

capturing a scene exterior of the driven vehicle by an at least one vision-based imaging device mounted on the driven vehicle;

detecting objects in the captured image;

determining a time-to-collision for each object detected in the captured image;

sensing objects in a vicinity of the driven vehicle by sensing devices;

determining a time-to-collision for each respective object sensed by the sensing devices;

determining a comprehensive time-to-collision for each object, the comprehensive time-to-collision for each object determined as a function of each of the time-to-collisions determined for each object;

generating an image of the captured scene by a processor, the image being dynamically expanded to include sensed objects in the image;

highlighting sensed objects in the dynamically expanded image that potential collisions to the driven vehicle, the highlighted objects identifying objects proximate to the driven vehicle that are potential collisions to the driven vehicle;

displaying the dynamically expanded image with highlighted objects and associated comprehensive collective time-to-collisions for each highlighted object in the display device that is determined.

2. The method of claim 1 further comprising the steps of:

communicating with a remote vehicle using vehicle-to-vehicle communications to obtain remote vehicle data for determining a time-to-collision with the remote vehicle, wherein the determined time-to-collision based on the vehicle-to-vehicle communication data is used to determine the comprehensive time-to-collision.

3. The method of claim 2 wherein determining a comprehensive time-to-collision for each object includes weighting each respective determined time-to-collision for each object.

4. The method of claim 3 wherein the determination of the comprehensive time-to-collision uses the following formula:

Δ

t_TTC^out=w_im1·

Δ

t_TTC^im1+w_im2·

Δ

t_TTC^im2+w_sens·

Δ

t_TTC^sens+w_v2v·

Δ

t_TTC^sv2v

where Δ

t is a determined time-to-collision, w is a weight factor, and

im1, im2, sens, and v2v represent each respective system that data is obtained for determining the time-to-collision.

5. The method of claim 4 wherein the weighting factors are predetermined weighting factors.

6. The method of claim 4 wherein the weighting factors are adjusted dynamically.

7. The method of claim 1 wherein the dynamically expanded image is displayed on an instrument panel display device.

8. The method of claim 1 wherein the dynamically expanded image is displayed on a center console display device.

9. The method of claim 1 wherein the dynamically expanded image is displayed on a rearview mirror display.

10. The method of claim 9 wherein the dynamically expanded image displayed on the rearview mirror is autonomously enabled in response to a detection of potential collision with a respective object.

11. The method of claim 10 wherein the potential collision is detected in response to a detection of the object and a detection of a lane change.

12. The method of claim 11 wherein the potential collision is detected in response to a detection of the object and an actuation of a turn signal indicating a lane change to a respective lane that the object is disposed.

13. The method of claim 11 wherein a collision warning symbol is displayed in the dynamic expanded image display providing a redundant warning to a driver for the highlighted object detected by a lane change assist system.

14. The method of claim 11 wherein a collision warning symbol is displayed in the dynamic expanded image display providing a redundant warning to a driver for the highlighted object detected by a lane departure warning system.

15. The method of claim 9 wherein the dynamically expanded image is disabled in response to not detecting potential collisions with objects, wherein the rearview mirror display device exhibits mirror reflective properties when the dynamic expanded image display is disabled.

16. The method of claim 9 wherein enabling and disabling the dynamically expanded image is actuated using a manual switch.

17. The method of claim 16 wherein the manual switch is disposed on the steering wheel for enabling and disabling the dynamically expanded image.

18. The method of claim 16 wherein the manual switch is disposed on the rearview mirror display for enabling and disabling the dynamically expanded image.

19. A method of displaying a captured image on a display device of a driven vehicle comprising the steps of:

capturing a scene exterior of the driven vehicle by an at least one vision-based imaging device mounted on the driven vehicle;

detecting objects in the captured image;

sensing objects in a vicinity of the driven vehicle by sensing devices;

generating an image of the captured scene by a processor, the image being dynamically expanded to include sensed objects in the image;

highlighting sensed objects in the dynamically expanded image that are potential collisions to the driven vehicle;

displaying the dynamically expanded image with highlighted objects on the rearview mirror, wherein the rearview mirror is switchable between displaying the dynamically expanded image and displaying mirror reflective properties.

20. The method of claim 19 wherein the dynamically expanded image displayed on the rearview mirror is autonomously enabled in response to a detection of potential collision with a respective object.

21. The method of claim 20 wherein the potential collision is detected in response to a detection of the object and a detection of a lane change.

22. The method of claim 21 wherein the potential collision is detected in response to a detection of the object and an actuation of a turn signal indicating a lane change to a respective lane that the object is disposed.

23. The method of claim 21 wherein a collision warning symbol is displayed in the dynamic expanded image display providing a redundant warning to a driver for the highlighted object detected by a lane change assist system.

24. The method of claim 21 wherein a collision warning symbol is displayed in the dynamic expanded image display providing a redundant warning to a driver for the highlighted object detected by a lane departure warning system.

25. The method of claim 19 wherein the dynamically expanded image is disabled in response to not detecting potential collisions with objects, wherein the rearview mirror display device exhibits mirror reflective properties when the dynamic expanded image display is disabled.

26. The method of claim 19 wherein enabling and disabling the dynamically expanded image is actuated using a manual switch.

27. The method of claim 26 wherein the manual switch is disposed on the steering wheel for enabling and disabling the dynamically expanded image.

28. The method of claim 26 wherein the manual switch is disposed on the rearview mirror display for enabling and disabling the dynamically expanded image.

29. The method of claim 19 further comprising the steps of:

determining a time-to-collision for each respective object detected by the at least one vision-based imaging device and the sensing devices

determining a comprehensive time-to-collision for each object, the comprehensive time-to-collision for each object determined as a function of each of the time-to-collisions determined for each object

displaying the comprehensive time-to-collision associated with each highlighted object on the rearview mirror.

30. The method of claim 29 wherein the driven vehicle communicates with a remote vehicle using vehicle-to-vehicle communications to obtain remote vehicle data for determining a time-to-collision with the remote vehicle, wherein the determined time-to-collision based on the vehicle-to-vehicle communication data is used in determining the comprehensive time-to-collision.

31. The method of claim 30 wherein determining a comprehensive time-to-collision for each object includes weighting each respective determined time-to-collision for each object.

32. The method of claim 31 wherein the determination of the comprehensive time-to-collision uses the following formula:

Δ

t_TTC^out=w_im1·

Δ

t_TTC^im1+w_im2·

Δ

t_TTC^im2+w_sens·

Δ

t_TTC^sens+w_v2v·

Δ

t_TTC^sv2v

where Δ

t is a determined time-to-collision, w is a weight factor, and

im1, im2, sens, and v2v represent each respective system that data is obtained for determining the time-to-collision.

33. The method of claim 32 wherein the weighting factors are predetermined weighting factors.

34. The method of claim 32 wherein the weighting factors are adjusted dynamically.