Virtual see-through instrument cluster with live video

• US 9,756,319 B2
• Filed: 02/27/2014
• Issued: 09/05/2017
• Est. Priority Date: 02/27/2014
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Di Censo, Davide, Marti, Stefan
• Primary Examiner(s): Pham, Nam
• Application Number:
  US14/192,043
• Publication Number:
  US 20150245017A1
• Time in Patent Office: 1,286 Days
• CPC Class Codes: B60K 2031/0025 Detecting position of target vehicle, e.g. vehicle driving ahead from host vehicle
• CPC Class Codes: B60K 2370/1531 Three-dimensional displays
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60K 35/211 producing three-dimensional [3D] effects, e.g. stereoscopic images
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/101 using cameras with adjustable capturing direction
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/202 displaying a blind spot scene on the vehicle part responsible for the blind spot
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/327 : Calibration thereof
• CPC Class Codes: H04N 13/366 : using viewer tracking
• CPC Class Codes: H04N 13/383 : for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

At least one embodiment disclosed herein includes generating a three-dimensional graphical output that includes a three-dimensional representation of a video of an area external to a vehicle, and displaying the three-dimensional graphical output on a three-dimensional display disposed within a vehicle.

Claims

1. A method for displaying live video in a vehicle, comprising:

capturing a video of an area external to the vehicle;

generating, via a stereoscopic display surface disposed within the vehicle, a plurality of display layers, wherein each display layer has a different perceived depth relative to a driver of the vehicle and the stereoscopic display surface;

causing one or more display elements to be displayed in a first stereoscopic image within a first display layer included in the plurality of display layers that has a first perceived depth relative to the driver of the vehicle and the stereoscopic display surface; and

causing a representation of the video to be displayed in a second stereoscopic image within a second display layer included in the plurality of display layers that has a second perceived depth relative to the driver of the vehicle and the stereoscopic display surface,

wherein the first perceived depth is different from the second perceived depth such that one display layer of the first display layer and the second display layer is disposed in front relative to the driver of the vehicle and is transparent to allow the driver of the vehicle to see through the one display layer.

2. The method of claim 1, wherein at least a portion of the area external to the vehicle is obstructed from view of the driver.

3. The method of claim 1, wherein capturing the video comprises:

detecting a gaze of the driver; and

causing a camera capturing the video to match the gaze of the driver.

4. The method of claim 3, further comprising:

determining that the gaze of the driver has changed; and

capturing subsequent frames of video to match the changed gaze of the driver.

5. The method of claim 1, further comprising:

generating a three-dimensional representation of a vehicle instrument cluster that includes a plurality of instruments;

outputting the three-dimensional representation of the vehicle instrument cluster at the first perceived depth; and

outputting the representation of the video at the second perceived depth.

6. The method of claim 1, wherein a distance between an object displayed in the representation of the video and the driver is based on a distance between the object and the vehicle.

7. The method of claim 1, further comprising causing at least one of the first perceived depth and the second perceived depth to be changed in response to at least one of a speed of the vehicle, a location of the vehicle, a change in a state of operability of the vehicle, a fuel or fluid level of the vehicle, and road conditions.

8. The method of claim 1, further comprising causing at least one of the first perceived depth and the second perceived depth to be changed in response to an object being located in a blind spot of the vehicle.

9. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to display live video in a vehicle, by performing the steps of:

capturing a video of an area external to the vehicle;

generating, via a stereoscopic display surface disposed within the vehicle, a plurality of display layers, wherein each display layer has a different perceived depth relative to a driver of the vehicle and the stereoscopic display surface;

causing one or more display elements to be displayed in a first stereoscopic image within a first display layer included in the plurality of display layers that has a first perceived depth relative to the driver of the vehicle and the stereoscopic display surface; and

causing a representation of the video to be displayed in a second stereoscopic image within a second display layer included in the plurality of display layers that has a second perceived depth relative to the driver of the vehicle and the stereoscopic display surface,

wherein the first perceived depth is different from the second perceived depth such that one display layer of the first display layer and the second display layer is disposed in front relative to the driver of the vehicle and is transparent to allow the driver of the vehicle to see through the one display layer.

10. The non-transitory computer-readable storage medium of claim 9, further comprising:

detecting a change in a viewing angle of the driver; and

adjusting a camera capturing the video to match the changed viewing angle of the driver.

11. The non-transitory computer-readable storage medium of claim 10, further comprising:

prior to adjusting the camera, determining that an amplitude of the change in the viewing angle exceeds a specified threshold.

12. The non-transitory computer-readable storage medium of claim 10, wherein the viewing angle of the driver is based on an orientation of a head of the driver.

13. The non-transitory computer-readable storage medium of claim 9, wherein the video is outputted as a three-dimensional video.

14. The non-transitory computer-readable storage medium of claim 9, wherein a distance between the video and the driver is based on a set of driver preferences.

15. A system to display live video in a vehicle, comprising:

one or more computer processors;

at least one camera configured to capture a video of an area external to the vehicle; and

a stereoscopic display surface disposed within the vehicle,

wherein the one or more computer processors execute instructions and perform the steps of;

generating, via the stereoscopic display surface disposed within the vehicle, a plurality of display layers, wherein each display layer has a different perceived depth relative to a driver of the vehicle and the stereoscopic display surface;

causing one or more display elements to be displayed in a first stereoscopic image within a first display layer included in the plurality of display layers that has a first perceived depth relative to the driver of the vehicle and the stereoscopic display surface; and

causing a representation of the video to be displayed in a second stereoscopic image within a second display layer included in the plurality of display layers that has a second perceived depth relative to the driver of the vehicle and the stereoscopic display surface,

wherein the first perceived depth is different from the second perceived depth such that one display layer of the first display layer and the second display layer is disposed in front relative to the driver of the vehicle and is transparent to allow the driver of the vehicle to see through the one display layer.

16. The system of claim 15, wherein the display comprises a stereoscopic, parallax-enabled three-dimensional display.

17. The system of claim 15, wherein the at least one camera comprises at least one of two robotic pan-tilt-shift cameras mounted on a front bumper of the vehicle, and an array of a plurality of fixed cameras mounted on the front bumper of the vehicle.

18. The system of claim 17, wherein the at least one camera comprises the two robotic pan-tilt-shift cameras, wherein the one or more computer processors perform the further steps of:

detecting a gaze of the driver; and

adjusting each camera in the set of cameras to capture the video to match the gaze of the driver.

19. The system of claim 17, wherein the set of cameras comprises the array of fixed cameras, wherein the one or more computer processors perform the further steps of

detecting a gaze of the driver;

and

adjusting a capture of the array of fixed cameras to capture the video to match the gaze of the driver.

20. The system of claim 19, wherein adjusting the capture of the array of fixed cameras comprises at least one of selecting two of the plurality of fixed cameras, and using a subset of an image data captured by two of the plurality of fixed cameras to create a composite view of the two of the plurality of fixed cameras.

21. The system of claim 17, wherein the one or more computer processors are configured to:

determine that a gaze of the driver has changed; and

modify the at least one camera to capture video based on the changed gaze of the driver.

22. The system of claim 15, further comprising:

a set of driver-facing sensors configured to detect a gaze of the driver.