Terrain visualization for a vehicle and vehicle driver

• US 10,063,836 B2
• Filed: 09/04/2012
• Issued: 08/28/2018
• Est. Priority Date: 09/06/2011
• Status: Active Grant

• Current Assignee: Jaguar Land Rover Limited (TATA Motors Limited)
• Original Assignee: Jaguar Land Rover Limited (TATA Motors Limited)
• Inventors: Popham, Thomas, Gaszczak, Anna
• Primary Examiner(s): Aynalem, Nathnael
• Application Number:
  US14/343,211
• Publication Number:
  US 20140247328A1
• Time in Patent Office: 2,184 Days
• Field of Search: 348 49, 348148, 348 47
• CPC Class Codes: B60R 2021/01259 Brakes
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 10/184 with wheel brakes
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2552/20 Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A visualization technique for a vehicle and a vehicle driver combines a forward facing video camera and a forward facing time of flight camera. The outputs of the camera are combined to provide the vehicle driver with an augmented reality display, whereby topographical features can be identified in relation to the vehicle even if out of line of sight of the driver.

Claims

1. A vehicle having a color camera for capturing a color image of a scene ahead of the vehicle, a visual display device for showing information from said scene to a driver of the vehicle, and a time of flight camera system adapted to repeatedly capture an electronic image of said scene for generating a three-dimensional model of the scene ahead of the vehicle, wherein the color image and information included in the three-dimensional model of the scene ahead of the vehicle are combined and displayed on said display device,

wherein a region of interest that is contained in said color image is visually enhanced on said display according to pre-determined criteria relating to one or more of size and shape of a topographical feature within the image of said region of interest to indicate the severity of the topographical feature in the region of interest according to the pre-determined criteria;

wherein a prominence of the visual enhancement of the region of interest indicates the severity of the topological features as determined by the information included in the three dimensional model, and wherein multiple visual enhancements are displayed in the region of interest in response to a detection of a plurality of topographical features, the multiple visual enhancements comprising at least two levels of prominence corresponding to different levels of severity of the associated topographical feature.

2. A vehicle according to claim 1, wherein said scene is displayed in plan.

3. A vehicle according to claim 1, wherein a point of view of said scene is variable.

4. A vehicle according to claim 3, wherein the point of view is selectable by the vehicle driver.

5. A vehicle according to claim 1, wherein a region of interest in said scene is visually enhanced on said display according to pre-determined criteria relating to the proximity of said region of interest.

6. A vehicle according to claim 1, wherein one or more of said criteria are driver selectable.

7. A vehicle according to claim 1, wherein attitude of said vehicle is determined for three-dimensional application of said criteria.

8. A vehicle according to claim 1, and having an electronically enabled braking system adapted to be activated automatically by a collision calculator.

9. A vehicle according to claim 8, wherein said calculator is adapted to be disabled by the vehicle driver.

10. A vehicle or method according to claim 1, wherein the region of interest is an obstacle.

11. A vehicle according to claim 1, wherein the topographical feature comprises a terrain feature, wherein the terrain feature comprises at least one of:

a rut, a gradient, a hole on the ground, a side clearance, and rocks.

12. A vehicle according to claim 1, wherein the visual enhancement is one of a shape displayed in the region of interest, a color displayed in the region of interest, and a pulsing technique displayed in the region of interest.

13. A vehicle according to claim 1, wherein the multiple visual enhancements comprise one of different colors and different pulsing techniques.

14. A method of visualizing a scene in front of a moving vehicle having a visual display device for the vehicle driver, the method comprising:

capturing from the vehicle a color representation of said scene;

capturing repeating image data of said scene in front of the vehicle from a vehicle mounted time of flight camera system;

generating a three-dimensional model of said scene in front of the vehicle from said repeating image data;

combining information included in said model and said image of said scene in front of the vehicle to give an augmented representation of said scene on said display;

visually enhancing a region of interest in said scene that is contained in said color representation of said scene on said display according to pre-determined criteria relating to one or more of size and shape within the image of said region of interest to indicate the severity of a topographical feature in the region of interest according to the pre-determined criteria, wherein a prominence of the visual enhancement of the region of interest indicates the severity of the topological features as determined by the information included in the model, wherein multiple visual enhancements are displayed in the region of interest in response to a detection of a plurality of topographical features, the multiple visual enhancements comprising at least two levels of prominence corresponding to different levels of severity of the associated topographical feature.

15. A method according to claim 14, and including the step of displaying said augmented representation in plan.

16. A method according to claim 14, and including the step of providing a selectable point of view of said augmented representation.

17. A method according to claim 14, and including the step of zooming said augmented representation to a selected enlargement.

18. A method according to claim 14, and including the step of determining the attitude of the vehicle in selecting the region of interest to be enhanced.

19. A method according to claim 14, wherein the topographical feature comprises a terrain feature, wherein the terrain feature comprises at least one of:

a rut, a gradient, a hole on the ground, a side clearance, and rocks.

20. A method according to claim 14, the method comprising visually enhancing a region of interest in said image on said display according to pre-determined criteria relating to position within the image of said region of interest.

21. A method according to claim 14, wherein the multiple visual enhancements comprise one of different colors and different pulsing techniques.