VIDEO IMAGE PRESENTATION SYSTEM

• US 20080239078A1
• Filed: 11/16/2007
• Published: 10/02/2008
• Est. Priority Date: 11/21/2006
• Status: Active Grant

• Current Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Inventors: Mohr, Ulrich, Stiegler, Andreas
• Granted Patent: US 9,013,578 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A system for presenting video images from a vehicle environment includes taking video images of the vehicle environment, detecting sensor data of the vehicle environment or the vehicle, processing the sensor data, recognizing objects in the video images, and visualizing the processed sensor data in the displayed video images by changing the representation of recognized objects in the displayed video images.

Claims

1. A method for presenting video images from a vehicle environment outside of a detecting vehicle, the method comprising:

taking video images of the vehicle environment;

detecting sensor data of the vehicle environment or the detecting vehicle itself;

processing the sensor data;

recognizing objects in the video images; and

visualizing the processed sensor data in the displayed video images by changing the representation of recognized objects in the displayed video images.

2. The method of claim 1, where objects in the video images are detected and interpreted by post processing the video images and are classified in object classes.

3. The method of claim 1, where the representation of the recognized objects is changed by introducing an element in an object in the displayed video images, and where the element is actually not present in the vehicle environment as viewed by a driver of the detecting vehicle through a windscreen of the vehicle.

4. The method of claim 1, where the video images are taken from the vehicle environment in front of the detecting vehicle.

5. The method of claim 1, where the video mages are taken from the vehicle environment in back of the detecting vehicle.

6. The method of claim 1, where the step of recognizing objects includes the step of recognizing vehicles in the video images.

7. The method of claim 1, where the step of detecting sensor data includes the step of determining the distance of an object shown in the video images relative to the detecting vehicle from where the distance is determined.

8. The method of claim 7 where, when it is detected that the distance of the object relative to the detecting vehicle is decreasing, changing the representation of the object in the displayed video image to visually alert a driver of the detecting vehicle of the object.

9. The method of claim 1, where the step of detecting sensor data includes the step of determining the velocity of a vehicle in front of the detecting vehicle.

10. The method of claim 9, where, when it is detected that the velocity of the vehicle in front of the detecting vehicle is lower than a predetermined threshold velocity, changing the representation of the front vehicle by emphasizing visualization of braking lights of the front vehicle in the displayed video images.

11. The method of claim 9, where, when it is detected that the velocity of the vehicle in front of the detecting vehicle is lower than a predetermined threshold velocity, changing the representation of the front vehicle by adding flashing warning lights to the front vehicle in the displayed video images.

12. The method of claim 1, where the step of detecting sensor data includes the step of detecting the outside temperature and the outside moisture level, where, when the outside temperature is lower than a predetermined threshold temperature and when the moisture is greater than a predetermined threshold moisture, the representation of a road displayed in the video images is changed.

13. The method of claim 12, where the representation of the road is changed by adding reflections to the displayed video images.

14. The method of claim 1, where the step of detecting sensor data includes the step of determining the outside visibility, and when the visibility is lower than a predetermined threshold visibility and when the distance to the vehicle in front is greater than a predetermined distance, the representation of the vehicle in front of the detecting vehicle is changed.

15. The method of claim 1, where the step for detecting sensor data of the vehicle includes the step of determining the position of the vehicle and to locate the vehicle in a digital map and to emphasize objects in the video images depending on information from the map data.

16. The method of claim 15, where object information is retrieved from the map data, and the objects in the video images are emphasized depending on the received sensor data.

17. A system for displaying video images from a vehicle environment, the system comprising:

an image sensor taking video images from the environment of the vehicle;

at least one sensor detecting data of the vehicle environment or the vehicle itself;

a sensor processing unit processing the sensor data;

an image processing unit post-processing the video images and recognizing objects in the video images; and

a graphics processing unit visualizing the processed sensor data in the displayed video images by changing the representation of recognized objects in the displayed video images.

18. The system of claim 17, where the sensor detecting data of the vehicle environment is selected from the group consisting of a distance sensor, a sensor detecting the velocity of vehicles driving in front of the vehicle, a temperature sensor, a moisture level detecting sensor, and a position detecting sensor.

19. The system of claim 17, where the image processing unit recognizes objects in the video images by applying edge detection filters to the video images and comparing the detected objects to predetermined types of objects stored in a database.

20. A method for presenting video images from a vehicle environment outside of a detecting vehicle, the method comprising:

taking video images of the vehicle environment;

recognizing a vehicle in front of the detecting vehicle in the video images;

determining the velocity of the recognized vehicle;

processing the determined velocity; and

visualizing the processed velocity in the displayed video images by changing the representation of the recognized vehicle in the displayed video images, where, when it is detected that the velocity of the recognized vehicle is lower than a predetermined threshold velocity, braking lights of the recognized vehicle as shown in the video images are displayed in an activated state.

21. A method for presenting video images from a vehicle environment outside of a detecting vehicle, the method comprising:

taking video images of the vehicle environment;

recognizing a vehicle in front of the detecting vehicle in the video images;

determining the velocity of the recognized vehicle;

processing the determined velocity; and

visualizing the processed velocity in the displayed video images by changing the representation of the recognized vehicle in the displayed video images, where the representation of the recognized vehicle is changed in the video images by adding flashing warning lights to the recognized vehicle as shown in the video images, when it is detected that the velocity is lower than a predetermined threshold velocity.

22. A system for displaying video images from a vehicle environment outside of a detecting vehicle, the system comprising:

an image sensor taking video images from the vehicle environment;

at least one velocity sensor detecting the velocity of other vehicles in the vehicle environment;

a velocity sensor processing unit processing the velocity of the other vehicles;

an image processing unit post-processing the video images and recognizing the other vehicles in the video images; and

a graphics processing unit visualizing the processed sensor data in the displayed video images, where, when the velocity sensor detects that the velocity of a vehicle moving in front of the detecting vehicle is lower than a predetermined threshold velocity, the graphics processor displays braking lights of the front vehicle in an activated state.

23. A system for displaying video images from a vehicle environment outside of a detecting vehicle, the system comprising:

an image sensor taking video images from the vehicle environment;

at least one velocity sensor detecting the velocity of other vehicles in the vehicle environment;

a velocity sensor processing unit processing the velocity of the other vehicles;

an image processing unit post-processing the video images and recognizing the other vehicles in the video images; and

a graphics processing unit visualizing the processed sensor data in the displayed video images, where, when the velocity sensor detects that the velocity of a vehicle moving in front of the detecting vehicle is lower than a predetermined threshold velocity, the graphics processor adds flashing warning lights to the front vehicle in the displayed video images.

24. A method for presenting video images from a vehicle environment outside of a detecting vehicle to a driver of the detecting vehicle, the method comprising:

taking a video image of the vehicle environment;

recognizing a plurality of objects in the video image;

presenting the video image including the recognized objects on a display of the detecting vehicle;

detecting sensor data of the vehicle environment or the detecting vehicle itself by operating a sensor of detecting vehicle;

based on the detected sensor data, determining whether one or more of the detected sensor data is to be selected for communication to the driver via the presented video image;

if it is determined that one or more of detected sensor data is to be selected for communication to the driver, changing the representation of one or more of the recognized objects in the presented video image to visualize the selected sensor data.

25. The method of claim 24, where detecting sensor data includes detecting data selected from the group consisting of the velocity of the detecting vehicle, the position of the detecting vehicle, the velocity of another vehicle, the position of another vehicle or of one or more of the recognized objects, the distance from the detecting vehicle to another vehicle or to one or more of the recognized objects, a temperature of the vehicle environment, a moisture level of the vehicle environment, a visibility of the vehicle environment, and combinations of two or more of the foregoing.

26. The method of claim 24, where detecting sensor data includes operating a navigation unit of the detecting vehicle.

27. The method of claim 24, where detecting sensor data includes accessing digital map data.

28. The method of claim 24, where detecting sensor data includes determining the position of the detecting vehicle, and further including comparing the position of the detecting vehicle relative to one or more of the recognized objects, and where changing the representation of the one or more recognized objects is based on the relative position.

29. The method of claim 24, where changing the representation includes rendering the visualization of the one or more recognized objects such that the one or more recognized objects are presented in the video image more prominently than other objects presented in the video image.

30. The method of claim 24, where changing the representation includes adding an additional object to the displayed video image, and where the additional object is not actually present in the vehicle environment or not actually visible by the driver while viewing the vehicle environment from the detecting vehicle.

31. The method of claim 24, where changing the representation includes adding an element to one or more of the recognized objects presented in the video and where the element is not actually present in the vehicle environment or not actually visible by the driver while viewing the vehicle environment from the detecting vehicle.

32. The method of claim 24, where determining whether one or more of the detected sensor data is to be selected includes determining whether one or more of the detected sensor data falls outside of a predetermined value or range of values.