Free space monitoring system for motor vehicles

• US 20040220724A1
• Filed: 04/12/2004
• Published: 11/04/2004
• Est. Priority Date: 04/11/2003
• Status: Active Grant

• Current Assignee: Bayerische Motoren Werke AG
• Original Assignee: Daimler Chrysler Company LLC (Stellantis NV)
• Inventors: Hahn, Stefan, Toepfer, Christian Friedrich
• Granted Patent: US 7,136,754 B2
• Time in Patent Office: Days
• US Class Current: 701/200
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/8086 for vehicle path indication
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image

Abstract

In complex vehicle maneuvers it is often difficult for the driver of the motor vehicle to estimate along which tract the vehicle will move and how much free space is necessary in order to avoid collision. A process is provided for monitoring the environment space in the direction of travel of the own vehicle, in which image data of the environment around the vehicle in the direction of travel of the vehicle is recorded using a camera system the three-dimensional free space required for unimpeded travel is calculated in advance in a signal processing unit and the vehicle operator is informed regarding whether or not a sufficient free space is available to him for unimpeded travel.

Claims

1-21. -21. (Cancelled)

22. A process for monitoring the free space in the direction of travel of a vehicle, comprising

acquiring image data of the vehicle environment in the direction of travel using a camera system,

additionally, on the basis of the operating parameters and dimensions of the vehicle, pre-calculating the three dimensional free space required for unimpeded travel of the vehicle in a signal processing unit, and

displaying to the operator of the vehicle at least the part of the detected image data corresponding to the pre-calculated required free space,

wherein the display of the image data occurs essentially as it would appear from the perspective of the operator of the vehicle directly viewing the vehicle environment in the direction of travel,

wherein at least the image data associated with the three dimensional free space is subjected to a further processing, and

wherein as a result of this further processing the vehicle operator is informed with regard to whether or not sufficient free space is available for unimpeded travel.

23. A process according to claim 22, further comprising taking into consideration,dynamic changes over time taking place in the image data during further processing.

24. A process according to claim 22, wherein the informing of the vehicle operator in the case that sufficient free space is not available for unimpeded travel occurs by warning means, wherein said warning means are optical, acoustic or haptic perceivable signal providers.

25. A process according to claim 22, wherein the informing of the vehicle operator in the case that sufficient free space is no longer available for an unimpeded travel occurs by pictorial display of at least a part of the further processed image data, wherein, in these image data, those objects which are located in an impeding manner in the free space required for the travel are represented enhanced or highlighted above the other image objects.

26. A process according to claim 25, wherein the enhancement occurs in such a manner that the objects which are located in an impeding manner in the free space required for the travel have symbols superimposed thereupon, which symbols represent the direction of travel and/or speed of movement of said objects.

27. A process according to claim 22, wherein the informing of the vehicle operator with regard to the existence of a sufficient free space occurs by the pictorial display of at least a part of the further processed image data,

wherein the displayed image data has superimposed a symbolic representation of the edge boundaries of the required free space as determined by the pre-calculations in the signal processing unit.

28. A process according to claim 27, wherein the symbolic representation of the edge boundaries occurs by symbolic representation of walls.

29. A process according to claim 27, wherein the symbolic display of the edge boundaries occurs by the symbolic representation of walls and a ceiling area and/or floor area, forming a tube or tunnel.

30. A process according to claim 27, wherein the edge boundaries are visualized by the display of a grid, a sequence of columns or a checkerboard pattern.

31. A process according to claim 22, wherein the wall area and/or ceiling area and/or the floor area is so designed that it

has a geometric distinct and unambiguously recognizable structure,

permits a “

through view”

to the video scene lying therebehind, and

is so calculated, that it appears to be standing still even while the vehicle is moving in the longitudinal direction.

32. A process according to claim 22, further comprising recording the scene displayed to the operator of the vehicle and intermediately storing such for at least a specified elapsed time interval or for a particular traveled-through vehicle path, such that said stored information can be called up by the vehicle operator for repeated display.

33. A process according to claim 27, wherein the camera image data having superimposed thereon the edge boundaries of the required free space are supplied to a video processing process based on optical flow, which compares the movement of the spatially-fixed edge boundaries in the image data with the actual movement of objects in the image,

wherein those objects in the image, for which the model flow is less than the actual measured flow, are assessed as potential collision objects.

34. A process according to claim 33, wherein in the case of the presence of a potential collision object, this object is optically accentuated or highlighted, for example with color, and/or results in a warning being given to the vehicle operator via a warning signal.

35. A device for monitoring the free space in the direction of travel of a vehicle, including:

a camera system for acquiring image data from the vehicle environment in the direction of travel of the vehicle,

a signal processing unit, which is in communication with sensors for detecting the operating parameters of the vehicle and which includes a memory in which data is recorded regarding the dimensions of the vehicle,

a display unit in communication with the signal processing unit, for display of at least that part of the image data representing the required three-dimensional free space,

wherein the memory associated with the signal processing unit is so designed and arranged, that a number data sets are recorded in the memory regarding different vehicle dimensions, each of which can be specifically selected and made available to the signal processing unit, and/or the memory is provided with a means via which the at least one data set stored in memory can be changed.

36. A device according to claim 35, wherein the means for specific selection of a set of data stored in memory include a keypad operable by the vehicle operator.

37. A device according to claim 35, wherein the means for changing the at least one set of data stored in memory includes a communication device for the wireless transmission of data.

38. A device according to claim 35, wherein the camera system includes at least one catadioptric camera.

39. A device according to claim 35, wherein the device includes a memory for intermediate storage of at least a part of the image data displayed to the operator of the vehicle, and wherein means is provided via which the vehicle operator can call up this intermediate stored image data for repeated viewing.

40. A device according to claim 35, wherein the device includes a unit for image processing on the basis of optical flow, via which the image data having the edge boundaries of the required free space superimposed thereon can be processed to the extent that the movement of the positionally fixed edge boundaries in the image can be compared with the actual movement of objects in the image, so that those locations in the image, at which the model flow is less than the actual measured flow, can be assessed as potential collision objects.

41. A device according to claim 40, wherein the device includes a unit for image processing that is in communication with a display for the vehicle operator and/or a warning signal generator,

such that a potential collision object can be accentuated in the represented image or the vehicle operator can be warned of these objects.

42. A process for monitoring the space behind a vehicle during backing-up or repositioning of the vehicle, comprising:

acquiring image data of the vehicle environment in the direction of travel using a camera system,

additionally, on the basis of the operating parameters and dimensions of the vehicle, pre-calculating the three dimensional free space required for unimpeded travel of the vehicle in a signal processing unit, and

displaying to the operator of the vehicle at least the part of the detected image data corresponding to the pre-calculated required free space,

wherein the display of the image data occurs essentially as it would appear from the perspective of the operator of the vehicle directly viewing the vehicle environment in the direction of travel,

wherein at least the image data associated with the three dimensional free space is subjected to a further processing, and

wherein as a result of this further processing the vehicle operator is informed with regard to whether or not sufficient free space is available for unimpeded travel.

43. A device for monitoring the space behind a vehicle during backing-up or repositioning of the vehicle,

said device including:

a camera system for acquiring image data from the vehicle environment in the direction of travel of the vehicle,

a signal processing unit, which is in communication with sensors for detecting the operating parameters of the vehicle and which includes a memory in which data is recorded regarding the dimensions of the vehicle,

a display unit in communication with the signal processing unit, for display of at least that part of the image data representing the required three-dimensional free space,

wherein the memory associated with the signal processing unit is so designed and arranged, that a number data sets are recorded in the memory regarding different vehicle dimensions, each of which can be specifically selected and made available to the signal processing unit, and/or the memory is provided with a means via which the at least one data set stored in memory can be changed.