VEHICLE DISPLAY DEVICE, VEHICLE DISPLAY METHOD, AND RECORDING MEDIUM FOR DISPLAYING IMAGES

• US 20180319337A1
• Filed: 07/13/2018
• Published: 11/08/2018
• Est. Priority Date: 05/10/2016
• Status: Active Grant

• Current Assignee: JVC Kenwood Corporation
• Original Assignee: JVC Kenwood Corporation
• Inventors: NAGAI, Katsuyuki
• Granted Patent: US 10,703,271 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 21/00 Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks safety belts or body harnesses in vehicles B60R22/00; devices, apparatus or methods for life-saving in general A62B; safety devices for propulsion unit control specially adapted for, or arranged in, vehicles B60K28/00; seats constructed to protect the occupant from the effect of abnormal g-forces, e.g. crash or safety seats, B60N2/42; energy-absorbing arrangements for hand wheels for steering vehicles B62D1/11; energy-absorbing arrangements for vehicle steering columns B62D1/19; harnessing in aircraft B64D25/00
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G01B 11/026 by measuring distance between sensor and object G01B11/0608 takes precedence
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 17/00 Measuring arrangements characterised by the use of infrasonic, sonic or ultrasonic vibrations
• CPC Class Codes: G01B 17/06 for measuring contours or curvatures
• CPC Class Codes: G01S 15/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurement G01S15/32
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 15/87 Combinations of sonar systems
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 : Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/931 : of land vehicles
• CPC Class Codes: G01S 7/4808 : Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/539 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

An image acquisition unit acquires an image around a vehicle. A generator generates a bird'"'"'s-eye image based on the acquired image. A road surface information acquisition unit acquires information on a road surface around the vehicle. A road surface determination unit determines whether a road surface that requires attention is found by referring to the acquired information on the road surface. When it is determined that a road surface that requires attention is found, an image processor superimposes an image showing the road surface that requires attention on the bird'"'"'s-eye image.

Claims

1. A vehicle display device comprising:

an image acquisition unit that acquires an image around a vehicle;

a generator that subjects the image acquired by the image acquisition unit to viewpoint transform to generate a bird'"'"'s-eye image as viewed from above the vehicle;

a road surface information acquisition unit that acquires information on a road surface shape or a road surface condition around the vehicle;

a road surface determination unit that determines whether a road surface shape or a road surface condition that requires attention is found by referring to the information on the road surface shape or the road surface condition acquired by the road surface information acquisition unit;

an image processor that, when the road surface determination unit determines that a road surface shape or a road surface condition that requires attention is found, superimposes an image showing the road surface shape or the road surface condition that requires attention on the bird'"'"'s-eye image; and

a display controller that causes a display unit to display the bird'"'"'s-eye image on which the image is superimposed by the image processor.

2. The vehicle display device according to claim 1, wherein

the road surface information acquisition unit acquires the information on a road surface shape by acquiring a distance from a body of the vehicle to a road surface in a range in which it is possible to acquire the information on the road surface shape.

3. The vehicle display device according to claim 1, further comprising:

a traveling condition acquisition unit that acquires a traveling condition of the vehicle, wherein

the road surface information acquisition unit acquires the information on the road surface shape around the vehicle when the traveling condition acquired by the traveling condition acquisition unit indicates that the vehicle is traveling at a speed less than a predetermined speed or the vehicle is at a stop.

4. The vehicle display device according to claim 1, wherein

the road surface determination unit determines whether a road surface shape that requires attention is found by determining whether a road surface shape having a step height that makes it difficult for the vehicle to travel is found.

5. The vehicle display device according to claim 4, wherein

the road surface determination unit determines whether a step shaped to descend in a direction away from the vehicle is found to identify a road surface shape having a step height that makes it difficult for the vehicle to travel.

6. The vehicle display device according to claim 1, wherein

the road surface determination unit determines whether a road surface shape that requires attention is found by determining whether a road surface shape having a step height that makes it difficult to open a door of the vehicle is found.

7. The vehicle display device according to claim 6, further comprising:

a seating information acquisition unit that acquires seated/non-seated information on a plurality of seats in the vehicle, wherein

the road surface determination unit determines whether a road surface shape having a step height that makes it difficult to open one or more of the doors associated with the seat acquired by the seating information acquisition unit as being occupied is found.

8. The vehicle display device according to claim 1, further comprising:

a stop determination unit that determines that the vehicle is at a stop when the vehicle comes to a stop or when the speed of the vehicle drops below a predetermined speed, wherein

the image processor superimposes an image showing a road surface condition that requires attention on the bird'"'"'s-eye image when the stop determination unit determines that the vehicle is at a stop and when the road surface determination unit determines that a road surface condition that requires attention is found.

9. The vehicle display device according to claim 8, further comprising:

a relative position determination unit that determines, when the road surface determination unit determines that a road surface condition that requires attention is found, determines whether the road surface condition that requires attention is positioned near a door of the vehicle by referring to relative positions of the road surface condition that requires attention and the door of the vehicle, wherein

the generator adds a driver'"'"'s vehicle icon indicating a position of the vehicle to the bird'"'"'s-eye image, and

when the relative position determination unit determines that the road surface condition that requires attention is positioned near the door of the vehicle, the image processor superimposes the image showing the road surface condition that requires attention on a position of the door of the vehicle in the driver'"'"'s vehicle icon in the bird'"'"'s-eye image.

10. The vehicle display device according to claim 8, wherein

the road surface information acquisition unit acquires a condition of ultrasonic reflection on a road surface to identify the road surface condition, and

the road surface determination unit determines whether the road surface is wet by referring to the condition of ultrasonic reflection on the road surface acquired by the road surface information acquisition unit.

11. The vehicle display device according to claim 8, wherein

the road surface information acquisition unit acquires shade change in the bird'"'"'s-eye image to identify the road surface condition, and

the road surface determination unit determines whether the road surface is wet by referring to the shade change acquired by the road surface information acquisition unit.

12. The vehicle display device according to claim 8, further comprising:

a seating information acquisition unit that acquires seated/non-seated information on a plurality of seats in the vehicle, wherein

the image processor superimpose an image showing the road surface condition that requires attention on a position of the one or more of doors of the vehicle associated with the seat that is occupied.

13. A vehicle display method comprising:

acquiring an image around a vehicle;

subjecting the image acquired to viewpoint transform to generate a bird'"'"'s-eye image as viewed from above the vehicle;

acquiring information on a road surface shape or a road surface condition around the vehicle;

determining whether a road surface shape or a road surface condition that requires attention is found by referring to the information on the road surface shape or the road surface condition acquired;

when it is determined that a road surface shape or a road surface condition that requires attention is found, superimposing an image showing the road surface shape or the road surface condition that requires attention on the bird'"'"'s-eye image; and

causing a display unit to display the bird'"'"'s-eye image on which the image is superimposed.

14. A recording medium encoded with a program comprising computer-implemented modules including:

acquiring an image around a vehicle;

subjecting the image acquired to viewpoint transform to generate a bird'"'"'s-eye image as viewed from above the vehicle;

acquiring information on a road surface shape or a road surface condition around the vehicle;

determining whether a road surface shape or a road surface condition that requires attention is found by referring to the information on the road surface shape or the road surface condition acquired;

when it is determined that a road surface shape or a road surface condition that requires attention is found, superimposing an image showing the road surface shape or the road surface condition that requires attention on the bird'"'"'s-eye image; and

causing a display unit to display the bird'"'"'s-eye image on which the image is superimposed.