Apparatus and method for guiding driving information and vehicle system having the same

• US 10,640,039 B2
• Filed: 09/19/2017
• Issued: 05/05/2020
• Est. Priority Date: 06/09/2017
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Motors Corporation (Kia Corp.)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Choi, Jae Seob, Kim, Jun Hyung, Kang, In Wan
• Primary Examiner(s): Hossain, Farzana
• Application Number:
  US15/708,657
• Publication Number:
  US 20180354419A1
• Time in Patent Office: 959 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A driving information guide apparatus of a vehicle includes a controller and a processor. The controller is communicatively connected to at least one of a sensor system and a communication device of the vehicle, and receives driving information of the vehicle. The processor collects driving information received through the controller while the vehicle runs and recognizes a current driving state of the vehicle based on the driving information. The processor determines a view mode based on a speed of the vehicle and a driving direction of the vehicle, and view-transforms images taken around the vehicle by a camera system based on a position and an angle of a virtual camera corresponding to the determined view mode. The processor thereby generates a view transformation image, and displays state information corresponding to the recognized driving state of the vehicle through the view transformation image on a display screen of the vehicle.

Claims

1. A driving information guide apparatus of a vehicle comprising:

a controller communicatively connected to a display screen and at least one of a sensor system and a communication device, and receiving driving information of the vehicle through the at least one of the sensor system and the communication device; and

a processor communicatively connected to the controller and configured to;

collect driving information received through the controller and at least one of the sensor system and the communication device while the vehicle runs and recognize a current driving state of the vehicle based on the collected driving information;

determine a view mode based on a speed of the vehicle and a driving direction of the vehicle;

view-transform images around the vehicle, which are taken by a camera system, based on a position and an angle of a virtual camera corresponding to the determined view mode, and generate a view transformation image for display on the display communicatively connected to the processor through the controller; and

display state information corresponding to the recognized driving state of the vehicle through the view transformation image on the display screen communicatively connected through the controller,

wherein the processor is configured to display a degree of driving risk and direction information of the vehicle in the view transformation image based on state information according to an approach state of an obstacle or a nearby vehicle, state information according to, a position of a driving lane of the vehicle, and state information according to a lane change of the vehicle,

wherein the processor is configured to display a plurality of bars indicative of the state information and respectively positioned around the vehicle in the view transformation image, and to display a corresponding stage among multiple stages of the degree of driving risk through the bars, and

wherein the processor is configured to display the bars separately in directions around the vehicle and differently display one or more of a color, contrast, shape, or number of one or more bars in a direction in which a difference in driving risk occurred in accordance with the degree of the driving risk.

2. The driving information guide apparatus of claim 1, wherein the view mode is selected from among first, second, third, and fourth modes in which the position and the angle of the virtual camera with respect to forward and backward directions of the vehicle are differently set in accordance with the speed of the vehicle, and from among fifth and sixth modes in which a position and an angle of the virtual camera with respect to left and right directions are differently set in accordance with the driving direction of the vehicle.

3. The driving information guide apparatus of claim 2, wherein the processor is configured to determine a view transformation mode to a first mode in which a reference position and a reference angle of the virtual camera are set with respect to the forward and backward directions and the left and right directions in a case that the vehicle runs straightly at constant speed.

4. The driving information guide apparatus of claim 3, wherein the processor is configured to determine whether a view transformation event occurs in the speed and the driving direction of the vehicle during the first mode.

5. The driving information guide apparatus of claim 3, wherein, in a case that the vehicle runs at high speed exceeding a predetermined speed, the processor is configured to determine the view mode to be the second mode in which the position of the virtual camera is set behind the reference position of the virtual camera and the angle of the virtual camera is set to an angle smaller than the reference angle of the virtual camera with respect to the forward and backward directions of the vehicle.

6. The driving information guide apparatus of claim 3, wherein, in a case that the vehicle runs at low speed smaller than a predetermined speed, the processor is configured to determine the view mode to be the third mode or the fourth mode each in which the position of the virtual camera is set ahead of the reference position of the virtual camera and the angle of the virtual camera is set to an angle greater than the reference angle of the virtual camera with respect to the forward and backward directions of the vehicle.

7. The driving information guide apparatus of claim 3, wherein, in a case that a left turn signal of the vehicle is turned on or the vehicle leaves a driving lane by crossing a left line with respect to the driving direction, the processor is configured to determine the view mode to be the fifth mode in which the position of the virtual camera is set to the left of the reference position of the virtual camera and the angle of the virtual camera is set to an angle smaller than the reference angle to the left direction with respect to the left and right directions of the vehicle.

8. The driving information guide apparatus of claim 3, wherein, in a case that a right turn signal of the vehicle is turned on or the vehicle leaves a driving lane by crossing a right line with respect to the driving direction, the processor is configured to determine the view mode to be the sixth mode in which the position of the virtual camera is set to the right of the reference position of the virtual camera and the angle of the virtual camera is set to an angle smaller than the reference angle to the right direction with respect to the left and right directions of the vehicle.

9. The driving information guide apparatus of claim 2, wherein the processor is configured to continuously adjust the position and the angle of the virtual camera to provide a view in which the position and the angle of the camera is set between at least two set positions and angles of the first, second, third, and fourth modes, and wherein the processor continuously adjusts the position and the angle of the virtual camera based on a value obtained by interpolating parameter values corresponding to at least two modes among the first to sixth modes.

10. The driving information guide apparatus of claim 1, wherein the processor is configured to determine the view mode in response to an input signal input by a user.

11. The driving information guide apparatus of claim 1, wherein the processor is configured to display an image corresponding to the obstacle or the nearby vehicle through a corresponding position of the view transfonnation image based on the state information according to the approach state of the obstacle or the nearby vehicle and to display the state information.

12. The driving information guide apparatus of claim 1, wherein the processor is configured to display an image corresponding to a line in a direction in which the vehicle leaves the driving lane in the view transformation image based on the state information according to the position of the driving lane of the vehicle and to display guide route information.

13. The driving information guide apparatus of claim 1, wherein the processor is configured to display caution state information according to a lane entry in the view transformation image based on the state information according to the lane change of the vehicle and to display guide route information ahead of the vehicle.

14. The driving information guide apparatus of claim 1, wherein the processor is configured to output an alarm sound with the view transformation image depending on a degree of driving risk of the vehicle.

15. The driving information guide apparatus of claim 1, wherein the processor is configured to recognize at least one of the speed of the vehicle, a driving route, a lane change state, a position of driving lane, a lane departure state, an operation state of an autonomous driving mode, an obstacle approach state, a distance between the vehicle and the obstacle, or a relative speed between the vehicle and the obstacle.

16. The driving information guide apparatus of claim 1, wherein the processor is configured to collect the driving information using a plurality of sensors of the sensor system installed in the vehicle, or

through the communication device providing communications through a vehicle-to-vehicle (V2V) communication between the vehicle and a nearby vehicle.

17. A driving information guide method comprising:

obtaining images around a vehicle, which are taken by a camera system of the vehicle while the vehicle runs;

collecting driving information of the vehicle, including a speed of the vehicle and a driving direction of the vehicle, by a processor of the vehicle communicatively connected to the camera system of the vehicle, while the vehicle runs and identifying a current driving state of the vehicle based on the collected driving information;

determining, by the processor, a view mode based on the speed of the vehicle and the driving direction of the vehicle;

view-transforming the images around the vehicle, by the processor, based on a position and an angle of a virtual camera corresponding to the determined view mode to generate a view transformation image; and

displaying, on a display communicatively connected to the processor of the vehicle, state information corresponding to the recognized driving state of the vehicle through the view transformation image,

wherein the step of displaying state information comprises;

displaying a plurality of bars separately in directions around the vehicle,

displaying a degree of driving risk and direction information of the vehicle in the view transformation image based on state information according to an approach state of an obstacle or a nearby vehicle, state information according to a position of a driving lane of the vehicle, and state information according to a lane change of the vehicle, and

displaying the bars indicative of the state information and respectively positioned around the vehicle in the view transformation image, to display a corresponding stage among multiple stages of the degree of driving risk through the bars, and

differently displaying one or more of a color, contrast, shape, or number of one or more bars in a direction in which a difference in driving risk occurred in accordance with the degree of the driving risk.

18. A vehicle system comprising:

an interface configured to display driving state information of a vehicle through a display screen;

a camera system configured to take images around the vehicle while the vehicle runs;

a sensor system configured to measure a driving state of the vehicle while the vehicle runs;

a communication device configured to communicate with a nearby vehicle to receive information about the nearby vehicle; and

a driving information guide device configured to;

collect driving information of the vehicle while the vehicle runs to recognize a current driving state of the vehicle based on the collected driving information;

view-transform the images taken around the vehicle, which are taken by the camera system, based on a position and an angle of a virtual camera corresponding to a view mode determined depending on a speed of the vehicle and a driving direction of the vehicle to generate a view transformation image; and

display, on the display screen of the interface, state information corresponding to the recognized driving state of the vehicle through the view transformation image,

wherein the driving information guide device is configured to display a degree of driving risk and direction information of the vehicle in the view transformation image based on state information according to an approach state of an obstacle or a nearby vehicle, state information according to a position of a driving lane of the vehicle, and state information according to a lane change of the vehicle,

wherein the driving information guide device is configured to display a plurality of bars indicative of the state information and respectively positioned around the vehicle in the view transformation image, to display a corresponding stage among multiple stages of the degree of driving risk through the bars, and

wherein the driving information guide device is configured to display the bars separately in directions around the vehicle and to differently display one or more of a color, contrast, shape, or number of one or more bars in a direction in which a difference in driving risk occurred in accordance with the degree of the driving risk.