Visual guidance for vehicle navigation system

• US 9,459,113 B2
• Filed: 11/21/2008
• Issued: 10/04/2016
• Est. Priority Date: 11/21/2008
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Sadekar, Varsha
• Primary Examiner(s): Le, Michelle L
• Application Number:
  US12/276,050
• Publication Number:
  US 20100131197A1
• Time in Patent Office: 2,874 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams

Abstract

A system and method that provide a video-based vehicle navigation system. The system positions an arrow on the video display that shows the specific turning direction for the vehicle for route guidance purposes. To determine the proper position of the guidance arrow, the process determines a distance from the current vehicle position to the location where the vehicle needs to turn using any suitable information, such as GPS position, range measurements and map information. The process then positions the guidance arrow on the ground at the turning location in world coordinates, and projects the guidance arrow onto the image. The camera can be calibrated to the ground using various techniques, such as an online automatic calibration process that uses detected objects in the scene around the vehicle.

Claims

1. A method for identifying a turn for a vehicle in association with a vehicle navigation system, said method comprising:

determining a vehicle route having a vehicle destination;

providing a video image using a camera of a scene in front of the vehicle as the vehicle is traveling along the route;

automatically calibrating the camera using the video image from the camera so that the orientation of the video image relative to the ground is known;

taking range measurements with vehicle mounted equipment that measures the distance from the vehicle to objects;

determining a distance from a current vehicle position to a turning location along the route, where the determination includes using the range measurements; and

projecting a guidance arrow onto the video image that identifies where the driver should turn the vehicle at the turning location to follow the route, wherein projecting the guidance arrow onto the display includes first placing a virtual guidance arrow laying flat on the ground at the turning location in world coordinates.

2. The method according to claim 1 wherein determining the distance from the vehicle position to the turning location includes using a vehicle GPS position and navigation map information.

3. The method according to claim 1 wherein determining the range measurements includes using vehicle radar.

4. The method according to claim 1 wherein calibrating the camera includes using an offline calibration process.

5. The method according to claim 1 wherein calibrating the camera includes automatically calibrating the camera using an online calibration process.

6. The method according to claim 5 wherein calibrating the camera includes employing range measurements and GPS information to determine infrastructure and distances to objects in the image, associating two-dimensional image points with their three-dimensional positions based on object detection and determining a camera rotation matrix and a translation vector using a two-dimensional image point and a corresponding three-dimensional position.

7. The method according to claim 6 wherein calibrating the camera includes using GPS messages that include information identifying other vehicles including the license plate numbers of other vehicles.

8. The method according to claim 6 wherein calibrating the camera includes using vehicle-to-vehicle communications and vehicle-to-infrastructure communications.

9. The method according to claim 1 wherein providing a video image of a scene includes using a display screen in the vehicle.

10. The method according to claim 1 wherein providing a video image of a scene includes using a display screen on a windshield of the vehicle.

11. The method according to claim 1 wherein determining the range measurements uses a stereo-camera.

12. A method for identifying a turn for a vehicle in association with a vehicle navigation system, said method comprising:

determining a vehicle route having a vehicle destination;

providing a video image from a camera of a scene in front of the vehicle as the vehicle is traveling along the route;

automatically calibrating the camera using an online camera calibration process that uses the video image from the camera so that the orientation of the camera relative to the ground is known; and

projecting a guidance arrow onto the windshield that identifies where the driver should turn the vehicle at a turning location to follow the route, wherein projecting a guidance arrow onto the windshield includes determining a distance from a current vehicle position to the turning location using a vehicle GPS position, range measurements and/or navigation map information, wherein projecting the guidance arrow onto the windshield includes first placing a virtual guidance arrow laying flat on the ground at the turning location in world coordinates, and another camera inside the vehicle to monitor the position of the driver'"'"'s head so that the orientation and position of the arrow can be changed relative to how the driver is viewing the windshield.

13. The method according to claim 12 wherein determining the range measurements includes using vehicle radar and/or a stereo-camera.

14. The method according to claim 12 wherein calibrating the camera includes employing range measurements and GPS information to determine infrastructure and distances to objects in the image, associating two-dimensional image points with their three-dimensional positions based on object detection and determining a camera rotation matrix and a translation vector using a two-dimensional image point and a corresponding three-dimensional position.

15. A system for identifying a turn for a vehicle in association with a vehicle navigation system, said system comprising:

means for determining a vehicle route having a vehicle destination;

means for providing from a camera a video image of a scene in front of the vehicle as the vehicle is traveling along the route;

means for determining a distance from a current vehicle position to a turning location along the route;

means for automatically calibrating the camera to the ground using the video image from the camera; and

means for projecting a guidance arrow onto the video image that identifies where the driver should turn the vehicle at the turning location to follow the route, wherein projecting the guidance arrow onto the display includes first placing a virtual guidance arrow laying flat on the ground at the turning location in world coordinates.

16. The system according to claim 15 further comprising means for automatically calibrating the camera using an online calibration process, said means for calibrating the camera employing range measurements and GPS information to determine infrastructure and distances to objects in the image, associating two-dimensional image points with their three-dimensional positions based on object detection and determining a camera rotation matrix and a translation vector using a two-dimensional image point and a corresponding three-dimensional position.

17. The system according to claim 15 wherein the means for determining a distance from a current vehicle position to a turning location along the route uses a vehicle GPS position, range measurements and navigation map information.