Movable-Body Navigation Information Display Method and Movable-Body Navigation Information Display Unit

• US 20080195315A1
• Filed: 09/27/2005
• Published: 08/14/2008
• Est. Priority Date: 09/28/2004
• Status: Active Grant

• Current Assignee: Kumamoto University
• Original Assignee: Kumamoto University
• Inventors: Uchimura, Keiichi, Hu, Zhencheng
• Granted Patent: US 8,195,386 B2
• Time in Patent Office: Days
• US Class Current: 701/212
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G08G 1/096861 where the immediate route instructions are output to the driver, e.g. arrow signs for next turn
• CPC Class Codes: G08G 1/0969 having a display in the form of a map

Abstract

A movable-body navigation information display unit is provided. In the movable-body navigation information display unit, a driver can intuitively and accurately recognize a relation between navigation information and a real picture or a real landscape. In addition, it is possible to avoid a state that visibility of a caution-needed picture such as a pedestrian in the real picture and a real picture of a road construction site is inhibited by an image of the navigation information. An image data creating section (405) matches road shape data with a road shape model to estimate posture data. In addition, the image data creating section creates picture (image) data for accurately compositing and displaying the image of the navigation information in an appropriate position in a real picture (or in a real landscape) of a road ahead of a movable body, and displays the navigation information as a three-dimensional icon or the like. A picture display section (5) performs display based on the picture data.

Claims

1-24. -24. (canceled)

25. A movable-body navigation information display method comprising the steps of:

detecting a current position of a movable body, and capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data;

creating a road shape model of a road assumed to be captured from the current position, based on the current position of the movable body and the road map data;

extracting road shape data as image data of the road included in the landscape from the real picture;

matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, and determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data; and

displaying an image in which the read navigation information is composited with display dimensions and tone set by a user in the determined position in the captured real picture.

31. The movable-body navigation information display method according to claim 25, wherein the road shape data is converted to image data in a perspective two-dimensional characteristics space and the data of the road shape model is converted to image data in the perspective two-dimensional characteristics space, and the two-dimensional road shape data is matched with the two-dimensional data of the road shape model in the two-dimensional characteristics space to estimate the posture data of the in-vehicle camera or the movable body to the road as the object.

32. The movable-body navigation information display method according to claim 31, wherein in creating the road shape model, the road shape model of a multilane road is created by performing modeling while considering a road structure in which one-way grade of a traveling road surface in a curved road is set to change according to a horizontal curvature of the curved road.

33. The movable-body navigation information display method according to claim 31, wherein an existence probability of a road white line included in the landscape is obtained from the real picture and an RSL value is calculated by using a road look-up table (RSL), the posture data is obtained so that an evaluation value based on the RSL value becomes the maximum, and thereby the data of the road shape model is matched with the road shape data.

34. The movable-body navigation information display method according to claim 31, wherein the navigation information is at least one of a course guide of a route to reach a destination of the movable body, a position of a vehicle, a driving lane of the vehicle, and a building as an eyemark for a driver of the movable body to check the course guide or the position of the vehicle.

26. A movable-body navigation information display method comprising the steps of:

detecting a current position of a movable body, and capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data;

creating a road shape model of a road assumed to be captured from the current position, based on the current position of the movable body and the road map data;

extracting road shape data as image data of the road included in the landscape from the real picture;

matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, and determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data; and

previously defining three classifications of warning information, guidance information, and additional information depending on urgency of display necessity for the navigation information, previously determining priority for each classification, determining to which classification the navigation information corresponds, and compositing and displaying the navigation information in the determined position in the captured real picture according to an order corresponding to the classifications,

wherein the navigation information other than the warning information is displayed on and after when the current position of the movable body to a warning target in the real picture enters within a given distance.

27. A movable-body navigation information display method comprising the steps of:

detecting a current position of a movable body, and capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data;

creating a road shape model of a road assumed to be captured from the current position, based on the current position of the movable body and the road map data;

extracting road shape data as image data of the road included in the landscape from the real picture;

matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, and determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data; and

displaying an image in which the read navigation information is composited in the determined position in the captured real picture, and displaying the navigation information other than graphic display for guiding the traveling direction not in a region under a virtual vanishing point or a virtual horizon line in the real picture but in a region above the virtual vanishing point or the virtual horizon line.

28. A movable-body navigation information display method comprising the steps of:

detecting a current position of a movable body, and capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data;

creating a road shape model of a road assumed to be captured from the current position, based on the current position of the movable body and the road map data;

extracting road shape data as image data of the road included in the landscape from the real picture;

matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, and determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data; and

compositing and displaying the read navigation information as a three-dimensional icon that rotates centering on a vertical axis or a horizontal axis in the determined position in the captured real picture,

wherein when a plurality of the rotatable three-dimensional icons are displayed in one screen, each of the plurality of rotatable three-dimensional icons is rotated at a phase different from each other and at a rotational speed different from each other.

29. A movable-body navigation information display method comprising the steps of:

detecting a current position of a movable body, and capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data;

creating a road shape model of a road assumed to be captured from the current position, based on the current position of the movable body and the road map data;

extracting road shape data as image data of the road included in the landscape from the real picture;

matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, and determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data; and

displaying an image of the read navigation information as a translucent icon through which the real picture is translucently viewed in the determined position in the captured real picture.

30. A movable-body navigation information display method comprising the steps of:

detecting a current position of a movable body, and capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data;

creating a road shape model of a road assumed to be captured from the current position, based on the current position of the movable body and the road map data;

extracting road shape data as image data of the road included in the landscape from the real picture;

matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, and determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data; and

displaying an image in which the read navigation information is composited in the determined position in the captured real picture, and displaying character information of the navigation information in a display region preset outside the real picture.

35. A movable-body navigation information display unit comprising:

a current position detecting means for detecting a current position of a movable body;

a capturing means for capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

a data processing means for reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data, creating a road shape model of a road assumed to be captured from the current position based on the current position of the movable body and the road map data, extracting road shape data as image data of the road included in the landscape from the real picture, matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data, and outputting data for displaying an image in which the read navigation information is composited as an image displayed with display dimensions and tone set by a user in the determined position in the captured real picture; and

an image display means for compositing and displaying the read navigation information as the image displayed with the display dimensions and the tone set by the user in the determined position of the captured real picture based on the data outputted from the data processing means.

41. The movable-body navigation information display unit according to any one of claims 35, wherein the data processing means converts the road shape data to image data in a perspective two-dimensional characteristics space and converts the data of the road shape model to image data in the perspective two-dimensional characteristics space, and matches the two-dimensional road shape data with the two-dimensional data of the road shape model in the two-dimensional characteristics space to estimate the posture data of the in-vehicle camera or the movable body to the road as the object.

42. The movable-body navigation information display unit according to claim 41, wherein in creating the road shape model, the data processing means creates the road shape model of a multilane road by performing modeling while considering a road structure in which one-way grade of a traveling road surface in a curved road is set to change according to a horizontal curvature of the curved road.

43. The movable-body navigation information display unit according to claim 41, wherein the data processing means matches the data of the road shape model with the road shape data by obtaining an existence probability of a road white line included in the landscape from the real picture to calculate an RSL value with the use of a road look-up table (RSL), and obtaining the posture data so that an evaluation value based on the RSL value becomes the maximum.

44. The movable-body navigation information display unit according to claim 41, wherein the navigation information is at least one of a course guide of a route to reach a destination of the movable body, a position of a vehicle, a driving lane of the vehicle, a building as an eyemark for a driver of the movable body to check the course guide or the position of the vehicle.

36. A movable-body navigation information display unit comprising:

a current position detecting means for detecting a current position of a movable body;

a capturing means for capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

a data processing means for reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data, creating a road shape model of a road assumed to be captured from the current position based on the current position of the movable body and the road map data, extracting road shape data as image data of the road included in the landscape from the real picture, matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data, previously defining three classifications of warning information, guidance information, and additional information depending on urgency of display necessity for the navigation information, previously determining priority for each classification, determining to which classification the navigation information corresponds, and outputting data for compositing and displaying the navigation information as an image in the determined position in the captured real picture according to an order corresponding to the classifications and;

an image display means for compositing and displaying the image of the read navigation information in the determined position of the captured real picture according to the order corresponding to the classifications, based on the data outputted from the data processing means,

wherein the data processing means outputs data for displaying the navigation information other than the warning information on and after when the current position of the movable body to a warning target in the real picture enters within a given distance.

37. A movable-body navigation information display unit comprising:

a current position detecting means for detecting a current position of a movable body;

a capturing means for capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

a data processing means for reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data, creating a road shape model of a road assumed to be captured from the current position based on the current position of the movable body and the road map data, extracting road shape data as image data of the road included in the landscape from the real picture, matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data, and outputting data for compositing and displaying the navigation information other than graphic display for guiding the traveling direction not in a region under a virtual vanishing point or a virtual horizon line in the real picture but in a region above the virtual vanishing point or the virtual horizon line; and

an image display means for compositing and displaying the read navigation information in the determined position of the captured real picture based on the data outputted from the data processing means.

38. A movable-body navigation information display unit comprising:

a current position detecting means for detecting a current position of a movable body;

a capturing means for capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

a data processing means for reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data, creating a road shape model of a road assumed to be captured from the current position based on the current position of the movable body and the road map data, extracting road shape data as image data of the road included in the landscape from the real picture, matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data, and outputting data for displaying an image in which the read navigation information is composited as a three-dimensional icon that rotates centering on a vertical axis or a horizontal axis in the determined position in the captured real picture; and

an image display means for compositing and displaying the read navigation information as the three-dimensional icon that rotates centering on the vertical axis or the horizontal axis in the determined position in the captured real picture based on the data outputted from the data processing means,

wherein when a plurality of the rotatable three-dimensional icons are displayed in one screen, the data processing means outputs data for displaying and rotating each of the plurality of three-dimensional icons at a phase different from each other and at a rotational speed different from each other.

39. A movable-body navigation information display unit comprising:

a current position detecting means for detecting a current position of a movable body;

a capturing means for capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

a data processing means for reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data, creating a road shape model of a road assumed to be captured from the current position based on the current position of the movable-body and the road map data, extracting road shape data as image data of the road included in the landscape from the real picture, matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data, and outputting data for displaying the read navigation information as an image composited as a translucently displayed image in the determined position in the captured real picture; and

an image display means for compositing and displaying the read navigation information as the translucently displayed image in the determined position in the captured real picture based on the data outputted from the data processing means.

40. A movable-body navigation information display unit comprising:

a current position detecting means for detecting a current position of a movable body;

a capturing means for capturing a real picture with an object of a landscape including a road in a traveling direction of the movable body by an in-vehicle camera set in the movable body;

a data processing means for reading navigation information on travel of the movable body corresponding to the detected current position of the movable body from navigation information previously stored in association with road map data, creating a road shape model of a road assumed to be captured from the current position based on the current position of the movable body and the road map data, extracting road shape data as image data of the road included in the landscape from the real picture, matching data of the road shape model with the road shape data to estimate posture data of the in-vehicle camera or the movable body to the road as the object, determining a display position in the captured real picture of the navigation information read according to the current position of the movable body based on the posture data, and outputting data for compositing and displaying the read navigation information in the determined position in the captured real picture and for displaying character information of the navigation information not in the real picture but in a display region preset outside the real picture; and

an image display means for compositing and displaying the read navigation information in the determined position in the captured real picture and for displaying the character information of the navigation information not in the real picture but in the display region preset outside the real picture, based on the data outputted from the data processing means.