Method and apparatus to estimate vehicle position and recognized landmark positions using GPS and camera

• US 20100176987A1
• Filed: 01/15/2009
• Published: 07/15/2010
• Est. Priority Date: 01/15/2009
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Hoshizaki, Takayuki
• Granted Patent: US 7,868,821 B2
• Time in Patent Office: Days
• US Class Current: 342/357.230
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system

Abstract

A navigation system with a capability of estimating positions of the platform vehicle and recognized landmarks using satellite information and camera measurements is disclosed. One aspect is to represent a landmark by one or few representative features by executing image recognition and feature extraction to find representative features inside the recognized landmark. Another aspect is to re-identify the recognized landmarks in the landmark database correlating the landmark attributes obtained by image recognition and mathematical feature characteristics obtained by feature extraction with the corresponding data in the landmark database. A further aspect is to enhance navigation accuracy through the Kalman filter architecture with additional imagery measurements of landmarks whose positions are known or previously estimated. According to the aspects noted above: (1) navigation accuracy improves with augmented imagery measurements; (2) a database of recognizable landmark attributes and associated feature characteristics can be created in real-time; (3) by re-visiting the same landmarks, navigation accuracy improves and the landmark database will be re-calibrated and augmented with new landmark data.

Claims

1. A method of estimating a platform position with high accuracy, comprising the following steps of:

receiving GPS signals from satellites and processing the GPS signals for producing position estimation information;

obtaining image data of landmarks surrounding the platform through a camera mounted on the platform;

conducting an image recognition process on the image data thereby extracting attributes and feature characteristics of the landmarks;

registering the attributes and feature characteristics of the landmarks in a landmark database thereby establishing and updating the landmark database; and

incorporating the attributes and feature characteristics of the landmarks into the position estimation information produced from the GPS signals, thereby improving accuracy of the position estimate.

2. The method of estimating the platform position as defined in claim 1, further comprising a step of:

comparing the attributes and feature characteristics of the landmark with data stored in the landmark database to check if landmarks recognized through the camera are registered in the landmark database as known or estimated data.

3. The method of estimating the platform position as defined in claim 2, wherein said step of comparing the attributes and feature characteristics of the landmark with data stored in the landmark database includes a step of re-identifying the recognized landmarks in the landmark database correlating the landmark attributes obtained by image recognition and mathematical feature characteristics obtained by feature extraction with the corresponding data in the landmark database.

4. The method of estimating the platform position as defined in claim 3, wherein said landmark attributes include position, type, color, facing streets, etc. of the landmark, and said mathematical feature characteristics include edge curvature gradient of the landmark.

5. The method of estimating the platform position as defined in claim 2, wherein said step of comparing the attributes and feature characteristics of the landmark with data stored in the landmark database includes a step of selecting known or estimated data of the landmarks with higher estimation accuracy than that of the other for incorporating the selected data into the position estimation information produced from the GPS signals.

6. The method of estimating the platform position as defined in claim 1, wherein said attributes and feature characteristics of the landmark extracted via the image recognition process represent the landmark by one or more representative features inside the recognized landmark

7. The method of estimating the platform position as defined in claim 1, wherein said step of incorporating the attributes and feature characteristics of the landmarks into the position estimation information produced from the GPS signals includes a step of executing a Kalman filter process with additional imagery measurements of landmarks whose positions are known or previously estimated, thereby enhancing the accuracy of the position estimate.

8. The method of estimating the platform position as defined in claim 1, wherein a plurality of different frequencies are used for conducting the steps:

a high frequency to update the position estimates in a dead reckoning condition;

a middle frequency to correct the position estimates based upon the image data obtained through the camera, and a low frequency to correct the position estimates based upon the GPS signals.

9. The method of estimating the platform position as defined in claim 1, wherein said landmark database is updated in real-time by registering the landmark attributes including position, type, color, facing streets, etc. of the landmark, and the feature characteristics including edge curvature gradient of the landmark.

10. The method of estimating the platform position as defined in claim 1, wherein said step of registering the attributes and feature characteristics of the landmarks in the landmark database includes a step of re-calibrating the landmark database by starting landmark position estimation with lastly estimated position registered in the landmark database as an initial condition as well as changing a choice of known landmarks to be used for navigation accuracy enhancement.

11. An integrated imagery/navigation system for estimating a platform position, comprising:

a GPS receiver for GPS measurements by receiving GPS signals from satellites and processing the GPS signals for producing position estimation information;

a camera for imagery measurements by retrieving image data of landmarks surrounding the platform;

means for conducting an image recognition process on the image data thereby extracting attributes and feature characteristics of the landmarks; and

a Kalman filter for conducting a Kalman filtering procedure on the position estimation information from the GPS receiver and the image data retrieved by the camera, thereby improving accuracy of the position estimate.

12. The integrated imagery/navigation system for estimating the platform position as defined in claim 11, further comprising a landmark database for storing and updating the attributes and feature characteristics of the landmarks.

13. The integrated imagery/navigation system for estimating the platform position as defined in claim 12, wherein the attributes and feature characteristics of the landmark extracted from the image data retrieved by the camera are compared with he attributes and feature characteristics of the landmark stored in the landmark database to check if the landmarks recognized through the camera are registered in the landmark database as known or estimated data.

14. The integrated imagery/navigation system for estimating the platform position as defined in claim 13, wherein said attributes and feature characteristics of the recognized landmarks in the landmark database correlating the landmark attributes obtained by image recognition and mathematical feature characteristics obtained by feature extraction are re-identified with the corresponding data in the landmark database.

15. The integrated imagery/navigation system for estimating the platform position as defined in claim 14, wherein said landmark attributes include position, type, color, facing streets, etc. of the landmark, and said mathematical feature characteristics include edge curvature gradient of the landmark.

16. The integrated imagery/navigation system for estimating the platform position as defined in claim 12, wherein said attributes and feature characteristics of the landmark are compared with the attributes and feature characteristics of the landmark stored in the landmark database and the known or estimated data of the landmarks with higher estimation accuracy than that of the other is selected for incorporating the selected data into the position estimation information produced from the GPS signals.

17. The integrated imagery/navigation system for estimating the platform position as defined in claim 11, wherein a plurality of different frequencies are used for conducting the position estimates:

a high frequency to update the position estimates in a dead reckoning condition;

a middle frequency to correct the position estimates based upon the image data obtained through the camera, and a low frequency to correct the position estimates based upon the GPS signals.

18. The integrated imagery/navigation system for estimating the platform position as defined in claim 12, wherein said landmark database is updated in real-time by registering the landmark attributes including position, type, color, facing streets, etc. of the landmark, and the feature characteristics including edge curvature gradient of the landmark.

19. The integrated imagery/navigation system for estimating the platform position as defined in claim 12, wherein said landmark database is re-calibrated by starting landmark position estimation with lastly estimated position registered in the landmark database as an initial condition as well as changing a choice of known landmarks to be used for navigation accuracy enhancement.

20. The integrated imagery/navigation system for estimating the platform position as defined in claim 11, further comprising a GUI (graphic user interface) for processing the outputs of the Kalman filter and displaying the estimated position of the platform as a man-machine interface.