Method and system for hierarchically matching images of buildings, and computer-readable recording medium

• US 8,639,023 B2
• Filed: 09/06/2010
• Issued: 01/28/2014
• Est. Priority Date: 09/16/2009
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Kim, Tae Hoon, Ryu, Jung Hee
• Primary Examiner(s): Wang, Claire X
• Assistant Examiner(s): JAMES, THOMAS AIDEN
• Application Number:
  US13/377,935
• Publication Number:
  US 20120093395A1
• Time in Patent Office: 1,240 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

The present invention relates to a method for hierarchically matching a building image. The method includes the steps of: matching a wall of a specific building in the building image inputted as a query with a wall(s) of a building(s) in at least one panoramic image by using a technology of matching a building'"'"'s shape or repeated pattern; selecting a candidate panoramic image(s) which includes a building(s) recognized to have the same or similar wall to the specific building in the panoramic image(s) as a result of matching its wall with others; matching at least one local region, if containing a recognizable string or figure, in the specific building with local region(s) in the building(s) of the candidate panoramic image(s) by using a technology of recognizing a string or a figure; and determining top n panoramic image(s) as the result of matching the local region.

Claims

1. A method for hierarchically matching a building image, comprising:

designating a building adjacent to be included in a specific building group if the adjacent building is next to a specific building in a building image inputted as a query and a distance between the adjacent building and the specific building is less than a certain distance;

matching the alignments of the buildings in the specific building group with those in a group of buildings included respectively in at least one panoramic image;

selecting a panoramic image that includes a certain group of buildings recognized to have the same or similar alignments to the buildings in the specific building group among the at least one panoramic image as the result of matching the alignments of the buildings;

matching a wall of the specific building with a wall of a building in the at least one panoramic image based on physical parameters of the wall by using a technology of matching a building'"'"'s shape or repeated pattern;

selecting a candidate panoramic image which includes a building recognized to have the same or similar wall to the specific building in the building image as a result of matching its wall with others;

matching at least one local region, if containing a recognizable string or figure, in the specific building with local region in the building of the candidate panoramic image by using a technology of recognizing a string or a figure; and

determining a top panoramic image with a top degree of matching accuracy in the candidate panoramic image as the result of matching the local region.

2. The method of claim 1, wherein the building in the panoramic image is three-dimensionally reconstructed by using a stereo matching technique and the matching processes for the building'"'"'s wall and the local region are performed by referring to the three-dimensionally reconstructed information.

3. The method of claim 2, wherein the three-dimensionally reconstructed information by using the stereo matching technique is stored in a database.

4. The method of claim 1, wherein the matching process for the alignments of buildings included in the groups of buildings is performed by referring to the information on the reconstructed buildings included in the groups of buildings in the panoramic image in use of the stereo matching technique.

5. The method of claim 1, wherein the matching process for the wall includes a matching process for at least one of the building'"'"'s boundary, its wall'"'"'s shape, form, color, window alignment and window pieces.

6. The method of claim 1, wherein the local region includes at least one signboard.

7. The method of claim 1, further comprising the step of providing the top n panoramic image with the top n degree of matching accuracy as the result for retrieving the query.

8. The method of claim 1, further comprising the step of referring to additional information to increase the degree of matching accuracy, wherein the additional information includes at least one of information items:

color distribution of at least one local region in the specific building,

the wall of a building adjacent to the specific building,

a local region of the building adjacent to the specific building, and

a location acquired from a GPS communicating with a user terminal, if the specific building is inputted as the query through the user terminal.

9. A system for hierarchically matching a building image, comprising:

a building alignment matching part for designating a building adjacent to be included in a specific building group if the adjacent building is next to a specific building in a building image inputted as a query and a distance between the adjacent building and the specific building is less than a certain distance, matching the alignments of the buildings in the specific building group with those in a group of buildings included respectively in at least one panoramic image and selecting a panoramic image that includes a certain group of buildings recognized to have the same or similar alignments to the buildings in the specific building group among the at least one panoramic image as the result of matching the alignments of the buildings;

a building wall matching part for matching a wall of the specific building with a wall of a building in at least one panoramic image based on physical parameters of the wall by using a technology of matching a building'"'"'s shape or repeated pattern and selecting a candidate panoramic image which includes a building recognized to have the same or similar wall to the specific building in the building image as a result of matching its walls with others;

a local region matching part for matching at least one local region, if containing a recognizable string or figure, in the specific building with the local region in the building of the candidate panoramic image by using a technology of recognizing a string or a figure; and

a matching result providing part for determining top panoramic image with the top degree of matching accuracy in the candidate panoramic image as the result of matching the local region.

10. The system of claim 9, further comprising a stereo matching part for reconstructing the building included in the panoramic image by using a stereo matching technique, wherein the building wall matching part and the local region matching part perform the matching processes for the building'"'"'s wall and the local region by referring to information on the reconstructed building by using the stereo matching technique.

11. The system of claim 10, further comprising a database for storing the information on the reconstructed building by using the stereo matching technique.

12. The system of claim 9, further comprising a stereo matching part for reconstructing the group of buildings included in the panoramic image by using the stereo matching technique, wherein the building alignment matching part performs the matching process for alignments of buildings included in the group of buildings by referring to the information on the reconstructed group of buildings in use of the stereo matching technique.

13. The system of claim 9, wherein the building wall matching part performs the matching process for the wall by matching at least one of the building'"'"'s boundary, its wall'"'"'s shape, form, color, window alignment and window pieces.

14. The system of claim 9, wherein the local region matching part performs matching for at least one signboard included in the local region.

15. The system of claim 9, further comprising a matching result providing part for providing the top n panoramic image with the top n degree of matching accuracy as the result for retrieving the query.

16. The system of claim 9, further comprising a supplementary matching part for referring to additional information to increase the degree of matching accuracy, wherein the additional information includes at least one of information items:

color distribution of at least one local region in the specific building,

the wall of a building adjacent to the specific building,

a local region of the building adjacent to the specific building, and

a location acquired from a GPS communicating with a user terminal, if the specific building is inputted as the query in the user terminal.

17. A non-transitory medium recording a computer readable program.