METHOD AND SYSTEM FOR HIERARCHICALLY MATCHING IMAGES OF BUILDINGS, AND COMPUTER-READABLE RECORDING MEDIUM

• US 20120093395A1
• Filed: 09/06/2010
• Published: 04/19/2012
• Est. Priority Date: 09/16/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Olaworks, Inc.
• Inventors: Kim, Tae Hoon, Ryu, Jung Hee
• Granted Patent: US 8,639,023 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

The present invention relates to a method for hierarchically matching a building image. The method includes the steps of: matching a wall of a specific building in the building image inputted as a query with a wall(s) of a building(s) in at least one panoramic image by using a technology of matching a building'"'"'s shape or repeated pattern; selecting a candidate panoramic image(s) which includes a building(s) recognized to have the same or similar wall to the specific building in the panoramic image(s) as a result of matching its wall with others; matching at least one local region, if containing a recognizable string or figure, in the specific building with local region(s) in the building(s) of the candidate panoramic image(s) by using a technology of recognizing a string or a figure; and determining top n panoramic image(s) as the result of matching the local region.

Claims

1. A method for hierarchically matching a building image, comprising the steps of:

(a) matching a wall of a specific building in the building image inputted as a query with a wall of a building in at least one panoramic image by using a technology of matching a building'"'"'s shape or repeated pattern;

(b) selecting a candidate panoramic image which includes a building recognized to have the same or similar wall to the specific building in the panoramic image as a result of matching its wall with others;

(c) matching at least one local region, if containing a recognizable string or figure, in the specific building with local region in the building of the candidate panoramic image by using a technology of recognizing a string or a figure; and

(d) determining a top n panoramic image with a top n degree of matching accuracy in the candidate panoramic image as the result of matching the local region.

2. The method of claim 1, wherein the building in the panoramic image is three-dimensionally reconstructed by using a stereo matching technique and the matching processes for the building'"'"'s wall and the local region are performed by referring to the three-dimensionally reconstructed information.

3. The method of claim 2, wherein the three-dimensionally reconstructed information by using the stereo matching technique is stored in a database.

4. The method of claim 2, before the step (a), further comprising the steps of:

designating at least one adjacent building, if it is next to the specific building in the building image inputted as the query and a distance between the at least one adjacent building and the specific building is less than a certain distance, as a specific building group, and matching the alignments of the buildings in the specific building group with those in a group of buildings included respectively in the at least one panoramic image; and

selecting a panoramic image which includes a certain group of buildings recognized to have the same or similar alignments to the buildings in the specific building group therein among the at least one panoramic image as the result of matching the alignments of the buildings.

5. The method of claim 4, wherein the matching process for the alignments of buildings included in the groups of buildings is performed by referring to the information on the reconstructed buildings included in the groups of buildings in the panoramic image in use of the stereo matching technique.

6. The method of claim 1, wherein the matching process for the wall includes a matching process for at least one of the building'"'"'s boundary, its wall'"'"'s shape, form, color, window alignment and window pieces.

7. The method of claim 1, wherein the local region includes at least one signboard.

8. The method of claim 1, further comprising the step of providing the top n panoramic image with the top n degree of matching accuracy as the result for retrieving the query.

9. The method of claim 1, further comprising the step of referring to additional information to increase the degree of matching accuracy,

wherein the additional information includes at least one of information items:

color distribution of at least one local region in the specific building,

the wall of a building adjacent to the specific building,

a local region of the building adjacent to the specific building, and

a location acquired from a GPS communicating with a user terminal, if the specific building is inputted as the query through the user terminal.

19. A medium recording a computer readable program to execute the method of claim 1.

10. A system for hierarchically matching a building image, comprising:

a building wall matching part for matching a wall of a specific building in the building image inputted as a query with a wall of a building in at least one panoramic image by using a technology of matching a building'"'"'s shape or repeated pattern and selecting a candidate panoramic image which includes a building recognized to have the same or similar wall to the specific building in the panoramic image as a result of matching its walls with others;

a local region matching part for matching at least one local region, if containing a recognizable string or figure, in the specific building with the local region in the building of the candidate panoramic image by using a technology of recognizing a string or a figure; and

a matching result providing part for determining top n panoramic image with the top n degree of matching accuracy in the candidate panoramic image as the result of matching the local region.

11. The system of claim 10, further comprising a stereo matching part for reconstructing the building included in the panoramic image by using a stereo matching technique, wherein the building wall matching part and the local region matching part perform the matching processes for the building'"'"'s wall and the local region by referring to information on the reconstructed building by using the stereo matching technique.

12. The system of claim 11, further comprising a database for storing the information on the reconstructed building by using the stereo matching technique.

13. The system of claim 10, further comprising a building alignment matching part for designating at least one adjacent building, if it is next to the specific building in the building image inputted as the query and a distance between the at least one adjacent building and the specific building is less than a certain distance, as a specific building group, and matching the alignments of the buildings in the specific building group with those in a group of buildings included respectively in the at least one panoramic image;

selecting a panoramic image which includes a certain group of buildings recognized to have the same or similar alignments to the buildings in the specific building group therein among the at least one panoramic image as the result of matching the alignments of the buildings; and

providing the selected panoramic image to the building wall matching part.

14. The system of claim 13, further comprising a stereo matching part for reconstructing the group of buildings included in the panoramic image by using the stereo matching technique, wherein the building alignment matching part performs the matching process for alignments of buildings included in the group of buildings by referring to the information on the reconstructed group of buildings in use of the stereo matching technique.

15. The system of claim 10, wherein the building wall matching part performs the matching process for the wall by matching at least one of the building'"'"'s boundary, its wall'"'"'s shape, form, color, window alignment and window pieces.

16. The system of claim 10, wherein the local region matching part performs matching for at least one signboard included in the local region.

17. The system of claim 10, further comprising a matching result providing part for providing the top n panoramic image with the top n degree of matching accuracy as the result for retrieving the query.

18. The system of claim 10, further comprising a supplementary matching part for referring to additional information to increase the degree of matching accuracy, wherein the additional information includes at least one of information items:

color distribution of at least one local region in the specific building,

the wall of a building adjacent to the specific building,

a local region of the building adjacent to the specific building, and

a location acquired from a GPS communicating with a user terminal, if the specific building is inputted as the query in the user terminal.