Location-based image retrieval

• US 10,025,798 B2
• Filed: 01/14/2014
• Issued: 07/17/2018
• Est. Priority Date: 01/14/2013
• Status: Active Grant

• Current Assignee: BAR ILAN University
• Original Assignee: BAR ILAN University
• Inventors: Kaminka, Gal, Kosti, Shahar Moshe, Sarne, David
• Primary Examiner(s): Woo, Isaac M
• Application Number:
  US14/767,368
• Publication Number:
  US 20170206222A1
• Time in Patent Office: 1,645 Days
• Field of Search: 707600-899
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method and a for location-based image retrieval, the method comprising using at least one hardware processor for: receiving N-dimensional coordinates of a point of interest, wherein N is an integer equal to or larger than 2; and retrieving one or more digital images in which the point of interest is visible. In addition, a computer program product configured to execute this method.

Claims

1. A method for location-based image retrieval, the method comprising using at least one hardware processor for:

receiving N-dimensional coordinates of a point of interest, wherein N is an integer equal to or larger than 2;

retrieving one or more digital images in which the point of interest is visible;

receiving multiple digital images and N-dimensional mapping data associated with each of the multiple digital images, wherein the N-dimensional mapping data comprises;

camera location data, camera orientation data and a distance measurement from a camera; and

for each of the digital images, computing a field of view polytope based on the N-dimensional mapping data, wherein the retrieved one or more digital images are those of the multiple digital images in which the point of interest is contained within the field of view polytope; and

ranking the one or more digital images according to at least one criterion selected from the group consisting of;

an area size visible in each of the one or more digital images,

a distance from the point of interest at which each of the one or more digital images was taken, and

a centeredness of the point of interest in each of the one or more digital images.

2. The method according to claim 1, wherein the field of view polytope is devoid of a visually-occluded space.

3. The method according to claim 1, further comprising using said at least one hardware processor or a different at least one hardware processor for computing, for each of the digital images, a discrete oriented polytope (DOP) which bounds the field of view polytope,

wherein the retrieved one or more digital images are those of the multiple digital images in which the point of interest is contained within the DOP.

4. The method according to claim 1, further comprising using said at least one hardware processor or a different at least one hardware processor for:

generating a spatial index of the N-dimensional mapping data; and

storing the N-dimensional mapping data and the spatial index in a database,

wherein said retrieving comprises searching the database using the spatial index.

5. The method according to claim 1, further comprising using said at least one hardware processor or a different at least one hardware processor for grouping the one or more digital images according to different view angles of the point of interest.

6. The method according to claim 5, wherein said grouping comprises dividing a space around the point of interest into sectors, and determining, based on the camera location data, in which of the sectors each of the one or more digital images was taken.

7. The method according to claim 1, further comprising using said at least one hardware processor or a different at least one hardware processor for at least one of:

computing the area size based on the distance measurement from the camera of each of the one or more digital images;

computing the distance from the point of interest based on the distance measurement from the camera of each of the one or more digital images; and

computing the centeredness of the point of interest based on two-dimensional coordinates of the point of interest in each of the one or more digital images.

8. A computer program product for location-based image retrieval, the computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor for:

receiving N-dimensional coordinates of a point of interest, wherein N is an integer equal to or larger than 2;

retrieving one or more digital images in which the point of interest is visible

receiving multiple digital images and N-dimensional mapping data associated with each of the multiple digital images, wherein the N-dimensional mapping data comprises;

camera location data, camera orientation data and a distance measurement from a camera; and

for each of the digital images, computing a field of view polytope based on the N-dimensional mapping data, wherein the retrieved one or more digital images are those of the multiple digital images in which the point of interest is contained within the field of view polytope; and

ranking the one or more digital images according to at least one criterion selected from the group consisting of;

an area size visible in each of the one or more digital images,

a distance from the point of interest at which each of the one or more digital images was taken, and

a centeredness of the point of interest in each of the one or more digital images.

9. The computer program product according to claim 8, wherein the field of view polytope is devoid of a visually-occluded space.

10. The computer program product according to claim 8, wherein the program code is further executable for computing, for each of the digital images, a discrete oriented polytope (DOP) which bounds the field of view polytope,

wherein the retrieved one or more digital images are those of the multiple digital images in which the point of interest is contained within the DOP.

11. The computer program product according to claim 8, wherein the program code is further executable for:

generating a spatial index of the N-dimensional mapping data; and

storing the N-dimensional mapping data and the spatial index in a database,

wherein said retrieving comprises searching the database using the spatial index.

12. The computer program product according to claim 8, wherein the program code is further executable for grouping the one or more digital images according to different view angles of the point of interest.

13. The computer program product according to claim 12, wherein said grouping comprises dividing a space around the point of interest into sectors, and determining, based on the camera location data, in which of the sectors each of the one or more digital images was taken.

14. The computer program product according to claim 8, wherein the program code is further executable for at least one of:

computing the area size based on the distance measurement from the camera of each of the one or more digital images;

computing the distance from the point of interest based on the distance measurement from the camera of each of the one or more digital images; and

computing the centeredness of the point of interest based on two-dimensional coordinates of the point of interest in each of the one or more digital images.