Scene classification for place recognition

• US 8,532,400 B1
• Filed: 07/24/2012
• Issued: 09/10/2013
• Est. Priority Date: 12/07/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Babenko, Boris, Adam, Hartwig, Flynn, John, Neven, Hartmut
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US13/556,350
• Time in Patent Office: 413 Days
• Field of Search: None
• US Class Current: 382/224
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Aspects of the invention pertain to identifying whether or not an image from a user'"'"'s device is of a place. Before undertaking time and resource consuming analysis of an image using specialized image analysis modules, pre-filtering classification is conducted based on image data and metadata associated with the image. The metadata may include geolocation information. One classification procedure analyzes the metadata to perform a high level determination as to whether the image is of a place. If the results indicate that it is of a place, then a further classification procedure may be performed, where the image information is analyzed, with or without the metadata. This process may be done concurrently with a place match filtering procedure. The results of the further classification will either find a match with a given place or not. The output is a place match either with or without geolocation information.

Claims

1. A method of classifying an image, comprising:

receiving an image and metadata of the image from a user device;

performing a first classification of the image using only the metadata to determine whether the image is a place;

performing a second classification of the image using both the metadata and information about the image to determine whether the image is the place; and

when either the first or second classification determines that the image is of the place, then preparing place information for presentation by the user device;

wherein if different filter modules identify the image as being of different types, then the first and second classifications are used to select which filter module identification is correct.

2. The method of claim 1, wherein the image information is either a zoom level or a shutter speed.

3. The method of claim 1, wherein the metadata includes a location of the user device when the image was taken.

4. The method of claim 1, wherein the metadata is location accuracy information.

5. The method of claim 1, wherein the metadata comprises location statistics.

6. The method of claim 1, wherein the metadata is a device orientation that includes at least one of angle, pitch, roll or compass point direction.

7. The method of claim 1, wherein the metadata includes an orientation of the image.

8. A tangible, non-transitory computer-readable recording medium storing a computer program therein, the computer program, when executed by a processor, causing the processor to perform a method of classifying an image, the method comprising:

receiving an image and metadata of the image from a user device;

performing a first classification of the image using only the metadata to determine whether the image is a place;

performing a second classification of the image using both the metadata and information about the image to determine whether the image is the place; and

when either the first or second classification determines that the image is of the place, then preparing place information for presentation by the user device;

wherein if different filter modules identify the image as being of different types, then the first and second classifications are used to select which filter module identification is correct.

9. The recording medium of claim 8, wherein the image information is either a zoom level or a shutter speed.

10. The recording medium of claim 8, wherein the metadata includes a location of the user device when the image was taken.

11. The recording medium of claim 8, wherein the metadata is location accuracy information.

12. The recording medium of claim 8, wherein the metadata comprises location statistics.

13. The recording medium of claim 8, wherein the metadata is a device orientation that includes at least one of angle, pitch, roll or compass point direction.

14. An apparatus for classifying an image, the apparatus comprising:

means for communicating with a mobile user device; and

at least one processor coupled to the communicating means, the at least one processor configured to;

receive an image and metadata of the image from a user device;

perform a first classification of the image using only the metadata to determine whether the image is a place;

perform a second classification of the image using both the metadata and information about the image to determine whether the image is the place; and

when either the first or second classification determines that the image is of the place, then prepare place information for presentation by the user device;

wherein if different filter modules identify the image as being of different types, then the first and second classifications are used to select which filter module identification is correct.

15. The apparatus of claim 14, wherein the image information is either a zoom level or a shutter speed.

16. The apparatus of claim 14, wherein the metadata includes a location of the user device when the image was taken.

17. The apparatus of claim 14, wherein the metadata is location accuracy information.

18. The apparatus of claim 14, wherein the metadata comprises location statistics.

19. The apparatus of claim 14, wherein the metadata is a device orientation that includes at least one of angle, pitch, roll or compass point direction.

20. The apparatus of claim 14, wherein the metadata includes an orientation of the image.