Speed-up template matching using peripheral information

• US 9,317,921 B2
• Filed: 07/10/2014
• Issued: 04/19/2016
• Est. Priority Date: 07/10/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chao, Hui, Krishnamoorthi, Raghuraman, Mohammad Mirzaei, Faraz
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US14/328,308
• Publication Number:
  US 20160012593A1
• Time in Patent Office: 649 Days
• Field of Search: 382/209
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 2201/09 Recognition of logos
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A method for template matching is described. In one implementation, an image including a target object is captured by a camera. A pose of the camera is estimated based, at least in part, on peripheral information from the image. Pre-selecting is performed on a set of pose templates by matching the pose of the camera to poses associated with pose templates to identify a set of matching pose templates. The template matching for the target object is then performed using the set of matching pose templates.

Claims

1. A method for estimating camera pose, the method comprising:

obtaining an image of a location captured via a camera, wherein the image includes a target object and peripheral information;

calculating a pose of the camera with respect to the target object based at least in part on the peripheral information;

accessing a template matching system, wherein the template matching system comprises a plurality of templates for each object of a plurality of objects, and wherein each template is associated with a template pose;

pre-selecting templates of the plurality of templates where the pose of the camera matches the template pose to identify a set of matching pose templates; and

processing the set of matching pose templates to identify a template match with the target object.

2. The method of claim 1 wherein the peripheral information comprises vanishing point information derived from processing the image.

3. The method of claim 2 wherein the peripheral information further comprises sensor information collected by a mobile device, wherein the mobile device comprises the camera.

4. The method of claim 3 wherein the sensor information comprises accelerometer data, and wherein the peripheral information comprises a gravity vector determined, at least in part, from the accelerometer data.

5. The method of claim 2 wherein the peripheral information consists of at least two horizontal vanishing points.

6. The method of claim 2 wherein the peripheral information consists of one horizontal vanishing point and a gravity vector.

7. The method of claim 1 wherein pre-selecting templates of the plurality of templates where the pose of the camera matches the template pose to identify the set of matching pose templates comprises:

identifying templates of the plurality of templates with a difference in pose angle of less than a threshold amount; and

selecting the templates of the plurality of templates with the difference in pose angle of less than the threshold amount as the set of matching pose templates.

8. The method of claim 1 wherein pre-selecting templates of the plurality of templates where the pose of the camera matches the template pose to identify the set of matching pose templates comprises:

for each object represented by at least template of the plurality of templates, determining the template associated with each object that differs from the pose of the camera by a smallest amount; and

selecting the template associated with each object that differs from the pose of the camera by the smallest amount as part of the set of matching pose templates.

9. The method of claim 1 further comprising:

identifying location information associated with the template matching the target object; and

estimating the location of the camera based on the location information associated with the template matching the target object and the pose of the camera.

10. The method of claim 9 further comprising:

providing location based services to a mobile device comprising the camera based on the location of the camera.

11. The method of claim 1 wherein the target object comprises a store logo.

12. The method of claim 1 wherein calculating the pose of the camera with respect to the target object based at least in part on the peripheral information comprises:

processing the image to identify a plurality of edge line features of the image.

13. The method of claim 12 wherein detecting the plurality of edge line features comprises detecting horizontal edge line features and vertical edge line features.

14. The method of claim 13 wherein detecting the plurality of edge line features comprises detecting at least one of a floor, a roof, or a door.

15. The method of claim 1 wherein:

calculating the pose of the camera comprises calculating the pose of the camera as at least one of pan, tilt, or roll with respect to a plane, defined with respect to the target object in the image, based on the peripheral information.

16. A mobile device for accelerated template matching using peripheral information comprising:

a processor;

a camera coupled to the processor; and

a memory coupled to the processor, wherein the memory comprises instructions that cause the processor to;

obtain an image of a location captured via the camera, wherein the image includes a target object and at least a portion of the peripheral information;

calculate a pose of the camera with respect to the target object based at least in part on the peripheral information;

access a template matching system, wherein the template matching system comprises a plurality of templates for each object of a plurality of objects, and wherein each template is associated with a template pose;

pre-select templates of the plurality of templates where the pose of the camera matches the template pose to identify a set of matching pose templates; and

process the set of matching pose templates to identify a template match with the target object.

17. The mobile device of claim 16 further comprising:

a display,

wherein the instructions further cause the processor to determine the location associated with the template matching the target object, and to display an estimated location of the mobile device based on the location associated with the template matching the target object.

18. The mobile device of claim 16 further comprising:

an accelerometer,

wherein the instructions further cause the processor to identify a gravity vector based on sensor data from the accelerometer, and to calculate the pose of the camera with respect to the target object using the gravity vector in conjunction with the peripheral information from the image.

19. A non-transitory computer-readable storage device comprising instructions that, when executed by a processor, cause a system to perform a method of template matching, the method comprising:

obtaining an image of a location captured via a camera, wherein the image includes a target object and peripheral information;

calculating a pose of the camera with respect to the target object based at least in part on the peripheral information;

accessing a template matching system, wherein the template matching system comprises a plurality of templates for each object of a plurality of objects, and wherein each template is associated with a template pose;

pre-selecting templates of the plurality of templates where the pose of the camera matches the template pose to identify a set of matching pose templates; and

processing the set of matching pose templates to identify a template match with the target object.

20. The non-transitory computer-readable storage medium of claim 19, wherein accessing the template matching system comprises communicating, via a wireless network, a request from a mobile device comprising the camera to a server system comprising a database that includes the plurality of templates.

21. A mobile device for accelerated template matching using peripheral information comprising:

means for obtaining an image of a location captured via a camera, wherein the image includes a target object and peripheral information;

means for calculating a pose of the camera with respect to the target object based at least in part on the peripheral information;

means for accessing a template matching system, wherein the template matching system comprises a plurality of templates for each object of a plurality of objects, and wherein each template is associated with a template pose;

means for pre-selecting templates of the plurality of templates where the pose of the camera matches the template pose to identify a set of matching pose templates; and

means for processing the set of matching pose templates to identify a template match with the target object.

22. The mobile device of claim 21 further comprising:

means for identifying a gravity vector;

wherein the means for calculating the pose of the camera with respect to the target object uses the gravity vector in conjunction with the peripheral information from the image.

23. The mobile device of claim 21 further comprising:

means for estimating a location of the mobile device based on the location associated with the template matching the target object; and

means for displaying the estimated location of the mobile device.

24. The mobile device of claim 21 further comprising:

means for extracting features from the image for use in calculating the pose of the camera; and

means for detecting points of interest in the image, wherein the points of interest comprise the target object.