Pose estimation based on peripheral information

• US 9,147,122 B2
• Filed: 03/14/2013
• Issued: 09/29/2015
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chao, Hui, Tsin, Yanghai, Mohammad Mirzaei, Faraz
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US13/829,099
• Publication Number:
  US 20130322767A1
• Time in Patent Office: 929 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method for estimating camera pose includes: obtaining an image of a location captured via a camera, where the image includes a target object and edge line features outside of the target object; and calculating a pose of the camera with respect to the target object based on the edge line features.

Claims

1. A method for estimating camera pose, the method comprising:

obtaining an image of a location captured via a camera, wherein the image includes a target object and edge line features outside of the target object;

identifying the target object, as an identified target object, from a set of known reference objects; and

calculating a pose of the camera with respect to the identified target object based on the edge line features.

2. The method of claim 1 wherein identifying the target object comprises identifying the target object via a computer vision algorithm, by comparing the target object with the set of known reference objects, independently of calculating the pose of the camera.

3. The method of claim 2 wherein the identifying comprises detecting key points of the target object in the image.

4. The method of claim 1 further comprising receiving an identity of the target object within the image from user input.

5. The method of claim 1 further comprising estimating one or more vanishing points associated with the edge line features, wherein the calculating comprises calculating the pose of the camera with respect to the target object based on the estimated vanishing points.

6. The method of claim 5 further comprising detecting the edge line features within the image.

7. The method of claim 6 wherein detecting the edge line features comprises detecting horizontal edge line features and vertical edge line features.

8. The method of claim 7 wherein detecting the edge line features comprises detecting at least one of a floor, a roof, or a door.

9. The method of claim 7 wherein estimating the one or more vanishing points comprises at least one of estimating a horizontal vanishing point from the horizontal edge line features or estimating a vertical vanishing point from the vertical edge line features.

10. The method of claim 5 wherein:

estimating the one or more vanishing points comprises estimating one vanishing point; and

calculating the pose of the camera comprises estimating rotation of the camera with respect to a plane, defined by a detectable target in the image, from a set of discrete rotation states based on the one estimated vanishing point.

11. The method of claim 5 wherein:

estimating the one or more vanishing points comprises estimating two or more vanishing points; and

calculating the pose of the camera comprises calculating the pose of the camera as at least one of pan, tilt, or roll with respect to a plane, defined by a detectable target in the image, based on the two or more vanishing points.

12. The method of claim 11 wherein calculating the pose comprises calculating the roll, the pan, and the tilt.

13. The method of claim 12 further comprising obtaining information relating to at least one of the tilt or the pan from an orientation sensor coupled to the camera.

14. The method of claim 12 further comprising determining the roll based at least in part on the tilt.

15. The method of claim 1 further comprising determining a location of the target object within a geographical area independently of calculating the pose of the camera based on map data for the geographical area.

16. The method of claim 15 further comprising estimating a position of a user of the camera within the geographical area based on the location of the target object and the pose of the camera.

17. A system for estimating camera pose, the system comprising:

a camera configured to capture an image of a location including a target object; and

a processor configured to;

detect edge line features outside of the target object within the image;

identify the target object, as an identified target object, from a set of known reference objects; and

calculate a pose of the camera with respect to the identified target object based on the edge line features.

18. The system of claim 17 wherein the processor is further configured to:

estimate one or more vanishing points associated with the edge line features, and

calculate the pose of the camera based on the vanishing points.

19. The system of claim 18 wherein the edge line features comprise horizontal edge line features and vertical edge line features.

20. The system of claim 19 wherein the edge line features correspond to at least one of a floor, a roof, or a door.

21. The system of claim 19 wherein the processor is further configured to perform at least one of estimating a horizontal vanishing point from the horizontal edge line features or estimating a vertical vanishing point from the vertical edge line features.

22. The system of claim 18 wherein the processor is further configured to:

estimate one vanishing point; and

estimate rotation of the camera with respect to a plane, defined by a detectable target in the image, from a set of discrete rotation states based on the one estimated vanishing point.

23. The system of claim 18 wherein the processor is further configured to:

estimate two or more vanishing points; and

calculate the pose of the camera as at least one of pan, tilt, or roll with respect to a plane, defined by a detectable target in the image, based on the two or more vanishing points.

24. The system of claim 23 wherein the processor is further configured to calculate roll, pan, and tilt to calculate the pose of the camera.

25. The system of claim 24 further comprising an orientation sensor coupled to the camera, wherein the processor is further configured to obtain information relating to at least one of the tilt or the pan from the orientation sensor.

26. The system of claim 24 wherein the processor is further configured to determine the roll based at least in part on the tilt.

27. The system of claim 17 wherein the processor is further configured to determine a location of the target object within a geographical area independently of calculating the pose of the camera based on map data for the geographical area.

28. The system of claim 27 wherein the processor is further configured to estimate a position of a user of the camera within the geographical area based on the location of the target object and the pose of the camera.

29. A system for estimating camera pose, the system comprising:

means for capturing an image of a location, wherein the image includes a target object;

means for detecting edge line features outside of the target object within the image;

means for identifying the target object, as an identified target object, from a set of known reference objects; and

means for calculating a pose of the means for capturing with respect to the identified target object based on the edge line features.

30. The system of claim 29 further comprising means for estimating one or more vanishing points associated with the edge line features, wherein the means for calculating comprises means for calculating the pose of the means for capturing based on the vanishing points.

31. The system of claim 29 wherein the means for detecting the edge line features comprises means for detecting horizontal edge line features and vertical edge line features.

32. The system of claim 31 wherein the means for detecting the edge line features comprises means for detecting at least one of a floor, a roof, or a door.

33. The system of claim 30 wherein the means for estimating the one or more vanishing points comprises at least one of means for estimating a horizontal vanishing point from the horizontal edge line features or means for estimating a vertical vanishing point from the vertical edge line features.

34. The system of claim 30 wherein:

the means for estimating the one or more vanishing points comprises means for estimating one vanishing point; and

the means for calculating the pose of the means for capturing comprises means for estimating rotation of the means for capturing with respect to a plane, defined by a detectable target in the image, from a set of discrete rotation states based on the one estimated vanishing point.

35. The system of claim 30 wherein:

the means for estimating the one or more vanishing points comprises means for estimating two or more vanishing points; and

the means for calculating the pose of the means for capturing comprises means for calculating the pose of the means for capturing as at least one of pan, tilt, or roll with respect to a plane, defined by a detectable target in the image, based on the two or more vanishing points.

36. The system of claim 35 wherein the means for calculating the pose of the means for capturing comprises means for calculating the pose of the means for capturing as roll, pan, and tilt.

37. The system of claim 36 further comprising means for obtaining information relating to at least one of the tilt or the pan from an orientation sensor coupled to the means for capturing.

38. The system of claim 36 further comprising means for determining the roll based at least in part on the tilt.

39. A non-transitory processor-readable storage medium comprising processor-executable instructions configured to cause a processor to:

obtain an image of a location captured via a camera, wherein the image includes a target object and edge line features outside the target object;

identify the target object, as an identified target object, from a set of known reference objects; and

calculate a pose of the camera with respect to the identified target object based on the edge line features.

40. The non-transitory processor-readable storage medium of claim 39 further comprising instructions configured to cause the processor to:

estimate one or more vanishing points associated with the edge line features; and

calculate the pose of the camera based on the vanishing points.