Algorithm to estimate yaw errors in camera pose

• US 9,986,150 B2
• Filed: 01/29/2016
• Issued: 05/29/2018
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Van Olst, Daniel G, Berkner, Kathrin, Narasimha, Ramya
• Primary Examiner(s): Le, Peter D
• Application Number:
  US15/010,620
• Publication Number:
  US 20170094158A1
• Time in Patent Office: 851 Days
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A system and method that calculates a yaw error in an image and provides a user interface to a user for correcting the yaw error. The method includes receiving an image, performing line detection in the image, computing a line parameterization for lines in the image, computing a yaw angle for the image and providing the yaw data calculated in the image.

Claims

1. A method comprising:

receiving, using one or more computing devices, a preview image of an object;

performing, using the one or more computing devices, line detection on the preview image to detect a plurality of lines, the plurality of lines including a first line and a second line;

determining, using the one or more computing devices, a first height between the first line and the second line at a first point in the preview image;

determining using the one or more computing devices a second height between the first line and the second line at a second point in the preview image;

computing, using the one or more computing devices, a line parameterization for the plurality of lines by calculating a first distance from a capture device point to the first point in the preview image, calculating a second distance from the capture device point to the second point in the preview image and determining one or more of start points end points and point and slope of the first line and the second line;

calculating, using the one or more computing devices, a yaw angle using the line parameterization; and

correcting, using the one or more computing devices, the preview image of the object using the yaw angle to adjust the preview image and remove a portion of an error introduced by the yaw angle.

2. The method of claim 1, wherein the first line and the second line are non-parallel.

3. The method of claim 1, further comprising:

stitching, using the one or more computing devices, the adjusted preview image into a plurality of previously captured images to create a linear panoramic image of the object.

4. The method of claim 1, wherein correcting the preview image of the object using the yaw angle includes presenting, using the one or more computing devices, an indicator representing the yaw angle in a user interface on a display device.

5. The method of claim 4, wherein the indicator representing the yaw angle is a vertical eyelid indicator, the method further comprising:

computing, using the one or more computing devices, a pitch angle and a roll angle for the preview image;

determining, using the one or more computing devices, the vertical eyelid indicator based on the yaw angle;

determining, using the one or more computing devices, a horizontal eyelid indicator based on the pitch angle;

determining, using the one or more computing devices, a ball indicator based on the roll angle;

presenting, using the one or more computing devices, the vertical eyelid indicator over a portion of an inner area of a smaller concentric circle centered on the display device;

presenting, using the one or more computing devices, the horizontal eyelid indicator over a portion of an inner area of a larger concentric circle centered on the display device; and

presenting, using the one or more computing devices, the ball indicator along an outside edge of the larger concentric circle centered on the display device.

6. The method of claim 1, further comprising:

determining, using the one or more computing devices, a vanishing point using the line parameterization of the first line and the second line.

16. The method of claim 1, further comprising:

determining a vanishing point using the line parameterization of the first line and the second line.

7. A system comprising:

a processor; and

a memory storing instructions that, when executed, cause the system to;

receive a preview image of an object;

perform line detection on the preview image to detect a plurality of lines the plurality of lines including a first line and a second line;

determine a first height between the first line and the second line at a first point in the preview image;

determine a second height between the first line and the second line at a second point in the preview image;

compute a line parameterization for the plurality of lines by calculating a first distance from a capture device to the first point in the preview image, calculating a second distance from the capture device to the second point in the preview image, and determining one or more of start points, end points and point and slope of the first line and the second line;

calculate a yaw angle using the line parameterization; and

correct the preview image of the object using the yaw angle to adjust the preview image and remove a portion of an error introduced by the yaw angle.

8. The system of claim 7, wherein the first line and the second line are non-parallel.

9. The system of claim 7, wherein the instructions that, when executed, further cause the system to:

stitch the adjusted preview image into a plurality of previously captured images to create a linear panoramic image of the object.

10. The system of claim 7, wherein the instructions that, when executed, cause the system to correct the preview image of the object using the yaw angle includes instructions that, when executed, cause the system to present an indicator representing the yaw angle in a user interface on a display device.

11. The system of claim 10, wherein the indicator representing the yaw angle is a vertical eyelid indicator, and wherein the instructions that, when executed, further cause the system to:

compute a pitch angle and a roll angle for the preview image;

determine the vertical eyelid indicator based on the yaw angle;

determine a horizontal eyelid indicator based on the pitch angle;

determine a ball indicator based on the roll angle;

present the vertical eyelid indicator over a portion of an inner area of a smaller concentric circle centered on the display device;

present the horizontal eyelid indicator over a portion of an inner area of a larger concentric circle centered on the display device; and

present the ball indicator along an outside edge of the larger concentric circle centered on the display device.

19. The system of claim 7, wherein the instructions that, when executed, further cause the system to determine a vanishing point using the line parameterization of the first line and the second line.

12. A computer program product comprising a non-transitory computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer cause the computer to:

receive a preview image of an object;

perform line detection on the preview image to detect a plurality of lines, the plurality of lines including a first line and a second line;

determine a first height between the first line and the second line at a first point in the preview image;

determine a second height between the first line and the second line at a second point in the preview image;

compute a line parameterization for the plurality of lines by calculating a first distance from a capture device point to the first point in the preview image, calculating a second distance from the capture device point to the second point in the preview image, and determining one or more of start points, end points, and point and slope of the first line and the second line;

calculate a yaw angle using the line parameterization; and

correct the preview image of the object using the yaw angle to adjust the preview image and remove a portion of an error introduced by the yaw angle.

13. The computer program product of claim 12, wherein the first line and the second line are non-parallel.

14. The computer program product of claim 12, wherein the computer readable program when executed on a computer causes the computer to correct the preview image of the object using the yaw angle and further causes the computer to present an indicator representing the yaw angle in a user interface on a display device.

15. The computer program product of claim 14, wherein the computer readable program when executed on a computer to further cause the computer to present the indicator representing the yaw angle is a vertical eyelid indicator further cause the computer to:

compute a pitch angle and a roll angle for the preview image;

determine the vertical eyelid indicator based on the yaw angle;

determine a horizontal eyelid indicator based on the pitch angle;

determine a ball indicator based on the roll angle;

present the vertical eyelid indicator over a portion of an inner area of a smaller concentric circle centered on the display device;

present the horizontal eyelid indicator over a portion of an inner area of a larger concentric circle centered on the display device; and

present the ball indicator along an outside edge of the larger concentric circle centered on the display device.

17. The computer program product of claim 12, wherein the computer readable program when executed on a computer causes the computer to determine a vanishing point using the line parameterization of the first line and the second line.

18. The computer program product of claim 12, wherein the computer readable program when executed on a computer causes the computer to stitch the adjusted preview image into a plurality of previously captured images to create a linear panoramic image of the object.