Methods, systems, and computer-readable media for utilizing radial distortion to estimate a pose configuration

• US 10,410,372 B1
• Filed: 06/14/2018
• Issued: 09/10/2019
• Est. Priority Date: 06/14/2018
• Status: Active Grant

• Current Assignee: University of North Carolina At Chapel Hill (University of North Carolina System)
• Original Assignee: University of North Carolina At Chapel Hill (University of North Carolina System)
• Inventors: Bapat, Akash Abhijit, Price, James True, Frahm, Jan-Michael
• Primary Examiner(s): Pontius, James M
• Application Number:
  US16/008,682
• Time in Patent Office: 453 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

Methods, systems, and computer-readable media for utilizing radial distortion to estimate a pose configuration are disclosed. According to one aspect, the method includes receiving, from each of a plurality of camera devices, an input pixel row of a radially distorted image and conducting a row comparison between each of the input pixel rows and a respectively associated synthesized pixel row. The method further includes approximating, for each row comparison, a span of a curve in an image space with a plurality of segments and computing, for each of the plurality of segments, a constraint. The method also includes utilizing the constraints to estimate a pose configuration.

Claims

1. A method for utilizing radial distortion to estimate a pose configuration, the method comprising:

receiving, from each of a plurality of camera devices, an input pixel row of a radially distorted image;

conducting a row comparison between each of the input pixel rows and a respectively associated synthesized pixel row;

approximating, for each row comparison, a span of a curve in an image space with a plurality of segments;

computing, for each of the plurality of segments, a constraint; and

utilizing the constraints to estimate a pose configuration.

2. The method of claim 1 wherein the plurality of camera devices includes a plurality of rolling shutter cameras.

3. The method of claim 1 wherein conducting the row comparison comprises generating the curve by synthesizing a distorted pixel row from previously captured frames of the radially distorted image using a homography-based warping technique.

4. The method of claim 3 comprising comparing the synthesized distorted pixel row with the input pixel row at multiple points that correspond to the centers of the plurality of segments.

5. The method of claim 3 wherein synthesizing a distorted pixel row includes extracting a distorted pixel row from the previously captured frames.

6. The method of claim 1 comprising storing data corresponding to each of the input pixel row and subsequently computing the constraints.

7. The method of claim 1 wherein the plurality of camera devices is configured as a camera cluster positioned on a head-mounted display.

8. A system utilizing radial distortion to estimate a pose configuration, the system comprising:

a plurality of camera devices; and

a pose-tracking device including at least one processor and a memory, wherein a tracking engine is stored in the memory and when executed by the at least one processor is configured for receiving an input pixel row of a radially distorted image from each of a plurality of camera devices, conducting a row comparison between each of the input pixel rows and a respectively associated synthesized pixel row, approximating, for each row comparison, a span of a curve in an image space with a plurality of segments, computing, for each of the plurality of segments, a constraint, and utilizing the constraints to estimate a pose configuration.

9. The system of claim 8 wherein the plurality of camera devices includes a plurality of rolling shutter cameras.

10. The system of claim 8 wherein the pose-tracking device is further configured for generating the curve by synthesizing a distorted pixel row from previously captured frames of the radially distorted image using a homography-based warping technique.

11. The system of claim 10 wherein the pose-tracking device is further configured for comparing the synthesized distorted pixel row with the input pixel row at multiple points that correspond to the centers of the plurality of segments.

12. The system of claim 10 wherein the pose-tracking device is further configured for extracting a distorted pixel row from the previously captured frames.

13. The system of claim 8 wherein the pose-tracking device is further configured for storing data corresponding to each of the input pixel row and subsequently computing the constraints.

14. The system of claim 8 wherein the plurality of camera devices is configured as a camera cluster positioned on a head-mounted display.

15. A non-transitory computer-readable medium having stored thereon executable instructions that when executed by a processor of a computer control the computer to perform steps comprising:

receiving, from each of a plurality of camera devices, an input pixel row of a radially distorted image;

conducting a row comparison between each of the input pixel rows and a respectively associated synthesized pixel row;

approximating, for each row comparison, a span of a curve in an image space with a plurality of line segments;

computing, for each of the plurality of segments, a constraint; and

utilizing the constraints to estimate a pose configuration.

16. The non-transitory computer-readable medium of claim 15 wherein the plurality of camera devices includes a plurality of rolling shutter cameras.

17. The non-transitory computer-readable medium of claim 15 wherein conducting a row comparison comprises generating the curve by synthesizing a distorted pixel row from previously captured frames of the radially distorted image using a homography-based warping technique.

18. The non-transitory computer-readable medium of claim 17 comprising comparing the synthesized distorted pixel row with the input pixel row at multiple points that correspond to the centers of the plurality of segments.

19. The non-transitory computer-readable medium of claim 17 wherein synthesizing a distorted pixel row includes extracting a distorted pixel row from the previously captured frames.

20. The non-transitory computer-readable medium of claim 17 comprising storing data corresponding to each of the input pixel row and subsequently computing the constraints.