Equatorial stitching of hemispherical images in a spherical image capture system

• US 10,332,237 B2
• Filed: 08/06/2018
• Issued: 06/25/2019
• Est. Priority Date: 08/12/2015
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Steel, Joseph, Macmillan, Timothy
• Primary Examiner(s): Chen, Xuemei G
• Application Number:
  US16/056,370
• Publication Number:
  US 20190043160A1
• Time in Patent Office: 323 Days
• Field of Search: 345427, 345441, 382284
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/073 Transforming surfaces of revolution to planar images, e.g. cylindrical surfaces to planar images

Abstract

Hyper-hemispherical images may be combined to generate a rectangular projection of a spherical image having an equatorial stitch line along of a line of lowest distortion in the two images. First and second circular images are received representing respective hyper-hemispherical fields of view. A video processing device may project each circular image to a respective rectangular image by mapping an outer edge of the circular image to a first edge of the rectangular image and mapping a center point of the circular image to a second edge of the first rectangular image. The rectangular images may be stitched together along the edges corresponding to the outer edge of the original circular image.

Claims

1. A camera apparatus configured to stitch hyper-hemispherical images, the camera apparatus, comprising:

a first camera that faces a first direction, the first camera configured to capture a first circular image corresponding to a first hyper-hemispherical field of view;

a second camera that faces a second direction, the second camera configured to capture a second circular image corresponding a second hyper-hemispherical field of view;

a processing apparatus; and

a non-transitory computer-readable apparatus comprising a storage medium having instructions for stitching the first and the second hyper-hemispherical fields of view to generate a rectangular projection of a spherical image, the instructions when executed by the processing apparatus, cause the processing apparatus to;

receive the first circular image corresponding to the first hyper-hemispherical field of view captured by the first camera facing the first direction and the second circular image corresponding to the second hyper-hemispherical field of view captured by the second camera facing the second direction;

project the first circular image to a first rectangular image via a map of an outer edge of the first circular image to a bottom edge of the first rectangular image and a map of a center point of the first circular image to a top edge of the first rectangular image;

project the second circular image to a second rectangular image via a map of an outer edge of the second circular image to a top edge of the second rectangular image and a map of a center point of the second circular image to a bottom edge of the second rectangular image; and

stitch the bottom edge of the first rectangular image with the top edge of the second rectangular image to generate the rectangular projection of the spherical image.

2. The camera apparatus of claim 1, wherein the stitch of the bottom edge of the first rectangular image with the top edge of the second rectangular image further comprises:

an alignment of features in a first overlapping range of the first rectangular image near the bottom edge with features in a corresponding second overlapping range of the second rectangular image near the top edge.

3. The camera apparatus of claim 1, wherein the instructions when executed by the processing apparatus, cause the processing apparatus to:

re-orient the rectangular projection to generate a re-orientated rectangular projection in which a center of the first circular image corresponds to a center point of the re-oriented rectangular projection.

4. The camera apparatus of claim 1, wherein the projection of the first circular image further comprises:

a map of each concentric ring of pixels in the first circular image to respective rows of pixels, wherein concentric rings from a center of the first circular image to an edge of the first circular image are mapped from a top row to a bottom row; and

horizontally stretch the rows of pixels to fill a fixed size rectangle.

5. The camera apparatus of claim 4, wherein the projection of the second circular image comprises:

a map of each concentric ring of pixels in the second circular image to respective rows of pixels, wherein concentric rings from a center of the second circular image to an edge of the second circular image are mapped from a bottom row to a top row; and

horizontally stretch the rows of pixels to fill a fixed size rectangle.

6. The camera apparatus of claim 1, wherein a stitch line between the first rectangular image and the second rectangular image corresponds to boundaries between the first hyper-hemispherical field of view and the second hyper-hemispherical field of view.

7. The camera apparatus of claim 1, wherein the first rectangular image comprises a 360 degree field of view in the horizontal direction and a 90+n degree field of view in the vertical direction, where n is an amount of overlap between the first hyper-hemispherical field of view and the second hyper-hemispherical field of view.

8. The camera apparatus of claim 1, wherein the first direction faces a different direction than the second direction.

9. The camera apparatus of claim 8, wherein the first camera and the second camera are disposed in a back-to-back orientation.

10. The camera apparatus of claim 9, wherein the rectangular projection of the spherical image comprises a polar view of the spherical image and a stitch line resultant from the stitch of the bottom edge of the first rectangular image with the top edge of the second rectangular image comprises an equatorial stitch line.

11. A computing device configured to stitch hyper-hemispherical images, the computing device, comprising:

a processing apparatus; and

a non-transitory computer-readable apparatus comprising a storage medium having instructions for stitching the first and the second hyper-hemispherical fields of view to generate a rectangular projection of a spherical image, the instructions when executed by the processing apparatus, cause the processing apparatus to;

receive a first circular image corresponding to a first hyper-hemispherical field of view captured by a first camera facing a first direction and a second circular image corresponding to a second hyper-hemispherical field of view captured by a second camera facing a second direction;

project the first circular image to a first rectangular image via a map of an outer edge of the first circular image to a bottom edge of the first rectangular image and a map of a center point of the first circular image to a top edge of the first rectangular image;

project the second circular image to a second rectangular image via a map of an outer edge of the second circular image to a top edge of the second rectangular image and a map of a center point of the second circular image to a bottom edge of the second rectangular image; and

stitch the bottom edge of the first rectangular image with the top edge of the second rectangular image to generate the rectangular projection of the spherical image.

12. The computing device of claim 11, wherein the stitch of the bottom edge of the first rectangular image with the top edge of the second rectangular image further comprises:

an alignment of features in a first overlapping range of the first rectangular image near the bottom edge with features in a corresponding second overlapping range of the second rectangular image near the top edge.

13. The computing device of claim 11, wherein the instructions when executed by the processing apparatus, cause the processing apparatus to:

re-orient the rectangular projection to generate a re-orientated rectangular projection in which a center of the first circular image corresponds to a center point of the re-oriented rectangular projection.

14. The computing device of claim 11, wherein the projection of the first circular image further comprises:

a map of each concentric ring of pixels in the first circular image to respective rows of pixels, wherein concentric rings from a center of the first circular image to an edge of the first circular image are mapped from a top row to a bottom row; and

horizontally stretch the rows of pixels to fill a fixed size rectangle.

15. The computing device of claim 14, wherein the projection of the second circular image comprises:

a map of each concentric ring of pixels in the second circular image to respective rows of pixels, wherein concentric rings from a center of the second circular image to an edge of the second circular image are mapped from a bottom row to a top row; and

horizontally stretch the rows of pixels to fill a fixed size rectangle.

16. The computing device of claim 11, wherein a stitch line between the first rectangular image and the second rectangular image corresponds to boundaries between the first hyper-hemispherical field of view and the second hyper-hemispherical field of view.

17. The computing device of claim 11, wherein the first rectangular image comprises a 360 degree field of view in the horizontal direction and a 90+n degree field of view in the vertical direction, where n is an amount of overlap between the first hyper-hemispherical field of view and the second hyper-hemispherical field of view.

18. The computing device of claim 11, wherein the computing device further comprises a display driver configured to display a portion of the rectangular projection of the spherical image.

19. The computing device of claim 18, wherein the rectangular projection of the spherical image comprises a polar view of the spherical image and a stitch line resultant from the stitch of the bottom edge of the first rectangular image with the top edge of the second rectangular image comprises an equatorial stitch line.

20. The computing device of claim 18, wherein the stitch line resides within an area of the first hyper-hemispherical field of view and the second hyper-hemispherical field of view that has a lower amount of distortion as compared with other areas of the first hyper-hemispherical field of view and the second hyper-hemispherical field of view.