Methods and apparatus for providing rotated spherical viewpoints

• US 10,650,592 B2
• Filed: 12/27/2018
• Issued: 05/12/2020
• Est. Priority Date: 01/23/2017
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Abbas, Adeel
• Primary Examiner(s): Torrente, Richard T
• Application Number:
  US16/234,346
• Publication Number:
  US 20190244424A1
• Time in Patent Office: 502 Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations

Abstract

Apparatus and methods for providing a rotated spherical viewpoint (RSV). In one or more embodiments, the RSV is implemented using equirectangular projections (ERPs). The RSV methodologies described herein are particularly suitable for panning within virtual reality (VR) or 360° panoramic content as there is minimal, if any, geometric distortions at the edge of a given viewport. Moreover, the RSV methodologies described herein may reduce the bandwidth requirements of, for example, devices operating in a client-server architecture. Additionally, the computation requirements for providing RSV are minimal and can be provided using relatively simple rotation transforms.

Claims

1. A method for generating a rotated sphere viewpoint (RSV), comprising:

retrieving first image data associated with a first portion of interest of a source image, the source image comprising a spherical image, the first image data being associated with an equatorial portion;

receiving data relating to a request for second image data;

based on the received data relating to the request for the second image data, transforming the first image data, the transforming comprising algorithmically rotating the source image such that the second image data is associated with a new equatorial portion; and

based on the transformed first image data, generating a second image associated with the second image data via conversion of the transformed first image data into an equirectangular projection (ERP) projection, and causing provision of the second image to a viewport, the ERP projection being disposed over the source image such that (i) prior to the transforming of the first image data, a surface of the ERP projection is collinear with the equatorial portion associated with the first image data, and (ii) subsequent to the transforming of the first image data, the surface of the ERP projection is collinear with the new equatorial portion.

2. The method of claim 1, wherein the generating of the second image comprises algorithmically converting the transformed first image data into an equirectangular projection (ERP) projection.

3. The method of claim 1, wherein the generating of the second image comprises generating an image with a distortion level of at least a portion of the first portion of interest that is less than a distortion level of a projection associated with the source image.

4. The method of claim 1, further comprising retrieving source image data, the source image data comprising at least data representative of a sequence of images, the sequence of images comprising the source image.

5. The method of claim 1, further comprising receiving the second image data, the second image data being associated with a second image that is offset latitudinally from the equatorial portion associated with a first image associated with the first image data.

6. The method of claim 1, wherein the transforming of the first image data comprises:

generating a virtual geometric structure; and

determining a new portion of interest from the first image data;

wherein the new portion of interest resides on the new equatorial portion, the new equatorial portion residing on the generated virtual geometric structure.

7. A computerized apparatus configured to generate a rotated sphere viewpoint (RSV), the computerized apparatus being further configured to:

retrieve first image frame data relating to a first image frame associated with a first portion of interest, the first image frame data being obtained from source data, the source data comprising a spherical image;

receive data relating to a request for a second image frame associated with a second portion of interest, the second image frame being offset in at least one dimension with respect to an equatorial portion that is associated with the first image frame;

based on the received data relating to the request for the second image frame, transform the first image frame data via at least a rotation of the spherical image to produce a transformed image frame that resides on a new equatorial portion; and

based on the transformed image frame, generate and cause provision of a second image frame, the generation of the second image frame comprising conversion of the transformed image frame into an equirectangular projection (ERP) projection;

wherein;

prior to the transformation of the first image frame, the ERP projection is disposed over the spherical image such that the equatorial portion associated with the first image frame is collinear with a surface of the ERP projection; and

subsequent to the transformation of the first image frame, the ERP projection is disposed over the spherical image such that the new equatorial portion is collinear with the surface of the ERP projection.

8. The apparatus of claim 7, wherein:

the first portion of interest resides on the equatorial portion associated with the first image frame, and the second portion of interest resides on the new equatorial portion; and

the transformed first image frame is reduced in at least a level of distortion associated with the first portion of interest.

9. The apparatus of claim 8, wherein the transformation of the first image frame further comprises a rotation of the spherical image associated with the first image frame.

10. The apparatus of claim 9, wherein the offset of the second image frame in at least one dimension comprises a latitudinal offset from the first portion of interest, the latitudinal offset being based on the rotation of the spherical image.

11. The apparatus of claim 7, wherein the source data comprises at least one video sequence.

12. The apparatus of claim 7, wherein:

the apparatus comprises a network interface configured to communicate with a display device; and

the provision of the resultant image frame comprises provision of the resultant image frame to the display device.

13. A non-transitory computer-readable apparatus comprising a storage medium, the storage medium having a computer program stored thereon, the computer program comprising a plurality of instructions configured to, when executed by a processor apparatus, cause a computing device to:

retrieve first image data associated with a first portion of interest within a source image, the source image comprising a spherical image, the first image data being associated with a first equatorial portion;

based at least on data relating to a different portion of interest than the first portion, transform the first image data via at least a rotation of the source image such that the different portion of interest resides on a second equatorial portion different than the first equatorial portion; and

using at least the transformed first image data, generate resultant image data corresponding to the different portion of interest via at least conversion of the transformed first image data into an equirectangular projection (ERP) projection, and provide the resultant image data to a viewport;

wherein, prior to the transformation of the first image data, the ERP projection is configured such that a surface of the ERP projection is collinear with the first equatorial portion associated with the first image frame; and

wherein, subsequent to the transformation of the first image frame, the ERP projection is configured such that the surface of the ERP projection is collinear with the second equatorial portion.

14. The non-transitory computer-readable apparatus of claim 13, wherein the transformation of the first image data comprises rotation of the source image.

15. The non-transitory computer-readable apparatus of claim 13, wherein the generation of the resultant image data comprises conversion of the transformed first image data into an equirectangular projection (ERP) projection.

16. The non-transitory computer-readable apparatus of claim 13, wherein:

the first portion of interest resides on the first equatorial portion, and the different portion of interest resides on the second equatorial portion;

the different portion includes at least some of the first image data; and

the transformation of the first image data comprises a reduction of at least a level of distortion associated with the first portion of interest.

17. The non-transitory computer-readable apparatus of claim 13, wherein:

the provision of the resultant image data to the viewport comprises enablement of a render of the ERP projection on the viewport; and

the rendered ERP projection is associated with a view angle, the view angle comprising a latitudinal angle and a range of longitudinal angles.