Systems and methods for modifying image distortion (curvature) for viewing distance in post capture

• US 10,817,976 B2
• Filed: 08/09/2019
• Issued: 10/27/2020
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Newman, David, Adsumilli, Balu
• Primary Examiner(s): Findley, Christopher G
• Application Number:
  US16/536,515
• Publication Number:
  US 20190362465A1
• Time in Patent Office: 445 Days
• Field of Search: None
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/053 Detail-in-context presentations fisheye or wide-angle transformations G06T3/047
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06V 10/22 by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Systems and methods for modifying image distortion (curvature) for viewing distance in post capture. Presentation of imaging content on a content display device may be characterized by a presentation field of view (FOV). Presentation FOV may be configured based on screen dimensions of the display device and distance between the viewer and the screen. Imaging content may be obtained by an activity capture device characterized by a wide capture field of view lens (e.g., fish-eye). Images may be transformed into rectilinear representation for viewing. When viewing images using presentation FOV that may narrower than capture FOV, transformed rectilinear images may appear distorted. A transformation operation may be configured to account for presentation FOV-capture FOV mismatch. In some implementations, the transformation may include fish-eye to rectilinear transformation characterized by a transformation strength that may be configured based on a ratio of the presentation FOV to the capture FOV.

Claims

1. A system for providing images of imaging content, the system comprising:

one or more processors configured to execute a plurality of computer readable instructions configured to;

access the imaging content, the imaging content characterized by a capture field of view and a capture projection;

determine a presentation field of view for displaying the images of the imaging content on a display based on dimensions of the display and a viewing distance between the display and a user; and

transform the images of the imaging content from the capture projection to a display projection for presentation on the display based on the capture field of view and the presentation field of view such that curvature of the images are changed based on size relationship between the capture field of view and the presentation field of view.

2. The system of claim 1, wherein the imaging content is characterized by a fish-eye projection.

3. The system of claim 1, wherein the viewing distance between the display and the user is determined based on sensor information generated by a sensor.

4. The system of claim 3, wherein the sensor includes a proximity sensor or a light sensor.

5. The system of claim 1, wherein the viewing distance between the display and the user is determined based on a typical viewing distance associated with a type of the client device.

6. The system of claim 1, wherein the transformation of the images accounts for a mismatch between size of the capture field of view and size of the presentation field of view such that the curvature of the images is reduced as the size of the presentation field of view approaches the size of the capture field of view.

7. The system of claim 6, wherein the transformation of the images includes a curve-to-rectilinear transformation based on a match between the size of the capture field of view and the size of the presentation field of view.

8. A method for transforming images of imaging content, the method comprising:

obtaining a capture field of view of the imaging content using metadata of the imaging content, the imaging content characterized by a capture projection;

obtaining a viewing distance between a display and a user;

obtaining a viewing field of view based on the viewing distance and a dimension of the display; and

transforming the images of the imaging content from the capture projection to a display projection based on the capture field of view and the viewing field of view such that curvature of the images are changed based on size relationship between the capture field of view and the viewing field of view.

9. The method of claim 8, wherein the transformation of the images reduces apparent motion within the images of the imaging content.

10. The method of claim 8, wherein the transformation of the images is performed by a server.

11. The method of claim 8, wherein the transformation of the images is performed by a camera or a smartphone.

12. The method of claim 8, wherein the transformation of the images is performed in real-time as a function of the viewing distance between the display and the user.

13. The method of claim 12, wherein the transformation of the images includes a first image transformation based on a first viewing field of view and a second image transformation based on a second viewing field of view.

14. The method of claim 12, wherein the viewing distance is obtained based on sensor information generated by a proximity sensor or a light sensor.

15. A portable content display system, comprising:

a display; and

one or more processors configured to execute a plurality of computer readable instructions configured to;

access a source image of imaging content, the source image characterized by a capture field of view and a capture projection;

obtain a presentation field of view based on dimensions of the display and a viewing distance between the display and a user; and

obtain a transformed image based on transformation of the source image from the capture projection to a display projection, wherein the transformation of the source image is performed based on the capture field of view and the presentation field of view such that curvature of the images are changed based on size relationship between the capture field of view and the presentation field of view.

16. The portable content display system of claim 15, wherein the transformation is performed based on a ratio of size of the capture field of view to size of the presentation field of view.

17. The portable content display system of claim 15, wherein the imaging content includes a video or a sequence of images.

18. The portable content display system of claim 15, wherein the capture field of view is obtained based on curvature information of an optical element used to capture the imaging content.

19. The portable content display system of claim 18, wherein the curvature information includes coefficients of a polynomial representation characterizing a curvature of the optical element.

20. The portable content display system of claim 18, wherein the curvature information is stored in metadata of the imaging content.