Systems and methods for modifying image distortion (curvature) for viewing distance in post capture

• US 10,402,938 B1
• Filed: 03/31/2017
• Issued: 09/03/2019
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Newman, David, Adsumilli, Balu
• Primary Examiner(s): Findley, Christopher G
• Application Number:
  US15/476,884
• Time in Patent Office: 886 Days
• Field of Search: None
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/053 Detail-in-context presentations fisheye or wide-angle transformations G06T3/047
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06V 10/22 by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Systems and methods for modifying image distortion (curvature) for viewing distance in post capture. Presentation of imaging content on a content display device may be characterized by a presentation field of view (FOV). Presentation FOV may be configured based on screen dimensions of the display device and distance between the viewer and the screen. Imaging content may be obtained by an activity capture device characterized by a wide capture field of view lens (e.g., fish-eye). Images may be transformed into rectilinear representation for viewing. When viewing images using presentation FOV that may narrower than capture FOV, transformed rectilinear images may appear distorted. A transformation operation may be configured to account for presentation FOV-capture FOV mismatch. In some implementations, the transformation may include fish-eye to rectilinear transformation characterized by a transformation strength that may be configured based on a ratio of the presentation FOV to the capture FOV.

Claims

1. A system for providing images of imaging content, the system comprising:

an electronic storage configured to store the imaging content;

a communications interface configured to communicate a bit stream to a client device, the client device including a display; and

one or more processors configured to execute a plurality of computer readable instructions configured to;

access the imaging content characterized by a capture field of view and a capture projection;

determine a presentation field of view associated with displaying the images of the imaging content on the display, wherein the presentation field of view is determined based on dimensions of the display and a viewing distance between the display and a user;

obtain a transformation operation configured to transform the images of the imaging content from the capture projection based on the capture field of view and the presentation field of view; and

transform the images of the imaging content for presentation on the display using the transformation operation.

2. The system of claim 1, wherein the capture projection includes a fish eye projection.

3. The system of claim 1, wherein the viewing distance between the display and the user is determined based on sensor information generated by a sensor.

4. The system of claim 3, wherein the sensor includes a proximity sensor or a light sensor.

5. The system of claim 1, wherein the viewing distance between the display and the user is determined based on a typical viewing distance associated with a type of the client device.

6. The system of claim 1, wherein the transformation operation accounts for a mismatch between the capture field of view and the presentation field of view.

7. The system of claim 1, wherein the transformation operation includes a curve-to-rectilinear transformation based on a match between the capture field of view and the presentation field of view.

8. A method for transforming images of imaging content, the method comprising:

obtaining an image capture field of view using a meta data portion of the imaging content;

determining a dimension of a display of a content display device;

obtaining a viewing distance between the display and a user using sensor information;

obtaining a viewing field of view based on the viewing distance and the dimension of the display; and

obtaining an image transformation operation to transform the images of the imaging content from an image source projection based on the capture field of view and the viewing field of view.

9. The method of claim 8, wherein the image transformation operation reduces apparent motion within the images of the imaging content.

10. The method of claim 8, wherein the image transformation operation is applied to the images of the imaging content by a server.

11. The method of claim 8, wherein the image transformation operation is applied to the images of the imaging content by a camera or a smartphone.

12. The method of claim 8, wherein the image transformation operation is applied to the images of the imaging content in real-time as a function of the viewing distance between the display and the user.

13. The method of claim 12, wherein the image transformation operation includes a first image transformation operation obtained based on a first viewing field of view and a second image transformation operation obtained based on a second viewing field of view.

14. A portable content display system, comprising:

a display;

a communications interface; and

one or more processors configured to execute a plurality of computer readable instructions configured to;

access a source image of imaging content via the communications interface;

obtain a capture field of view of the source image;

obtain dimensions of the display;

obtain a presentation field of view based on a viewing distance between the display and a user and the dimensions of the display;

obtain an image transformation to transform the source image from a capture projection to a display projection, the image transformation configured based on the capture field of view and the presentation field of view; and

obtain a transformed image based on applying the image transformation to the source image.

15. The portable content display system of claim 14, wherein the image transformation is configured based on a ratio of the capture field of view to the presentation field of view.

16. The portable content display system of claim 14, wherein the imaging content includes a video or a sequence of images.

17. The portable content display system of claim 14, wherein the capture field of view is obtained based on curvature information of an optical element used to capture the imaging content.

18. The portable content display system of claim 17, wherein the curvature information includes coefficients of a polynomial representation characterizing a curvature of the optical element.

19. The portable content display system of claim 17, wherein the curvature information is stored in meta data of the imaging content.