Methods and systems for 360-degree video post-production

• US 10,554,948 B2
• Filed: 04/30/2018
• Issued: 02/04/2020
• Est. Priority Date: 05/05/2017
• Status: Active Grant

• Current Assignee: Torus Media Labs Inc.
• Original Assignee: Torus Media Labs Inc.
• Inventors: Sevigny, Michel
• Primary Examiner(s): Wu, Sing-Wai
• Application Number:
  US15/967,466
• Publication Number:
  US 20180324404A1
• Time in Patent Office: 645 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/351 for displaying simultaneously
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

In accordance with example embodiments, the method and system for 360-degree video post-production generally makes use of points of view (POVs) to facilitate the 360-degree video post-production process. A POV is a rectilinear subset view of a 360-degree composition based on a particular focal length, angle of view, and orientation for each frame of the 360-degree composition. Video post-production editing can be applied to a POV by the user, using rectilinear video post-production methods or systems. The rectilinear video post-production editing done on the POV is integrated back into the 360-degree environment of the 360-degree composition. In accordance with example embodiments, the method and system for 360-degree video post-production comprises identifying a point of view in a 360-degree composition; applying video post-production editing to the point of view; and aligning a new layer containing the video post-production editing with the point of view in the 360-degree composition.

Claims

1. A method for 360-degree video post-production, the method being performed by at least one processor, the method comprising:

displaying an interface screen for a 360-degree composition on an output interface device comprising a display, the interface screen configured to display an omni directional camera viewer that is configured to display a rectilinear view of the 360-degree composition as a consequence to orientation navigating through the omni directional camera viewer;

identifying a point of view comprising receiving selection through the interface screen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view, comprising tracking the identified point of view to identify tracking information comprising one or more point cloud layers and a tracked camera layer;

linking the one or more point cloud layers to the tracked camera layer;

aligning the one or more point cloud layers in the 360-degree composition with the identified point of view; and

saving one or more layers, comprising the point cloud layers and the tracked camera layer, to memory.

2. A method for 360-degree video post-production in accordance with claim 1, further comprising:

identifying an existing or additional point of view through the interface screen;

said applying video post-production editing comprising video post-production editing of the identified existing or additional point of view to create one or more additional layers;

aligning the one or more additional layers in the 360-degree composition with the identified existing or additional point of view; and

saving the one or more additional layers to memory.

3. A method for 360-degree video post-production in accordance with claim 2, wherein:

said applying video post-production editing comprises importing media of a media file.

4. A method for 360-degree video post-production in accordance with claim 3, wherein the media file is a 360-degree media file of 360-degree footage taken from a 360-degree camera.

5. A method for 360-degree video post-production in accordance with claim 1, further comprising stabilizing the 360-degree composition based on the tracking information.

6. A method for 360-degree video post-production in accordance with claim 5, wherein said stabilizing further comprises:

orienting the 360-degree composition with an inversion of an orientation of the tracked camera layer; and

updating an orientation of the identified point of view to match the orientation of the tracked camera layer after the tracked camera layer is aligned with the identified point of view.

7. A method for 360-degree video post-production in accordance with claim 6, wherein said stabilizing further comprises:

selecting one or more points of view in addition to the identified point of view;

calculating a difference in orientation between the selected one or more points of view and the identified point of view before the orientation of the identified point of view is updated to match the orientation of the tracked camera layer; and

updating the orientation of the one or more selected points of view to match the addition of the calculated difference and the orientation of the tracked camera layer after the tracked camera layer is aligned with the identified point of view.

8. A method for 360-degree video post-production in accordance with claim 1, wherein said orientation navigating and/or said receiving selection are controlled using an input interface device.

9. A method for 360-degree video post-production in accordance with claim 1, wherein the displayed rectilinear view is also the consequence of navigating one or more of the angle of view and focal length.

10. A method for 360-degree video post-production in accordance with claim 1, wherein the displayed rectilinear view is generated by identifying and displaying a subset of the 360-degree composition corresponding to the displayed rectilinear view.

11. A method for 360-degree video post-production in accordance with claim 1, wherein the identified point of view includes a particular focal length, angle of view, and orientation of the 360-degree composition.

12. A method for 360-degree video post-production in accordance with claim 1, wherein the identified point of view is selectable from any viewing angle.

13. A method for 360-degree video post-production in accordance with claim 1, further comprising rendering a new 360-degree video file which renders the 360-degree composition together with the one or more layers, and saving the new 360-degree video file to the memory.

14. A method for 360-degree video post-production in accordance with claim 13, wherein the new 360-degree video file is rendered by flattening all of the one or more layers in the 360-degree composition.

15. A method for 360-degree video post-production in accordance with claim 2, wherein the video post-production editing comprises:

generating a sub-composition of the identified existing or additional point of view; and

said applying video post-production editing comprises applying rectilinear post-production editing to the sub-composition.

16. A method for 360-degree video post-production in accordance with claim 1, wherein the video post-production editing comprises rectilinear post-production editing.

17. A method for 360-degree video post-production in accordance with claim 1, further comprising outputting the 360-degree composition together with the one or more layers to a display.

18. A method for 360-degree video post-production in accordance with claim 17, wherein the display is a 360-degree display.

19. A system for 360-degree video post-production, comprising:

memory;

an output interface device comprising a display;

an input interface device;

a processor operably coupled to the memory, the output interface device and the input interface device, the processor for executing instructions stored in the memory which, when executed, causes the processor to perform;

displaying an interface screen for a 360-degree composition on an output interface device comprising a display, the interface screen configured to display an omni directional camera viewer that is configured to display a rectilinear view of the 360-degree composition as a consequence to orientation navigating through the omni directional camera viewer;

identifying a point of view comprising receiving selection through the interface screen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view, comprising tracking the identified point of view to identify tracking information comprising one or more point cloud layers and a tracked camera layer;

linking the one or more point cloud layers to the tracked camera layer;

aligning the one or more point cloud layers in the 360-degree composition with the identified point of view; and

saving one or more layers, comprising the point cloud layers and the tracked camera layer, to memory.

20. A non-transitory computer readable medium comprising instructions for 360-degree video post-production and executable by one or more processors, the instructions comprising instructions to perform:

displaying an interface screen for a 360-degree composition on an output interface device comprising a display, the interface screen configured to display an omni directional camera viewer that is configured to display a rectilinear view of the 360-degree composition as a consequence to orientation navigating through the omni directional camera viewer;

identifying a point of view comprising receiving selection through the interface screen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view, comprising tracking the identified point of view to identify tracking information comprising one or more point cloud layers and a tracked camera layer;

linking the one or more point cloud layers to the tracked camera layer;

aligning the one or more point cloud layers in the 360-degree composition with the identified point of view; and

saving one or more layers, comprising the point cloud layers and the tracked camera layer, to memory.