Methods and systems for 360-degree video post-production

• US 9,959,905 B1
• Filed: 05/05/2017
• Issued: 05/01/2018
• Est. Priority Date: 05/05/2017
• Status: Active Grant

• Current Assignee: Torus Media Labs Inc.
• Original Assignee: Torus Media Labs Inc.
• Inventors: Sevigny, Michel
• Primary Examiner(s): Wu, Sing-Wai
• Application Number:
  US15/588,073
• Time in Patent Office: 361 Days
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/685 performed by mechanical compensation
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

In accordance with example embodiments, the method and system for 360-degree video post-production generally makes use of points of view (POVs) to facilitate the 360-degree video post-production process. A POV is a rectilinear subset view of a 360-degree composition based on a particular focal length, angle of view, and orientation for each frame of the 360-degree composition. Video post-production editing can be applied to a POV by the user, using rectilinear video post-production methods or systems. The rectilinear video post-production editing done on the POV is integrated back into the 360-degree environment of the 360-degree composition. In accordance with example embodiments, the method and system for 360-degree video post-production comprises identifying a point of view in a 360-degree composition; applying video post-production editing to the point of view; and aligning a new layer containing the video post-production editing with the point of view in the 360-degree composition.

Claims

1. A method for 360-degree video post-production, the method being performed by at least one processor, the method comprising:

displaying an interface screen for a 360-degree composition on an output interface device comprising a display, the interface screen configured to display an omni directional camera viewer that is configured to display a rectilinear view of the 360-degree composition as a consequence to orientation navigating through the omni directional camera viewer;

identifying a point of view comprising receiving selection through the interface screen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view, comprising tracking the identified point of view to identify tracking information comprising one or more point cloud layers and a tracked camera layer;

linking the one or more point cloud layers to the tracked camera layer;

aligning the one or more point cloud layers with the identified point of view in the 360-degree composition;

identifying a further point of view comprising receiving selection through the interface screen of a further rectilinear view displayed in the interface screen;

said applying video post-production editing further being applied to the further point of view and further comprising tracking the further point of view to identify secondary tracking information comprising one or more secondary point cloud layers and a secondary tracked camera layer;

linking the one or more secondary point cloud layers to the secondary tracked camera layer;

aligning the secondary tracked camera layer with the identified further point of view;

scaling the secondary tracked camera layer position parameters to match position parameters of the tracked camera layer; and

saving one or more layers, comprising the points cloud layers and the tracked camera layers, to memory.

2. A method for 360-degree video post-production in accordance with claim 1, further comprising:

identifying an existing or additional point of view;

said applying video post-production editing comprising video post-production editing of the identified existing or additional point of view to create one or more additional layers;

aligning the one or more additional layers with the identified existing or additional point of view in the 360-degree composition; and

saving the one or more additional layers to memory.

3. A method for 360-degree video post-production in accordance with claim 2, wherein:

said applying video post-production editing comprises importing media of a media file.

4. A method for 360-degree video post-production in accordance with claim 3, wherein the media file is a 360-degree media file of 360-degree footage take from a 360-degree camera.

5. A method for 360-degree video post-production in accordance with claim 2, further comprising rendering a new 360-degree video file which renders the 360-degree composition together with the one or more layers, and saving the new 360-degree video file to the memory.

6. A system for 360-degree video post-production in accordance with claim 5, wherein the new 360-degree video file is rendered by flattening all of the one or more layers in the 360-degree composition.

7. A method for 360-degree video post-production in accordance with claim 2, wherein the video post-production editing comprises:

generating a sub-composition of the identified existing or additional point of view; and

said applying video post-production editing comprises applying rectilinear post-production editing to the sub-composition.

8. A method for 360-degree video post-production in accordance with claim 1, further comprising:

orienting the 360-degree composition with an inversion of an orientation of the tracked camera layer; and

updating an orientation of the identified point of view to match the orientation of the tracked camera layer after the tracked camera layer is aligned with the identified point of view.

9. A method for 360-degree video post-production in accordance with claim 8, further comprising:

selecting one or more points of view in addition to the identified point of view;

calculating a difference in orientation between the selected one or more points of view and the identified point of view before the orientation of the identified point of view is updated to match the orientation of the tracked camera layer; and

updating the orientation of the one or more selected points of view to match the addition of the calculated difference and the orientation of the tracked camera layer after the tracked camera layer is aligned with the identified point of view.

10. A method for 360-degree video post-production in accordance with claim 1, wherein said orientation navigating and/or said receiving selection are controlled using an input interface device.

11. A method for 360-degree video post-production in accordance with claim 1, wherein the displayed rectilinear view is also the consequence of navigating one or more of the angle of view and focal length.

12. A method for 360-degree video post-production in accordance with claim 1, wherein the displayed rectilinear view is generated by identifying and displaying a subset of the 360-degree composition corresponding to the displayed rectilinear view.

13. A method for 360-degree video post-production in accordance with claim 1, wherein the identified point of view includes a particular focal length, angle of view, and orientation of the 360-degree composition.

14. A method for 360-degree video post-production in accordance with claim 1, wherein the identified point of view is selectable from any viewing angle.

15. A method for 360-degree video post-production in accordance with claim 1, further comprising stabilizing the 360-degree composition based on the tracking information.

16. A method for 360-degree video post-production in accordance with claim 1, further comprising outputting the 360-degree composition together with the one or more layers to a 360-degree display.

17. A method for 360-degree video post-production in accordance with claim 1, wherein said scaling further comprises scaling the one or more secondary point cloud layers based on said secondary tracked camera layer position parameters.

18. A system for 360-degree video post-production, comprising:

memory;

an output interface device comprising a display;

an input interface device;

a processor operably coupled to the memory, the output interface device and the input interface device, the processor for executing instructions stored in the memory which, when executed, causes the processor to perform;

displaying an interface screen for a 360-degree composition on an output interface device comprising a display, the interface screen configured to display an omni directional camera viewer that is configured to display a rectilinear view of the 360-degree composition as a consequence to orientation navigating through the omni directional camera viewer;

identifying a point of view comprising receiving selection through the interface screen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view, comprising tracking the identified point of view to identify tracking information comprising one or more point cloud layers and a tracked camera layer;

linking the one or more point cloud layers to the tracked camera layer;

aligning the one or more point cloud layers with the identified point of view in the 360-degree composition;

identifying a further point of view comprising receiving selection through the interface screen of a further rectilinear view displayed in the interface screen;

said applying video post-production editing being applied to the further point of view and further comprising tracking the further point of view to identify secondary tracking information comprising one or more secondary point cloud layers and a secondary tracked camera layer;

linking the one or more secondary point cloud layers to the secondary tracked camera layer;

aligning the secondary tracked camera layer with the identified further point of view;

scaling the secondary tracked camera layer position parameters to match position parameters of the tracked camera layer; and

saving one or more layers, comprising the points cloud layers and the tracked camera layers, to memory.

19. A non-transitory computer readable medium comprising instructions for 360-degree video post-production and executable by one or more processors, the instructions comprising instructions to perform:

displaying an interface screen for a 360-degree composition on an output interface device comprising a display, the interface screen configured to display an omni directional camera viewer that is configured to display a rectilinear view of the 360-degree composition as a consequence to orientation navigating through the omni directional camera viewer;

identifying a point of view comprising receiving selection through the interface screen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view, comprising tracking the identified point of view to identify tracking information comprising one or more point cloud layers and a tracked camera layer;

linking the one or more point cloud layers to the tracked camera layer;

aligning the one or more point cloud layers with the identified point of view in the 360-degree composition;

identifying a further point of view comprising receiving selection through the interface screen of a further rectilinear view displayed in the interface screen;

said applying video post-production editing being applied to the further point of view and further comprising tracking the further point of view to identify secondary tracking information comprising one or more secondary point cloud layers and a secondary tracked camera layer;

linking the one or more secondary point cloud layers to the secondary tracked camera layer;

aligning the secondary tracked camera layer with the identified further point of view;

scaling the secondary tracked camera layer position parameters to match position parameters of the tracked camera layer; and

saving one or more layers, comprising the points cloud layers and the tracked camera layers, to memory.