System and method for processing digital video

• US 10,460,501 B2
• Filed: 07/04/2016
• Issued: 10/29/2019
• Est. Priority Date: 07/04/2016
• Status: Active Grant

• Current Assignee: Liquid Cinema Inc.
• Original Assignee: Liquid Cinema Inc.
• Inventors: Wallner, Thomas, Herman, Scott, Hildgen, Franz
• Primary Examiner(s): Chen, Frank S
• Application Number:
  US15/201,564
• Publication Number:
  US 20180005447A1
• Time in Patent Office: 1,212 Days
• Field of Search: None
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/205 Image-based rendering

Abstract

A computer-implemented method of processing digital video is provided. The method includes determining at least one frame region the contents of which would be rendered substantially invisible, were frames of the digital video to be subjected to a predetermined texture-mapping onto a predetermined geometry; and inserting non-image data into at least one selected frame by modifying contents within at least one determined frame region of the selected frame. Another computer-implemented method of processing digital video is provided. The method includes, for each of a plurality of frames of the digital video: processing contents in one or more predetermined regions of the frame to extract non-image data therefrom; subjecting the frame to a predetermined texture-mapping onto a predetermined geometry, wherein after the texture-mapping the contents of the one or more predetermined regions are rendered substantially invisible; and causing the texture-mapped frame to be displayed. Another computer-implemented method of processing digital video is provided. The method includes for each of a plurality of frames of the digital video: extracting a frame identifier uniquely identifying a respective frame by processing contents in one or more predetermined regions of the frame; and for each of a different plurality of frames of the digital video: estimating the frame identifier based on playback time of the digital video. Another computer-implemented method of processing digital video is provided. The method includes causing frames of the digital video to be displayed; for a period beginning prior to an estimated time of display of an event-triggering frame and ending after the estimated time, causing a frame that is to be displayed prior to the beginning of the period to remain displayed; and after the period, executing at least one event associated with the event-triggering frame and resuming display of subsequent frames of the digital video. Systems and computer-readable media are also provided.

Claims

1. A computer-implemented method of processing digital video, the method comprising:

determining at least one two-dimensional frame region of frames of the digital video, wherein each of the at least one two-dimensional frame region is defined by a respective vertex trio, that would be at least substantially mapped out of the frames upon executing a predetermined texture-mapping of the frames onto a predetermined three-dimensional geometry;

inserting non-image data into at least one selected frame of the digital video, the inserting comprising;

modifying contents only within one or more of the determined two-dimensional regions, wherein modifying contents only within one or more of the determined two-dimensional regions enables the inserted none image data to be at least substantially mapped out of each selected frame upon execution of the predetermined texture-mapping of each selected frame thereby to enable displaying each selected frame subsequent to the texture-mapping at least substantially without displaying the non-image data.

2. The computer-implemented method of claim 1, wherein the non-image data comprises:

a frame identifier uniquely identifying each of the at least one selected frame.

3. The computer-implemented method of claim 2, further comprising:

creating at least one association between a frame identifier and one or more events to be executed by a media player;

storing the at least one association with event parameters in a metadata file; and

associating the metadata file with the digital video.

4. The computer-implemented method of claim 3, wherein the one or more events comprise:

a forced perspective wherein, beginning with the selected frame, the view is forced to a predetermined visual perspective.

5. The computer-implemented method of claim 3, wherein the one or more events comprise:

a projection switch from a geometrical projection to a flat projection wherein, beginning with the selected frame, frames of the digital video are to be flat-projected by the media player prior to display.

6. The computer-implemented method of claim 3, wherein the one or more events comprise:

a projection switch from a flat projection to a geometrical projection wherein, beginning with the selected frame, frames of the digital video are to be texture-mapped by the media player to the predetermined geometry prior to display.

7. The computer-implemented method of claim 2, wherein each frame identifier comprises:

blocks of uniformly-coloured pixels inserted into respective ones of a plurality of the determined regions, each block of uniformly-coloured pixels being coloured according to a value.

8. The computer-implemented method of claim 7, wherein each of the uniformly-coloured pixels has a maximum intensity.

9. The computer-implemented method of claim 7, wherein the number of blocks is correlated with a total number of frames in the digital video.

10. The computer-implemented method of claim 9, wherein the value of each block is either 0 or 1.

11. The computer-implemented method of claim 2, wherein inserting the non-image data comprises:

based at least on the resolution of the digital video, selecting a frame identifier digital video from a set of frame identifier digital videos, the selected frame identifier digital video comprising frames having respective frame identifiers positioned and dimensioned to correspond to the at least one determined region; and

forming a composite video using the frame identifier digital video and the digital video.

12. The computer-implemented method of claim 11, wherein selecting the frame identifier digital video from a set of frame identifier digital videos is also based on the total number of frames of the digital video.

13. The computer-implemented method of claim 2, wherein inserting a respective frame identifier comprises:

based at least on the resolution of the digital video, executing a script to overlay respective frame identifiers onto the digital video in the determined at least one region.

14. The computer-implemented method of claim 13, wherein the script overlays frame identifiers onto respective frames based on the total number of frames in the digital video.

15. The computer-implemented method of claim 1, wherein the non-image data comprises:

at least one instruction for a media player.

16. The computer-implemented method of claim 15, wherein the at least one instruction comprises:

an instruction for the media player to execute an event when the media player is displaying the selected frame.

17. The computer-implemented method of claim 16, wherein the event comprises:

a forced perspective wherein, beginning with the selected frame, the view is forced to a predetermined visual perspective.

18. The computer-implemented method of claim 16, wherein the instruction comprises event parameters.

19. The computer-implemented method of claim 1, wherein the non-image data comprises:

digital rights management data.

20. The computer-implemented method of claim 1, wherein the predetermined geometry is a spherical mesh.

21. The method of claim 20, wherein the vertices within frames are determined by subdividing frame area into equal rectangular portions based on a number of latitudinal subdivisions, a number of longitudinal subdivisions and a radius of the spherical mesh, the vertices of the spherical mesh being based on the intersections of the latitudinal and longitudinal subdivisions according to the radius.

22. The method of claim 21, wherein determining at least one two-dimensional frame region comprises:

selecting equal rectangular portions that correspond to a mapped position adjacent to a zenith of the spherical mesh and/or the equal rectangular portions that correspond to a mapped position adjacent to a nadir of the spherical mesh;

wherein the determined regions are regions encompassed by respective triangles above each diagonal in respective ones of the selected equal rectangular portions.

23. The computer-implemented method of claim 1, further comprising:

determining resolution of the digital video,

wherein the step of determining at least one two-dimensional frame region comprises;

based on the resolution, determining dimension and position of the determined at least one region.

24. A computer-implemented method of processing digital video, the method comprising:

for each of a plurality of frames of the digital video;

processing contents in one or more predetermined two-dimensional regions of the frame to extract non-image data therefrom, the one or more predetermined regions defined by respective vertex trios;

executing a predetermined texture-mapping of the frame onto a three-dimensional geometry, wherein the predetermined two-dimensional regions are at least substantially mapped out of the frame by the executing of the predetermined texture-mapping thereby to at least substantially map the non-image data in the one or more predetermined regions out of the frame; and

subsequent to the at least substantially mapping the non-image data out of the frame, causing the texture-mapped frame to be displayed.

25. The computer-implemented method of claim 24, further comprising:

determining location and dimension of the predetermined regions based on resolution of the digital video.

26. The computer-implemented method of claim 24, wherein the predetermined geometry is a spherical mesh.

27. The computer-implemented method of claim 24, wherein the non-image data comprises at least one frame identifier each uniquely identifying a respective frame.

28. The computer-implemented method of claim 27, further comprising:

determining a metadata file associated with the digital video;

identifying any events associated in the metadata file with the at least one frame identifier; and

for each of the plurality of frames of the digital video, while causing the frame to be displayed;

executing, in accordance with event parameters stored in the metadata file, each of any events that is/are associated in the metadata file with the frame identifier for the frame.

29. The computer-implemented method of claim 28, wherein the events comprise:

a projection switch from a geometrical projection to a flat projection wherein, beginning with the frame to be displayed, frames of the digital video are to be flat-projected by the media player prior to display.

30. The computer-implemented method of claim 28, wherein the events comprise:

a projection switch from a flat projection to a geometrical projection wherein, beginning with the frame to be displayed, frames of the digital video are to be subjected to the predetermined texture-mapping onto the predetermined geometry.

31. The computer-implemented method of claim 28, wherein the at least one instruction comprises:

an instruction to execute an event upon displaying the selected frame.

32. The computer-implemented method of claim 31, wherein the instruction comprises event parameters.

33. The computer-implemented method of claim 24, wherein the non-image data comprises:

at least one instruction for a media player.

34. The computer-implemented method of claim 24, wherein the non-image data comprises:

digital rights management data.

35. A computer-implemented method of processing digital video, the method comprising:

causing frames of the digital video to be displayed;

for a period beginning prior to an estimated time of display of an event-triggering frame and ending after the estimated time, causing a frame that is to be displayed prior to the beginning of the period to remain displayed; and

after the period, executing at least one event associated with the event-triggering frame and resuming display of subsequent frames of the digital video.

36. A computer-implemented method of processing digital video, the method comprising:

for each of a plurality of frames of the digital video;

extracting a frame identifier uniquely identifying a respective frame by processing contents in one or more predetermined regions of the frame; and

for each of a different plurality of frames of the digital video;

estimating the frame identifier based on playback time of the digital video.

37. The computer-implemented method of claim 36, comprising:

conducting the extracting only in the event that a quality of the frames to be processed is at or above a threshold level; and

otherwise;

conducting the estimating.

38. The computer-implemented method of claim 37, further comprising:

causing the digital video to be displayed;

while conducting the estimating;

for a period beginning prior to an estimated time of display of an event-triggering frame and ending after the estimated time, causing a frame that is to be displayed prior to the beginning of the period to remain displayed; and

after the period, executing at least one event associated with the event-triggering frame and resuming display of subsequent frames of the digital video.

39. The computer-implemented method of claim 38, further comprising:

while conducting the extracting, executing at least one event associated with an event-triggering frame at the time of display of the event-triggering frame.

40. A non-transitory processor readable medium embodying a computer program for processing digital video, the computer program comprising:

program code for determining at least one two-dimensional frame region of frames of the digital video, wherein each of the at least one two-dimensional frame region is defined by a respective vertex trio, that would be at least substantially mapped out of the frames upon executing a predetermined texture-mapping of the frames onto a predetermined three-dimensional geometry;

program code for inserting non-image data into at least one selected frame of the digital video, the inserting comprising;

modifying contents only within one or more of the determined two-dimensional regions,

wherein modifying contents only within one or more of the determined two-dimensional regions enables the inserted non-image data to be at least substantially mapped out of each selected frame upon execution of the predetermined texture-mapping of each selected frame thereby to enable displaying each selected frame subsequent to the texture-mapping at least substantially without displaying the non-image data.

41. A non-transitory processor readable medium embodying a computer program for processing digital video, the computer program comprising:

program code that, for each of a plurality of frames of the digital video;

processes contents in one or more predetermined two-dimensional regions of the frame to extract non-image data therefrom, the one or more predetermined regions defined by respective vertex trios;

executes a predetermined texture-mapping of the frame onto a three-dimensional geometry, wherein the predetermined two-dimensional regions are at least substantially mapped out of the frame by the executing of the predetermined texture-mapping thereby to at least substantially map the non-image data in the one or more predetermined regions out of the frame; and

subsequent to the at least substantially mapping the non-image data out of the frame, causes the texture-mapped frame to be displayed.

42. A system for processing digital video comprising:

processing structure determining at least one two-dimensional frame region of frames of the digital video, wherein each of the at least one two-dimensional frame region is defined by a respective vertex trio, that would be at least substantially mapped out of the frames upon executing a predetermined texture-mapping of the frames onto a predetermined three-dimensional geometry, the processing structure inserting non-image data into at least one selected frame of the digital video by modifying contents only within one or more of the determined two-dimensional regions.

43. A system for processing digital video comprising:

processing structure that, for each of a plurality of frames of the digital video;

processes contents in one or more predetermined two-dimensional regions of the frame to extract non-image data therefrom, the one or more predetermined regions defined by respective vertex trios;

executes a predetermined texture-mapping of the frame onto a three-dimensional geometry, wherein the predetermined two-dimensional regions are at least substantially mapped out of the frame by the executing of the predetermined texture-mapping thereby to at least substantially map the non-image data in the one or more predetermined regions out of the frame; and

subsequent to the at least substantially mapping the non-image data out of the frame, causes the texture-mapped frame to be displayed.