ENCODING AND DECODING SELECTIVELY RETRIEVABLE REPRESENTATIONS OF VIDEO CONTENT

• US 20150382001A1
• Filed: 09/14/2015
• Published: 12/31/2015
• Est. Priority Date: 04/25/2003
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Newman, David A., Schunck, Brian G.
• Granted Patent: US 9,854,263 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G11B 20/00007 Time or data compression or expansion audio compression based on psychoacoustics G10L19/00; data processing for reproducing audio data at different playback speeds G10L21/04; video compression H04N19/00; data compression per se H03M7/30
• CPC Class Codes: G11B 2020/00072 the compressed signal including a video signal
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets

Abstract

A system and method disposed to enable encoding, decoding and manipulation of digital video with substantially less processing load than would otherwise be required. In particular, one disclosed method is directed to generating a compressed video data structure that is selectively decodable to a plurality of resolutions including the full resolution of the uncompressed stream. The desired number of data components and the content of the data components that make up the compressed video data, which determine the available video resolutions, are variable based upon the processing carried out and the resources available to decode and process the data components. During decoding, efficiency is substantially improved because only the data components necessary to generate a desired resolution are decoded. In variations, both temporal and spatial decoding are utilized to reduce frame rates, and hence, further reduce processor load. The system and method are particularly useful for real-time video editing applications.

Claims

1. A method for processing compressed video data, the method comprising:

accessing, by a computer system, a plurality of encoded video frames, each video frame comprising a plurality of components each associated with a different one of a plurality of image resolutions such that the video frame can be displayed at one of the plurality of image resolutions in part by combining all of the components associated with a resolution equal to or less than the resolution;

identifying a hardware display communicatively coupled to the computer system;

determining an available bandwidth between the computer system and the hardware display;

selecting one of the plurality of image resolutions based on the determined available bandwidth;

selecting, for each frame of the plurality of video frames, one or more image components of the frame, the selected image components comprising all image components of the frame associated with a resolution equal to or less than the selected resolution; and

transmitting, for each frame, the selected one or more image components to the hardware display, the hardware display configured to decode, for each frame, the transmitted one or more image components by combining the transmitted one or more image components to produce a video frame at the selected resolution.

2. The method of claim 1, wherein the resolution is selected by a user.

3. The method of claim 1, wherein the resolution is selected by the computer system automatically in response to the determination of the available bandwidth.

4. The method of claim 1, wherein the resolution is selected in response to a determination that the determined available bandwidth is lower than a threshold bandwidth required to support a current resolution, the selected resolution being less than the current resolution.

5. The method of claim 1, wherein the hardware display is located remotely from the computer system.

6. The method of claim 1, wherein the hardware display is communicatively coupled to the computer system via the internet.

7. The method of claim 1, wherein the hardware display is communicatively coupled to the computer system via a wireless network.

8. The method of claim 1, wherein one of the plurality of image resolutions comprises 4k resolution, and wherein the selected resolution comprises a resolution less than 4k resolution.

9. The method of claim 1, wherein the hardware display is configured to, for each frame:

decode the transmitted image components corresponding to the frame to produce decoded image components corresponding to the frame;

combine the decoded image components corresponding to the frame to produce the video frame at the selected resolution; and

display the video frame at the selected resolution.

10. A system for processing compressed video data, the system comprising:

a non-transitory computer-readable storage medium storing executable computer instructions that, when executed by a processor, performs steps comprising;

accessing a plurality of encoded video frames, each video frame comprising a plurality of components each associated with a different one of a plurality of image resolutions such that the video frame can be displayed at one of the plurality of image resolutions in part by combining all of the components associated with a resolution equal to or less than the resolution;

identifying a hardware display communicatively coupled to the system;

determining an available bandwidth between the system and the hardware display;

selecting one of the plurality of image resolutions based on the determined available bandwidth;

selecting, for each frame of the plurality of video frames, one or more image components of the frame, the selected image components comprising all image components of the frame associated with a resolution equal to or less than the selected resolution; and

transmitting, for each frame, the selected one or more image components to the hardware display, the hardware display configured to decode, for each frame, the transmitted one or more image components by combining the transmitted one or more image components to produce a video frame at the selected resolution; and

a processor configured to execute the computer instructions.

11. The system of claim 10, wherein the resolution is selected by a user.

12. The system of claim 10, wherein the resolution is selected by the system automatically in response to the determination of the available bandwidth.

13. The system of claim 10, wherein the resolution is selected in response to a determination that the determined available bandwidth is lower than a threshold bandwidth required to support a current resolution, the selected resolution being less than the current resolution.

14. The system of claim 10, wherein the hardware display is located remotely from the system.

15. The system of claim 10, wherein the hardware display is communicatively coupled to the system via the internet.

16. The system of claim 10, wherein the hardware display is communicatively coupled to the system via a wireless network.

17. The system of claim 10, wherein one of the plurality of image resolutions comprises 4k resolution, and wherein the selected resolution comprises a resolution less than 4k resolution.

18. The system of claim 10, wherein the hardware display is configured to, for each frame:

decode the transmitted image components corresponding to the frame to produce decoded image components corresponding to the frame;

combine the decoded image components corresponding to the frame to produce the video frame at the selected resolution; and

display the video frame at the selected resolution.

19. A non-transitory computer-readable storage medium storing executable computer instructions for processing compressed video data that when executed by a processor of a computer system, perform steps comprising:

accessing a plurality of encoded video frames, each video frame comprising a plurality of components each associated with a different one of a plurality of image resolutions such that the video frame can be displayed at one of the plurality of image resolutions in part by combining all of the components associated with a resolution equal to or less than the resolution;

identifying a hardware display communicatively coupled to the computer system;

determining an available bandwidth between the computer system and the hardware display;

selecting one of the plurality of image resolutions based on the determined available bandwidth;

selecting, for each frame of the plurality of video frames, one or more image components of the frame, the selected image components comprising all image components of the frame associated with a resolution equal to or less than the selected resolution; and

transmitting, for each frame, the selected one or more image components to the hardware display, the hardware display configured to decode, for each frame, the transmitted one or more image components by combining the transmitted one or more image components to produce a video frame at the selected resolution.

20. The computer-readable storage medium of claim 19, wherein the hardware display is configured to, for each frame:

decode the transmitted image components corresponding to the frame to produce decoded image components corresponding to the frame;

combine the decoded image components corresponding to the frame to produce the video frame at the selected resolution; and

display the video frame at the selected resolution.