APPARATUS AND METHODS FOR VIDEO COMPRESSION USING MULTI-RESOLUTION SCALABLE CODING

• US 20180054624A1
• Filed: 10/31/2017
• Published: 02/22/2018
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Abbas, Adeel, Adsumilli, Balineedu Chowdary, Newman, David
• Granted Patent: US 10,212,438 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

Apparatus and methods for digital video data compression via a scalable, multi-resolution approach. In one embodiment, the video content may be encoded using a multi-resolution and/or multi-quality scalable coding approach that reduces computational and/or energy load on a client device. In one implementation, a low fidelity image is obtained based on a first full resolution image. The low fidelity image may be encoded to obtain a low fidelity bitstream. A second full resolution image may be obtained based on the low fidelity bitstream. A portion of a difference image obtained based on the second full resolution image and the first full resolution may be encoded to obtain a high fidelity bitstream. The low fidelity bitstream and the high fidelity bitstream may be provided to e.g., a receiving device.

Claims

1-9. -9. (canceled)

10. A method for providing video content, the method comprising:

obtaining a first panoramic image by decoding a first bitstream using a first decoder;

obtaining a view frame image by at least decoding a second bitstream using a second decoder;

obtaining a second panoramic image at least in part by up-sampling the first panoramic image to the resolution of the view frame image;

obtaining an output view frame image at least in part by combining the view frame image and the second panoramic image; and

display the obtained output view frame image via a rendering device.

11. The method of claim 10, wherein the first panoramic image comprises a low fidelity image characterized by at least a first resolution, and the second panoramic image comprises a high fidelity image characterized by at least a second resolution that is greater than the first resolution.

12. The method of claim 11, wherein a ratio of the first resolution to the second resolution is characterized by a resolution scalability parameter.

13. The method of claim 12, wherein:

the resolution scalability parameter is based at least on a display capability of a display device; and

the method further comprises displaying the output view frame image via the display device.

14. The method of claim 10, wherein the resolution scalability parameter is based at least on a processing or memory limitation of a display device.

15. The method of claim 10, further comprising:

obtaining an intermediate image by at least decoding an intermediate bitstream using at least a third decoder; and

wherein the act of up-sampling the first panoramic image to the resolution of the view frame image further comprises combining the intermediate image with the first panoramic image.

16-18. -18. (canceled)

19. A method for providing video content, the method comprising:

for a first viewport display, obtaining a first output image by combining a panoramic image and a first difference image for first coordinates of the first viewport display;

determining when a change occurs in the first viewport display;

responsive to the change, requesting a second difference image; and

obtaining a second output image for the first viewport display by combining the panoramic image and the second difference image.

20. The method of claim 19, wherein at least one of a resolution quality, a size, and/or a bitrate associated with the second difference image is a scaled according to a Quality of Service (QoS) parameter.

21. A computerized apparatus for providing video content, the apparatus comprising:

an electronic storage apparatus configured to store a sequence of images;

a communications interface configured to enable communication of bitstreams to a client device; and

one or more processors configured to execute a plurality of computer readable instructions; and

a storage apparatus in data communication with the one or more processors and comprising at least one computer program, the at least one computer program comprising a plurality of instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to;

obtain a first panoramic image by a decode of a first bitstream using a first decoder;

obtain a view frame image by at least a decode of a second bitstream using a second decoder;

obtain a second panoramic image at least in part by up-sampling the first panoramic image to the resolution of the view frame image;

obtain an output view frame image at least in part by a combination of the view frame image and the second panoramic image; and

cause display of the obtained output view frame image via a rendering device.

22. The computerized apparatus of claim 21, wherein the first panoramic image comprises a low fidelity image characterized by at least a first resolution, and the second panoramic image comprises a high fidelity image characterized by at least a second resolution that is greater than the first resolution.

23. The computerized apparatus of claim 22, wherein a ratio of the first resolution to the second resolution is characterized by a resolution scalability parameter.

24. The computerized apparatus of claim 23, wherein:

the resolution scalability parameter is based at least on a display capability of a display device; and

the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to;

display the output view frame image via the display device.

25. The computerized apparatus of claim 23, wherein the resolution scalability parameter is based at least on a processing or memory limitation of a display device.

26. The computerized apparatus of claim 21, wherein the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:

obtain an intermediate image by at least a decode of an intermediate bitstream using at least a third decoder; and

wherein up-sampling of the first panoramic image to the resolution of the view frame image further comprises a combination of the intermediate image with the first panoramic image.

27. A non-transitory computer readable apparatus comprising a storage medium in data communication with one or more processors and comprising at least one computer program, the at least one computer program comprising a plurality of instructions which, when executed by the one or more processors, cause a computerized apparatus to:

obtain a first panoramic image by a decode of a first bitstream using a first decoder;

obtain a view frame image by at least a decode of a second bitstream using a second decoder;

obtain a second panoramic image at least in part by up-sampling the first panoramic image to the resolution of the view frame image;

obtain an output view frame image at least in part by a combination of the view frame image and the second panoramic image; and

cause display of the obtained output view frame image via a rendering device.

28. The non-transitory computer readable apparatus of claim 27, wherein the first panoramic image comprises a low fidelity image characterized by at least a first resolution, and the second panoramic image comprises a high fidelity image characterized by at least a second resolution that is greater than the first resolution.

29. The non-transitory computer readable apparatus of claim 28, wherein a ratio of the first resolution to the second resolution is characterized by a resolution scalability parameter.

30. The non-transitory computer readable apparatus of claim 29, wherein:

the resolution scalability parameter is based at least on a display capability of a display device; and

the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to;

display the output view frame image via the display device.

31. The non-transitory computer readable apparatus of claim 29, wherein the resolution scalability parameter is based at least on a processing or memory limitation of a display device.

32. The non-transitory computer readable apparatus of claim 27, wherein the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:

obtain an intermediate image by at least a decode of an intermediate bitstream using at least a third decoder; and

wherein up-sampling of the first panoramic image to the resolution of the view frame image further comprises a combination of the intermediate image with the first panoramic image.