Technology to encode 360 degree video content

• US 10,587,800 B2
• Filed: 04/10/2017
• Issued: 03/10/2020
• Est. Priority Date: 04/10/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Boyce, Jill M., Baran, Stanley J., Mohan, Sumit, Chiu, Yi-Jen, Tanner, Jason, Mohammed, Atthar H., Hicks, Richmond, Das, Barnan
• Primary Examiner(s): Atala, Jamie J
• Assistant Examiner(s): Cammarata, Michael Robert
• Application Number:
  US15/483,787
• Publication Number:
  US 20180295282A1
• Time in Patent Office: 1,065 Days
• CPC Class Codes: G08B 13/19636 pertaining to the camera
• CPC Class Codes: G08B 29/181 due to failing power supply
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 23/65 Control of camera operation in relation to power supply
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/917 for bandwidth reduction
• CPC Class Codes: H04N 7/0117 involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

Systems, apparatuses and methods may determine, on a per camera basis, an interest level with respect to panoramic video content, identify a subset of cameras in a plurality of cameras for which the interest level is below a threshold, and reduce power consumption in the subset of cameras. Additionally, technology may determine a projection format associated with panoramic video content, identify one or more discontinuous boundaries in the projection format, and modify an encoding scheme associated with the panoramic video content based on the discontinuous boundaries. Moreover, an encoded frame may be assigned to a temporal scalability layer that has a higher priority than a layer to which an asynchronous space warp frame is assigned. Additionally, technology may reduce the encoding complexity of a boundary between an active region and an inactive region in fisheye content.

Claims

1. A system comprising:

a plurality of cameras to capture panoramic video content;

a memory including a set of instructions; and

a processor coupled to the plurality of cameras and the memory, wherein when executed by the processor, the instructions cause the system to;

determine, on a per camera basis, an interest level with respect to the panoramic video content,

identify a subset of cameras in the plurality of cameras for which the interest level is below a threshold,

determine a projection format associated with the panoramic video content,

identify one or more discontinuous boundaries in the projection format,

modify an encoding scheme associated with the panoramic video content based on the one or more discontinuous boundaries, and

reduce power consumption in the subset and reduce a quantization parameter variation across the one or more discontinuous boundaries.

2. The system of claim 1, wherein the instructions, when executed, cause the system to one or more of:

reduce a frame rate of the subset of cameras,

reduce a resolution of the subset of cameras,

reduce an audio sample rate of one or more microphones corresponding to the subset of cameras, or

disable the subset of cameras.

3. The system of claim 2, wherein disablement of the subset of cameras reduces an active field of view of the plurality of cameras from 360 degrees to 180 degrees.

4. The system of claim 1, wherein the instructions, when executed, cause the system to reduce power consumption in the processor by virtue of a bypass of one or more of a transmission, a stitching or an encoding of frames corresponding to the subset of cameras.

5. The system of claim 1, further including:

a directional microphone;

a physiological sensor;

a motion sensor, wherein the interest level is to be determined based on one or more of a signal from the directional microphone, a signal from the physiological sensor, a signal from the motion sensor or a semantic feature extraction from the panoramic video content.

6. The system of claim 1, wherein the projection format is to be a cube map including a plurality of faces and the one or more discontinuous boundaries are to be one or more seams between the plurality of faces.

7. The system of claim 1, wherein the instructions, when executed, cause the system to align encoding block partition boundaries with the one or more discontinuous boundaries.

8. The system of claim 1, wherein the instructions, when executed, cause the system to reduce motion searches across the one or more discontinuous boundaries.

9. The system of claim 1, wherein the instructions, when executed, cause the system to reduce intra prediction across the one or more discontinuous boundaries.

10. The system of claim 1, wherein the instructions, when executed, cause the system to increase a bit allocation to a distortion portion of rate distortion optimization at the one or more discontinuous boundaries.

11. An apparatus comprising:

a substrate; and

logic coupled to the substrate, wherein the logic is implemented in one or more of configurable logic or fixed-functionality hardware logic, the logic to;

determine a projection format associated with panoramic video content,

identify one or more discontinuous boundaries in the projection format, and

modify an encoding scheme associated with the panoramic video content based on the one or more discontinuous boundaries, wherein the logic is to reduce a quantization parameter variation across the one or more discontinuous boundaries.

12. The apparatus of claim 11, wherein the projection format is to be a cube map including a plurality of faces and the one or more discontinuous boundaries are to be one or more seams between the plurality of faces.

13. The apparatus of claim 11, wherein the logic is to align encoding block partition boundaries with the one or more discontinuous boundaries.

14. The apparatus of claim 11, wherein the logic is to reduce motion searches across the one or more discontinuous boundaries.

15. The apparatus of claim 11, wherein the logic is to reduce intra prediction across the one or more discontinuous boundaries.

16. The apparatus of claim 11, wherein the logic is to increase a bit allocation to a distortion portion of rate distortion optimization at the one or more discontinuous boundaries.