METHOD AND APPARATUS FOR ALLOCATING DIFFERENTIAL BANDWIDTH FOR EACH SCREEN REGION BY USING IMAGE COMPLEXITY INFORMATION

• US 20190320007A1
• Filed: 09/27/2017
• Published: 10/17/2019
• Est. Priority Date: 11/24/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: KIM, Kil-jong, HONG, Soon-gi, JOUNG, Do-young, LEE, Chang-hyun
• Granted Patent: US 11,017,517 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

Provided are a method and apparatus for allocating differential bandwidth for each screen region by using image complexity information. According to an embodiment, a method and apparatus for receiving streaming data request a server for information relating to content to differentially determine quality of a plurality of regions divided from a predetermined frame of the content according to a network state; among at least one segment for each region encoded with at least one quality for each of the plurality of regions, receive a segment for each region encoded according to the quality for each region determined based on the information relating to the content from the server over a network; generate a reconstruction image for each region by decoding the segment for each region; and merge and render the reconstruction image for each region.

Claims

1. A method of receiving streaming data, the method comprising:

requesting a server for information relating to content to differentially determine quality of a plurality of regions divided from a predetermined frame of the content, according to a network state;

among at least one segment for each region encoded with at least one quality for each of the plurality of regions, receiving a segment for each region encoded according to the quality for each region determined based on the information relating to the content from the server over a network;

generating a reconstruction image for each region by decoding the segment for each region; and

merging and rendering the reconstruction image for each region.

2. The method of claim 1,

wherein the information relating to the content comprises additional information comprising importance information with respect to the plurality of regions and segment information with respect to the at least one segment for each region,

wherein the importance information with respect to the plurality of regions comprises at least one of complexity information with respect to the plurality of regions, object state information, and information about whether viewing is possible;

and

wherein the segment information comprises at least one of quality information of the at least one segment for each region and location information of the at least one segment for each region within the predetermined frame.

3. The method of claim 2, further comprising:

inspecting a state of the network;

generating a quality configuration candidate group for each of the plurality of regions, based on the state of the network; and

selecting a quality for each region from the quality configuration candidate group for each region, based on the importance information with respect to the plurality of regions.

4. The method of claim 2, wherein the additional information comprises information indicating a quality of each network state with respect to each of the plurality of regions,

the method further comprises:

inspecting a state of the network; and

determining a quality for each region, based on information indicating the state of the network and a quality for each network state.

5. The method of claim 3, wherein the quality configuration candidate group for each region is generated based on the state of the network and extracted region of interest information.

6. The method of claim 4, wherein the quality for each region is determined based on the state of the network, information indicating the quality for each network state, and extracted region of interest information.

7. The method of claim 1, further comprising:

transmitting extracted region of interest information to the server,

wherein the quality for each region is determined based on the region of interest information.

8. The method of claim 2, further comprising:

inspecting a state of the network;

determining whether to determine the quality for each region based on a region of interest; and

when the quality for each region is determined based on region of interest information;

extracting the region of interest information; and

determining quality information for each region based on the region of interest, or

when the quality for each region is not determined based on the region of interest;

generating a quality configuration candidate group for each of the plurality of regions based on the state of the network; and

selecting the quality for each region among the quality configuration candidate group for each region based on the importance information with respect to the plurality of regions.

9. The method of claim 2, further comprising:

inspecting a state of the network;

determining whether to determine the quality for each region based on a region of interest; and

when the quality for each region is determined based on region of interest information;

extracting the region of interest information; and

determining quality information for each region based on the region of interest, or

when the quality for each region is not determined based on the region of interest;

determining the quality information for each region based on quality information for each network state with respect to each of the plurality of regions included in the additional information.

10. A method of transmitting streaming data, the method comprising:

dividing a predetermined frame of content into a plurality of regions to differentially determine quality according to a network state;

generating at least one segment for each region by performing encoding on each of the plurality of regions with at least one quality; and

transmitting to a client over a network a segment for each of the encoded plurality of regions according to a quality for each region among the at least one segment for each region.

11. The method of claim 10, further comprising:

generating additional information comprising importance information with respect to the plurality of regions and segment information with respect to the at least one segment for each region; and

transmitting the additional information to the client over the network,

wherein the importance information with respect to the plurality of regions comprises at least one of complexity information with respect to the plurality of regions, object state information, and information about whether viewing is possible; and

wherein the segment information comprises at least one of quality information of the at least one segment for each region and location information of the at least one segment for each region within the predetermined frame.

12. The method of claim 11, wherein the generating of the additional information comprises:

determining a quality for each network state with respect to each of the plurality of regions, based on the importance information with respect to the plurality of regions; and

generating the additional information comprising the segment information and information indicating the quality for each network state.

13. The method of claim 10, further comprising:

inspecting a state of the network;

generating a quality configuration candidate group for each of the plurality of regions, based on the state of the network; and

selecting a quality for each region from the quality configuration candidate group for each region, based on the importance information with respect to the plurality of regions.

14. The method of claim 10, further comprising:

receiving region of interest information from the client;

inspecting a state of the network;

generating a quality configuration candidate group for each of the plurality of regions, based on the region of interest information and the state of the network; and

selecting a quality for each region from the quality configuration candidate group for each region, based on the importance information with respect to the plurality of regions.

15. An apparatus for receiving streaming data, the apparatus comprising:

a data transceiver configured to request a server for information relating to content to differentially determine quality of a plurality of regions divided from a predetermined frame of the content according to a network state, and, among at least one segment for each region encoded with at least one quality for each of the plurality of regions, receive a segment for each region encoded according to the quality for each region determined based on the information relating to the content from the server over a network;

a decoder configured to generate a reconstruction image for each region by decoding the segment for each region; and

a renderer configured to merge and render the reconstruction image for each region.