Server apparatus and method for content delivery based on content-aware neural network

• US 10,560,731 B2
• Filed: 03/19/2018
• Issued: 02/11/2020
• Est. Priority Date: 01/10/2018
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Han, Dong Su, Yeo, Hyun Ho, Do, Seong Hyeon
• Primary Examiner(s): Bruckart, Benjamin R
• Assistant Examiner(s): Nguyen, Tien M
• Application Number:
  US15/924,637
• Publication Number:
  US 20190215549A1
• Time in Patent Office: 694 Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82

Abstract

The present disclosure provides a method and a server apparatus for delivering content based on content-aware using a neural network. A server apparatus for content delivery is provided, including a content clustering unit for clustering multiple contents provided from a content provider based on a similarity; a training unit for training a cluster-wise content reconstruction model by using contents contained in each cluster in accordance with a result of clustering performed by the content clustering unit; a storage unit for storing the multiple contents and the cluster-wise content reconstruction model; and a transmission unit for transmitting content requested by a user and a content reconstruction model corresponding to a cluster containing the content requested to a user terminal.

Claims

1. A server apparatus for content delivery, comprising:

a content clustering unit, implemented by one or more processors, configured to cluster a plurality of contents provided from a content provider by using metadata of the plurality of contents based on a similarity;

a training unit, implemented by the one or more processors, configured to train a cluster-wise content reconstruction model by using contents contained in each cluster in accordance with a result of clustering performed by the content clustering unit, and configured to record an association between the cluster-wise content reconstruction model and the contents in each cluster on association information metadata of a manifest file;

a content storage unit, implemented by a hardware chip, configured to store the plurality of contents and a corresponding cluster-wise content reconstruction model; and

a transmission unit, implemented by the one or more processors, configured to transmit the manifest file to a user terminal, a content reconstruction model corresponding to the cluster containing the plurality of contents being contained in metadata of the manifest file received by the user terminal, and in response to a request for a content from the user terminal by utilizing the metadata of the manifest file, transmit, to the user terminal, the content and the content reconstruction model corresponding to the cluster containing the content,

wherein the transmission unit is configured to transmit an original content to the user terminal in accordance with a communication connection status in good condition, and transmit an alternative content to the user terminal in accordance with a communication connection status in congested condition for the user terminal so as to compute to generate a high-quality content by using the alternative content and the content reconstruction model, the alternative content being generated by compressing the original content, and

wherein the content clustering unit is configured to, responsive to a new content from the content provider for determining a similarity of the new content with existing clusters, cluster the new content into a cluster that shares a largest redundancy with the new content, and cluster the new content into a new cluster when the similarity is determined to be the same or less than a predetermined level.

2. The server apparatus for content delivery of claim 1, wherein the content clustering unit is further configured to cluster the plurality of contents based on determining the similarity through the use of a neural network-based image classification model.

3. The server apparatus for content delivery of claim 1, wherein the training unit is configured to train the content reconstruction model so as to provide the original content from the alternative content.

4. The server apparatus for content delivery of claim 3, wherein the training unit is configured to generate a low-resolution image contained in the cluster and train the content reconstruction model to provide an original image from the low-resolution image.

5. The server apparatus for content delivery of claim 3, wherein the training unit is configured to extract a luminance image or a contour image contained in the cluster, and to train the content reconstruction model so as to provide the original content from the luminance image or the contour image.

6. The server apparatus for content delivery of claim 3, wherein the training unit is configured to train the content reconstruction model so as to learn a frame interpolation on the contents contained in the cluster.

7. The server apparatus for content delivery of claim 3, wherein the transmission unit is configured to replace the content requested with the alternative content, and transmit the alternative content to the user terminal in accordance with the communication connection status in congested condition of the user terminal.

8. The server apparatus for content delivery of claim 1, further comprising a metadata storage unit, implemented by a second hardware chip, configured to store the metadata which is related to the plurality of contents.

9. A method of performing content delivery by a content delivery server included in a content delivery system, the method comprising:

clustering a plurality of contents provided from a content provider by using metadata of the plurality of contents based on a similarity;

training a cluster-wise content reconstruction model by using contents contained in each cluster in accordance with a result of the clustering of the plurality of contents;

recording an association between the cluster-wise content reconstruction model and the contents in each cluster on association information metadata of a manifest file;

transmitting the manifest file to a user terminal, a content reconstruction model corresponding to the cluster containing the plurality of contents being contained in metadata of the manifest file received by the user terminal, and in response to a request for a content from the user terminal by utilizing the metadata of the manifest file, transmit, to the user terminal, the content and the content reconstruction model corresponding to the cluster containing the content, and

transmitting an original content to the user terminal in accordance with a communication connection status in good condition, and transmitting an alternative content to the user terminal in accordance with a communication connection status in congested condition for the user terminal so as to compute to generate a high-quality content by using the alternative content and the content reconstruction model, the alternative content being generated by compressing the original content,

wherein the clustering of the plurality of contents comprises;

responsive to a new content from the content provider for determining a similarity of the new content with existing clusters, clustering the new content into a cluster that shares a largest redundancy with the new content, and clustering the new content into a new cluster when the similarity is determined to be the same or less than a predetermined level.

10. The method of claim 9, wherein the clustering of the plurality of contents comprises:

clustering the plurality of contents by determining the similarity based on a neural network-based image classification model.

11. The method of claim 9, wherein the training of the cluster-wise content reconstruction model comprises:

training the content reconstruction model so as to provide the original content from the alternative content.

12. The method of claim 11, wherein the transmitting of the content requested by the user terminal comprises:

replacing the content requested with the alternative content, and transmitting the alternative content to the user terminal in accordance with the communication connection status in congested condition the user terminal.