×

Summarizing video content based on memorability of the video content

  • US 10,311,913 B1
  • Filed: 02/22/2018
  • Issued: 06/04/2019
  • Est. Priority Date: 02/22/2018
  • Status: Active Grant
First Claim
Patent Images

1. A method for summarizing video content based on memorability of the video content, the method performed by one or more processing devices and comprising:

  • accessing segments of an input video;

    computing memorability scores for the segments, respectively, wherein computing a memorability score for a segment comprises;

    generating (i) a semantic feature computed from an auto-captioning operation applied to the segment and (ii) a visual feature computed from one or more of a saliency analysis operation applied to the segment, a color analysis operation applied to the segment, and a spatio-temporal analysis operation applied to the segment,computing a first component score by applying a first predictor to the semantic feature, where the first predictor is trained to determine first component memorability scores by comparing user-generated memorability values with training semantic features generated by the auto-captioning operation,computing a second component score by applying a second predictor to the semantic feature, where the second predictor is trained to determine second component memorability scores by comparing the user-generated memorability values with training visual features generated by the one or more of the saliency analysis operation, the color analysis operation, and the spatio-temporal analysis operation, andcomputing the memorability score from an averaging operation applied to the first component score and the second component score;

    selecting a subset of segments from the segments based on each computed memorability score in the subset having a threshold memorability score; and

    generating visual summary content from the subset of the segments.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×