×

Computerized system and method for automatically detecting and rendering highlights from streaming videos

  • US 10,390,082 B2
  • Filed: 04/01/2016
  • Issued: 08/20/2019
  • Est. Priority Date: 04/01/2016
  • Status: Active Grant
First Claim
Patent Images

1. A method comprising steps of:

  • receiving, at a computing device, a video stream comprising a plurality of frames of content, said video stream comprising a broadcast of a currently occurring live event over a network;

    analyzing via the computing device via first image recognition software, as the stream is being received, a set of frames from the plurality of frames of the video stream, said analysis comprising performing, via the first image recognition software, transformations of the frame content within the frame set resulting in an identification of attributes of the frame content;

    training, by the computing device, a deep learning algorithm, the training comprising;

    determining and storing labels for frames of a first set of training videos based on attributes of the frames, a label comprising an indication as to a scene type depicted by content of the frames of the training videos and the label associated with said predetermined set of machine learned attributes,automatically applying the labels via the first image recognition software to a second set of training videos,analyzing said automatically applied labels and adjusting the automatically applied labels identified as being inaccurate, andupdating indications associated with the labels based on the analysis;

    classifying, via the computing device, a scene type depicted in the frame set by determining whether the scene type is a game scene or whether the scene type is a non-game scene based on said frame content transformations, the classifying comprising applying the deep learning algorithm to frames in the frame set, the deep learning algorithm comprising multiple layers, each layer producing a feature collection for a given frame, the deep learning algorithm comprising overlapping, for each frame, the feature collections associated with each layer, pooling the overlapping feature collections, and assigning a label to the frame set based on the pooled feature collections;

    determining, via the computing device, that said scene type is a game scene based on the assigned label, said game scene comprising content associated with game play occurring in said live event;

    discarding, by the computing device, the frame set upon determining that the scene type is a non-game scene;

    upon determining that said scene type is a game scene, determining, via the computing device, that said content within said game scene is a highlight and designating said game scene as a highlight based on said determination, said determination comprising computing a highlight score for said game scene by analyzing frames of the game scene via the second image recognition software and determining that an output from said second image recognition software satisfies a threshold, said output based on a comparison, performed by the second image recognition software, of the attributes of the frame content in the game scene against a predetermined set of machine learned highlight attributes;

    generating, via the computing device, an output file corresponding to the video stream, said output file comprising time-stamped information associated with the scene label and the highlight label; and

    automatically creating, via the computing device, a highlight video segment from the video stream based on said output file, said highlight video segment created from and comprising frames of the video stream identified in the output file as the game scene and highlight.

View all claims
  • 6 Assignments
Timeline View
Assignment View
    ×
    ×