Generating synthetic frame features for sentinel frame matching

• US 10,097,865 B2
• Filed: 05/12/2016
• Issued: 10/09/2018
• Est. Priority Date: 05/12/2016
• Status: Active Grant

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Inventors: Li, Renxiang, Emeott, Stephen P., Ishtiaq, Faisal
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Andramuno, Franklin S
• Application Number:
  US15/153,370
• Publication Number:
  US 20170332112A1
• Time in Patent Office: 880 Days
• Field of Search: 725 17, 725 46
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: H04N 21/232 Content retrieval operation locally within server, e.g. reading video streams from disk arrays storage management G06F3/0604; details of querying and searching of video data from a database G06F16/739
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/24 Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

Particular embodiments can refine a seed sentinel frame signature for a seed sentinel frame. The seed sentinel frame may be predictable or partially predictable content that demarks a beginning and/or end of certain content in a video program. The seed sentinel frame may be first used to detect other sentinel frames in the video program. However, other sentinel frames throughout the video program, or in other video programs, may be slightly different from the given sentinel frame due to different reasons. The seed sentinel frame signature may not detect the sentinel frames of a video program with a desired accuracy. Accordingly, particular embodiments may refine the sentinel frame signature to a synthetic sentinel frame signature. The synthetic sentinel frame signature may then be used to analyze the current video program or other video programs. The synthetic sentinel frame signature may more accurately detect the sentinel frames within the video program.

Claims

1. A method comprising:

receiving, by a computing device, a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;

analyzing, by the computing device, the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content, wherein the analyzing the plurality of frames comprises;

generating a plurality of matching measures between the first signature and the plurality of frames, and

applying a first threshold to the plurality of matching measures to select the first set of frames; and

analyzing, by the computing device, the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features;

wherein the first set of frames has a first matching measure in the plurality of matching measures that meets the first threshold, and

wherein a second set of frames has a second matching measure in the plurality of matching measures that does not meet the first threshold, wherein the second set of frames are not used to generate the second signature.

2. A method comprising:

receiving, by a computing device, a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;

analyzing, by the computing device, the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content;

analyzing, by the computing device, the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features, wherein the analyzing the first set of frames to generate the second signature comprises;

comparing a distance between the first signature and the second signature to a deviation threshold, and

when the distance is within the deviation threshold, generating the second signature; and

outputting, by the computing device, the second signature as a replacement for the first signature.

3. The method of claim 2, further comprising:

applying, by the computing device, the second signature to the plurality of frames to generate a second set of frames that are defined as the sentinel frames of the video.

4. The method of claim 3, wherein the second set of frames include different frames from the first set of frames.

5. The method of claim 3, further comprising:

performing a service on the second set of frames, wherein the second set of frames are of the second type of video content.

6. The method of claim 5, wherein the service is not performed on frames in the plurality of frame that are not the second set of frames, wherein the first set of frames are of the first type of video content.

7. The method of claim 2, wherein analyzing the first set of frames comprises:

applying a process to generate a set of clusters from frames in the first set of frames, wherein a cluster includes one or more of the first set of frames; and

generating the second signature based on one of the set of clusters.

8. The method of claim 7, wherein the second signature is based on a position within the one of the set of clusters.

9. The method of claim 8, wherein the position is a center of the one of the set of clusters.

10. The method of claim 7, wherein the second signature is defined based on features from the one of the set of clusters.

11. The method of claim 2, wherein:

the first set of frames is selected based on applying a first threshold to the first signature, and

a second set of frames is selected based on applying a second threshold to the second signature, wherein the second threshold is different from the first threshold.

12. The method of claim 11, wherein the second threshold is less than the first threshold.

13. The method of claim 2, wherein the video comprises a first video, the method further comprising:

applying the second signature to a second plurality of frames of a second video to generate a second set of frames that are considered the sentinel frames of the second video.

14. The method of claim 13, further comprising:

performing a service on the second set of frames, the second set of frames being the second type of video content.

15. A non-transitory computer-readable storage medium containing instructions that, when executed, control a computer system to be configured for:

receiving a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;

analyzing the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content;

analyzing the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features, wherein the analyzing the first set of frames to generate the second signature comprises;

comparing a distance between the first signature and the second signature to a deviation threshold, and

when the distance is within the deviation threshold, generating the second signature; and

outputting the second signature as a replacement for the first signature.

16. The non-transitory computer-readable storage medium of claim 15, wherein the instructions, when executed, further control the computer system to be configured for:

applying, by the computer system, the second signature to the plurality of frames to generate a second set of frames that are defined as the sentinel frames of the video.

17. A method comprising:

receiving, by a computing device, a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;

analyzing, by the computing device, the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content;

analyzing, by the computing device, the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features, wherein the analyzing the first set of frames to generate the second signature comprises;

comparing a distance between the first signature and the second signature to a deviation threshold, and

when the distance is within the deviation threshold, generating the second signature;

outputting, by the computing device, the second signature as a replacement for the first signature; and

when the second signature should be applied to the video, applying, by the computing device, the second signature to the plurality of frames to generate a second set of frames that replace the first set of frames as the sentinel frames of the video.

18. The method of claim 17, wherein the video comprises a first video, the method further comprising:

when the second signature should be applied to a second video, applying the second signature to a second plurality of frames of the second video to generate a third set of frames that are considered the sentinel frames of the second video.