Detection of solid color frames for determining transitions in video content

• US 9,973,662 B2
• Filed: 01/13/2015
• Issued: 05/15/2018
• Est. Priority Date: 01/13/2015
• Status: Active Grant

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Inventors: Li, Renxiang, Ishtiaq, Faisal, Smith, Alfonso Martinez
• Primary Examiner(s): Pontius, James
• Application Number:
  US14/595,660
• Publication Number:
  US 20160205289A1
• Time in Patent Office: 1,218 Days
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

Particular embodiments detect a solid color frame, such as a black frame, that may include visible content other than the solid color in a portion of the frame. These frames may conventionally not be detected as a solid color frame because of the visible content in the portion of the frame. However, these solid color frames may be “functional” black or white frames, in that the solid color frames are performing the function of the solid color frame even though the frames include the visible content. The visible content may be content that may always be displayed on the screen even if the video content is transitioning to an advertisement. Particular embodiments use techniques to detect the functional solid color frames even when visible content appears in the solid color frames. Particular embodiments use color layout information and edge distribution information to detect solid color frames.

Claims

1. A method for determining transitions in video content, the method comprising:

determining color layout information for a frame of the video content, the color layout information describing a distribution of color in the frame;

determining edge distribution information for the frame, the edge distribution information describing a number of edges found in a plurality of regions of the frame;

determining a set of color layout features from the color layout information regarding a color change within the frame;

determining a set of edge distribution features from the edge distribution information in a set of regions in the plurality of regions, wherein the set of edge distribution features indicate a number of edges in the set of regions;

determining whether the frame is classifiable as a solid color frame based on the set of color layout features and the set of edge distribution features; and

when the frame is classifiable as the solid color frame, marking the solid color frame being a transition between a first type of video content and a second type of video content;

wherein determining whether the frame is classifiable as a solid color frame comprises;

determining when a first color layout feature in the set of color layout features indicates a luminance of the frame is below a first color layout threshold;

determining when the number of edges in the set of edge distribution features for the plurality of regions is below a first edge threshold;

determining when a second color layout feature in the set of color layout features indicates a change in luminance of the frame is above a second color layout threshold;

determining when the number of edges in the set of edge distribution features for the set of regions is above a second edge threshold; and

determining when the number of edges for regions not in the set of regions includes a number of bins including zero edges that is above a third edge threshold.

2. The method of claim 1, wherein the frame is classified as the solid color frame when visible content other than a color of the solid color frame exists in the set of regions in the frame.

3. The method of claim 2, wherein the visible content is not in regions other than the set of regions.

4. The method of claim 1, wherein the frame is not classified as the solid color frame when visible content other than a color of the solid color frame exists in regions other than the set of regions in the frame.

5. The method of claim 1, wherein:

the first color layout feature is a first coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame.

6. The method of claim 1, wherein:

the number of edges comprises a number that indicates no edges are found in portions of the plurality of regions.

7. The method of claim 1, wherein:

the second color layout feature is a second coefficient or third coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame.

8. The method of claim 1, wherein:

the second color layout feature is a first coefficient or fifth coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame.

9. The method of claim 1, wherein determining when the frame is classified as the solid color frame comprises:

performing a first test to determine when a first color layout feature in the set of color layout features and the set of edge distribution features indicate the frame includes a solid color in the plurality of regions; and

performing a second test to determine when a second color layout feature in the set of color layout features and the set of edge distribution features indicate visible content other than the solid color is found in the set of regions and not in regions other than the set of regions.

10. The method of claim 1, wherein the solid color frame comprises a black color frame or a white color frame.

11. A method for determining transitions in video content, the method comprising:

determining color layout information for a frame of the video content, the color layout information describing a distribution of color in the frame;

determining edge distribution information for the frame, the edge distribution information describing a number of edges found in a plurality of regions of the frame;

determining a set of color layout features from the color layout information regarding a color change within the frame;

determining a set of edge distribution features from the edge distribution information in a set of regions in the plurality of regions, wherein the set of edge distribution features indicate a number of edges in the set of regions;

determining whether the frame is classifiable as a solid color frame based on the set of color layout features and the set of edge distribution features; and

when the frame is classifiable as the solid color frame, marking the solid color frame being a transition between a first type of video content and a second type of video content;

wherein determining whether the frame is classifiable as a solid color frame comprises;

determining when a first color layout feature in the set of color layout features indicates a luminance of the frame is below a first color layout threshold;

determining when the number of edges in the set of edge distribution features for the plurality of regions is below a first edge threshold;

determining when a second color layout feature in the set of color layout features indicates a change in luminance of the frame is above a second color layout threshold;

determining when the number of edges in the set of edge distribution features for the set of regions is above a second edge threshold; and

determining when a third color layout feature in the set of color layout features indicates the change in luminance of the frame is above a third color layout threshold.

12. The method of claim 11, wherein the frame is classified as the solid color frame when visible content other than a color of the solid color frame exists in the set of regions in the frame.

13. The method of claim 11, wherein the frame is classified as the solid color frame when visible content other than a color of the solid color frame exists in the set of regions in the frame.

14. The method of claim 11, wherein the frame is not classified as the solid color frame when visible content other than a color of the solid color frame exists in regions other than the set of regions in the frame.

15. The method of claim 11, wherein:

the first color layout feature is a first coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame.

16. The method of claim 11, wherein:

the number of edges comprises a number that indicates no edges are found in portions of the plurality of regions.

17. The method of claim 11, wherein:

the second color layout feature is a second coefficient or third coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame.

18. The method of claim 11, wherein:

the second color layout feature is a first coefficient or fifth coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame.

19. The method of claim 11, wherein determining when the frame is classified as the solid color frame comprises:

performing a first test to determine when a first color layout feature in the set of color layout features and the set of edge distribution features indicate the frame includes a solid color in the plurality of regions; and

performing a second test to determine when a second color layout feature in the set of color layout features and the set of edge distribution features indicate visible content other than the solid color is found in the set of regions and not in regions other than the set of regions.

20. The method of claim 11, wherein the solid color frame comprises a black color frame or a white color frame.

21. A system for determining transitions in video content, the system comprising:

a solid color frame detector comprising one or more computer processors to be communicatively coupled to a storage device storing the video content, the one or more computer processors further configured for;

receiving the video content from the storage device;

determining color layout information for a frame of the video content, the color layout information describing a distribution of color in the frame;

determining edge distribution information for the frame, the edge distribution information describing a number of edges found in a plurality of regions of the frame;

determining a set of color layout features from the color layout information regarding a color change within the frame;

determining a set of edge distribution features from the edge distribution information in a set of regions in the plurality of regions, wherein the set of edge distribution features indicate a number of edges in the set of regions;

determining whether the frame is classifiable as a solid color frame based on the set of color layout features and the set of edge distribution features; and

when the frame is classifiable as the solid color frame, marking the solid color frame as a transition between a first type of video content and a second type of video content;

wherein determining whether the frame is classifiable as a solid color frame comprises;

determining when a first color layout feature in the set of color layout features indicates a luminance of the frame is below a first color layout threshold;

determining when the number of edges in the set of edge distribution features for the plurality of regions is below a first edge threshold;

determining when a second color layout feature in the set of color layout features indicates a change in luminance of the frame is above a second color layout threshold;

determining when the number of edges in the set of edge distribution features for the set of regions is above a second edge threshold; and

determining when the number of edges for regions not in the set of regions includes a number of bins including zero edges that is above a third edge threshold;

a video service manager comprising one or more computer processors configured for receiving the video content comprising the solid color frame, and performing a service on the video content based on the transition between a first type of video content and a second type of video content to produce serviced video content, the one or more computer processors of the video service manager further configured to provide the serviced video content to a video server configured for sending the serviced video content to a client device.

22. The system of claim 21, wherein the frame is classified as the solid color frame when visible content other than a color of the solid color frame exists in the set of regions in the frame.

23. A method for determining transitions in video content, the method comprising:

determining a set of color layout features from color layout information regarding a color change within a frame of the video content;

determining a set of edge distribution features from edge distribution information in a set of regions of a plurality of regions in the frame, wherein the set of edge distribution features indicate a number of edges in the set of regions;

performing a first test to determine when a first color layout feature in the set of color layout features and the set of edge distribution features indicate the frame includes a solid color in the plurality of regions;

performing a second test to determine when a second color layout feature in the set of color layout features and at least a portion of the set of edge distribution features indicate visible content other than the solid color is found in the set of regions and not in regions other than the set of regions;

determining whether the frame is classifiable as a solid color frame based at least on the first test or the second test, and wherein determining whether the frame is classifiable as a solid color frame further comprises;

determining when a first color layout feature in the set of color layout features indicates a luminance of the frame is below a first color layout threshold;

determining when the number of edges in the set of edge distribution features for the plurality of regions is below a first edge threshold;

determining when a second color layout feature in the set of color layout features indicates a change in luminance of the frame is above a second color layout threshold;

determining when the number of edges in the set of edge distribution features for the set of regions is above a second edge threshold; and

determining when a third color layout feature in the set of color layout features indicates the change in luminance of the frame is above a third color layout threshold; and

when the frame is classified as the solid color frame, marking the solid color frame as a transition between a first type of content and a second type of content.

24. The method of claim 23, wherein when either the first test or the second test indicates the frame is classified as the solid color frame, outputting an indication that the frame is the solid color frame.

25. The method of claim 23, wherein:

the first color layout feature is a zero coefficient in a scan order of discrete cosine transform (DCT) coefficients for the frame, and

the second color layout feature is a second coefficient or third coefficient in the scan order of discrete cosine transform (DCT) coefficients for the frame or the second color layout feature is a first coefficient or fifth coefficient in the scan order of discrete cosine transform (DCT) coefficients for the frame.

26. The method of claim 23, wherein the at least a portion of the set of edge distribution features comprises a row or column of edge distribution features.