Learning-based automatic commercial content detection

• US 7,565,016 B2
• Filed: 01/15/2007
• Issued: 07/21/2009
• Est. Priority Date: 02/18/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Hua, Xian-Sheng, Zhang, Hong-Jiang, Lu, Lie
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US11/623,304
• Publication Number:
  US 20070112583A1
• Time in Patent Office: 918 Days
• Field of Search: 382/229, 725/22
• US Class Current: 382/229
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

Systems and methods for learning-based automatic commercial content detection are described. In one aspect, the systems and methods include a training component and an analyzing component. The training component trains a commercial content classification model using a kernel support vector machine. The analyzing component analyzes program data such as video and audio data using the commercial content classification model and one or more of single-side left neighborhood(s) and right neighborhood(s) of program data segments. Based on this analysis, each of the program data segments are classified as being commercial or non-commercial segments.

Claims

1. A computer-implemented method for learning-based automatic commercial content detection, the method comprising:

dividing program data into multiple segments;

analyzing the segments to determine visual, audio, and context-based feature sets that differentiate commercial content from non-commercial content;

wherein the context-based features are a function of one or more single-side left and/or right neighborhoods of segments of the multiple segments; and

calculating context-based feature sets from segment-based visual features as an average value of visual features of S^k, S^k representing a set of all segments of the multiple segments that are partially or totally included in the single-side left and/or right neighborhoods such that S^k={C_j^k;

0≦

j<

M^k}={C_i;

C_i∩

N^k≠

Φ

}, M^k being a number of segments in S^k, and

wherein N^k represents 2n+1 neighborhoods, n represents a number of neighborhoods left and/or right of a current segment C_i, S^k is a set of segments that are partially or totally included in N^k, C^k _i represents is a j-th element of S^k, M^k represents a total number of elements in S^k, and Φ

represents an empty set.

2. A computer-implemented method of claim 1, wherein the visual feature set comprises segment-based features and time-based features.

3. A computer-implemented method of claim 1, wherein the single-side left and/or right neighborhoods comprise a current segment having n neighborhoods left of the current segment and/or n neighborhoods right of the current segment.

4. A computer-implemented method of claim 1, wherein the method further comprise:

classifying each of the multiple segments as being commercial or non-commercial content as a function of a trained commercial content classification model, and the visual, audio, and context-based feature sets.

5. A computer-implemented method of claim 1, wherein the method further comprise:

classifying each of the multiple segments as being commercial or non-commercial content as a function of a trained commercial content classification model, and the visual, audio, and context-based feature sets; and

post-processing classified commercial and/or non-commercial segments to generate commercial and/or non-commercial blocks, each block comprising one or more segments of the multiple segments.

6. A tangible computer-readable data storage medium for learning-based automatic commercial content detection, the computer-readable medium comprising computer-program executable instructions executable by a processor for:

dividing program data into multiple segments;

analyzing the segments to determine visual, audio, and context-based feature sets that differentiate commercial content from non-commercial content, wherein the context-based features are a function of one or more single-side left and/or right neighborhoods of segments of the multiple segments; and

calculating context-based feature sets from segment-based visual features as an average value of visual features of S^k, S^k representing a set of all segments of the multiple segments that are partially or totally included in the single-side left and/or right neighborhoods such that S^k={C_j^k;

0≦

j<

M^k}={C_i;

C_i∩

N^k≠

Φ

}, M^k being a number of segments in S^k, and

wherein N^k represents 2n+1 neighborhoods, n represents a number of neighborhoods left and/or right of a current segment C_i, S^k is a set of segments that are partially or totally included in N^k, C^k represents is a j-th element of S^k, M^k represents a total number of elements in S^k, and Φ

represents an empty set.

7. The tangible computer-readable data storage medium of claim 6, wherein the visual feature set comprises segment-based features and time-based features.

8. The tangible computer-readable data storage medium of claim 6, wherein the single-side left and/or right neighborhoods comprise a current segment, the current segment having n neighborhoods left of the current segment and/or n neighborhoods right of the current segment.

9. The tangible computer-readable data storage medium of claim 6, wherein the computer-program instructions further comprise instructions for classifying each of the multiple segments as being commercial or non-commercial content as a function of a trained commercial content classification model, and the visual, audio, and context-based feature sets.

10. The tangible computer-readable data storage medium of claim 6, wherein the computer-program instructions further comprise instructions for:

classifying each of the multiple segments as being commercial or non-commercial content as a function of a trained commercial content classification model, and the visual, audio, and context-based feature sets; and

post-processing classified commercial and/or non-commercial segments to generate commercial and/or non-commercial blocks, each block comprising one or more segments of the multiple segments.