Methods and apparatus for filtering video packets for large-scale video stream monitoring

• US 7,912,284 B2
• Filed: 05/13/2008
• Issued: 03/22/2011
• Est. Priority Date: 07/01/2005
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lin, Ching-Yung, Verscheure, Olivier, Amini, Lisa Dronette
• Primary Examiner(s): Bali; Vikkram
• Assistant Examiner(s): Zeilberger; Daniel
• Application Number:
  US12/119,804
• Publication Number:
  US 20090080777A1
• Time in Patent Office: 1,043 Days
• Field of Search: 382/170, 382/190, 382/195, 382/206, 348700-702, 725/52, 725/53
• US Class Current: 382/170
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

A method of filtering video packets for video stream monitoring is provided. A video packet of a video stream is received. One or more features are extracted from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame. One or more concept detectors are implemented on the one or more features creating one or more confidence values. The confidence values are transmitted to a display module for filtering of video packets.

Claims

1. A method of filtering video packets for video stream monitoring comprising the steps of:

receiving a video packet of a video stream;

extracting one or more features from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame;

implementing one or more concept detectors on the one or more features creating one or more confidence values; and

transmitting the confidence values to a display module for filtering of video packets;

wherein the extracting step comprises the steps of;

parsing the video packet to attain the specified frame;

calculating a histogram for each of one or more slices of the specified frame using the frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms;

wherein the step of calculating a histogram comprises the steps of;

determining frequency domain coefficients of the specified frame;

partitioning the specified frame into one or more slices;

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame; and

truncating selected frequency domain coefficients in each of the one or more slices of the specified frame.

2. The method of claim 1, wherein, in the step of extracting one or more features, the one or more features comprise compressed-domain slice features.

3. The method of claim 1, wherein the step of implementing one or more concept detectors comprises the step of matching user interests with a confidence output of semantic model vectors.

4. The method of claim 1, wherein the step of transmitting the confidence values comprises the steps of:

storing the confidence values as metadata; and

transmitting the metadata to the display module for filtering of video packets.

5. The method of claim 1, wherein the step of determining frequency domain coefficients comprises the step of mapping a data compression coding scheme to frequency domain coefficients of the specified frame.

6. The method of claim 5, wherein, in the step of mapping a data compression coding scheme, the data compression coding scheme comprises variable length codes.

7. The method of claim 5, wherein, in the step of mapping a data compression coding scheme, the frequency domain coefficients comprise discrete cosine transform coefficients.

8. The method of claim 1, wherein the step of calculating a histogram comprises the steps of using a DC coefficient and one or more AC coefficients of the discrete cosine transformation coefficients on each color plane.

9. The method of claim 8, wherein, in the step of using a DC coefficient and one or more AC coefficients, the one or more AC coefficients comprise two lowest frequency AC coefficients.

10. Apparatus for filtering video packets for video stream monitoring, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

receive a video packet of a video stream;

extract one or more features from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame;

implement one or more concept detectors on the one or more features creating one or more confidence values; and

transmit the confidence values to a display module for filtering of video packets;

wherein the extracting operation comprises the operations of;

parsing the video packet to attain the specified frame;

calculating a histogram for each of one or more slices of the specified frame using the frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms; and

wherein the operation of calculating a histogram comprises the operations of;

determining frequency domain coefficients of the specified frame;

partitioning the specified frame into one or more slices; and

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame; and

truncating selected frequency domain coefficients in each of the one or more slices of the specified frame.

11. The apparatus of claim 10, wherein, in the operation of extracting one or more features, the one or more features comprise compressed-domain slice features.

12. The apparatus of claim 10, wherein the operation of implementing one or more concept detectors comprises the operation of matching user interests with a confidence output of semantic model vectors.

13. The apparatus of claim 10, wherein the operation of transmitting the confidence values comprises the operations of:

storing the confidence values as metadata; and

transmitting the metadata to the display module for filtering of video packets.

14. The apparatus of claim 10, wherein the operation of determining frequency domain coefficients comprises the step of mapping a data compression coding scheme to frequency domain coefficients of the specified frame.

15. The apparatus of claim 14, wherein, in the operation of mapping a data compression coding scheme, the data compression coding scheme comprises variable length codes.

16. The apparatus of claim 14, wherein, in the operation of mapping a data compression coding scheme, the frequency domain coefficients comprise discrete cosine transform coefficients.

17. The apparatus of claim 10, wherein the operation of calculating a histogram comprises the steps of using a DC coefficient and one or more AC coefficients of the discrete cosine transformation coefficients on each color plane.

18. The apparatus of claim 17, wherein, in the operation of using a DC coefficient and one or more AC coefficients, the one or more AC coefficients comprise two lowest frequency AC coefficients.

19. An article of manufacture for filtering video packets for video stream monitoring, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

receiving a video packet of a video stream;

extracting one or more features from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame;

implementing one or more concept detectors on the one or more features creating one or more confidence values; and

transmitting the confidence values to a display module for filtering of video packets;

wherein the extracting step comprises;

parsing the video packet to attain the specified frame;

calculating a histogram for each of one or more slices of the specified frame using the frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms; and

wherein the step of calculating a histogram comprises the steps of;

determining frequency domain coefficients of the specified frame;

partitioning the specified frame into one or more slices;

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame; and

truncating selected frequency domain coefficients in each of the one or more slices of the specified frame.