Methods and apparatus for filtering video packets for large-scale video stream monitoring

• US 20070286489A1
• Filed: 07/01/2005
• Published: 12/13/2007
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lin, Ching-Yung, Verscheure, Olivier, Amini, Lisa
• Granted Patent: US 7,545,978 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

A method of filtering video packets for video stream monitoring is provided. A video packet of a video stream is received. One or more features are extracted from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame. One or more concept detectors are implemented on the one or more features creating one or more confidence values. The confidence values are transmitted to a display module for filtering of video packets.

Claims

1. A method of filtering video packets for video stream monitoring comprising the steps of:

receiving a video packet of a video stream;

extracting one or more features from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame;

implementing one or more concept detectors on the one or more features creating one or more confidence values; and

transmitting the confidence values to a display module for filtering of video packets.

2. The method of claim 1, wherein, in the step of implementing one or more concept detectors, the one or more concept detectors are implemented on two or more hardware devices.

3. The method of claim 1, wherein, in the step of transmitting the confidence values, the filtering of video packets comprises indexing video packets for video database retrieval.

4. The method of claim 1, wherein, in the step of extracting one or more features, the one or more features comprise compressed-domain slice features.

5. The method of claim 1, wherein the step of implementing one or more concept detectors comprises the step of matching user interests with a confidence output of semantic model vectors.

6. The method of claim 1, wherein the step of transmitting the confidence values comprises the steps of:

storing the confidence values as metadata; and

transmitting the metadata to the display module for filtering of video packets.

7. The method of claim 1, wherein the step of extracting one or more features comprises the steps of:

parsing the video packet to attain a specified frame;

calculating a histogram for each of one or more slices of the specified frame using frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms.

8. The method of claim 7, wherein the step of calculating a histogram comprises the steps of:

determining frequency domain coefficients of the specified frame;

partitioning the specified frame into one or more slices; and

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame.

9. The method of claim 8, further comprising the step of truncating selected frequency domain coefficients in each of the one or more slices of the specified frame.

10. The method of claim 8, wherein the step of determining frequency domain coefficients comprises the step of mapping a data compression coding scheme to frequency domain coefficients of the specified frame.

11. The method of claim 10, wherein, in the step of mapping a data compression coding scheme, the data compression coding scheme comprises variable length codes.

12. The method of claim 10, wherein, in the step of mapping a data compression coding scheme, the frequency domain coefficients comprise discrete cosine transform coefficients.

13. The method of claim 8, wherein the step of calculating a histogram comprises the steps of using a DC coefficient and one or more AC coefficients of the discrete cosine transformation coefficients on each color plane.

14. The method of claim 13, wherein, in the step of using a DC coefficient and one or more AC coefficients, the one or more AC coefficients comprise two lowest frequency AC coefficients.

15. A method of extracting features from video packets for filtering purposes comprising the steps of:

parsing a video packet to attain a specified frame;

calculating a histogram for each of one or more slices of the specified frame using frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms.

16. A method of calculating one or more histograms for feature extraction from a video packet, comprising the steps of:

determining frequency domain coefficients of a specified frame of the video packet;

partitioning the specified frame into one or more slices; and

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame.

17. Apparatus for filtering video packets for video stream monitoring, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

receive a video packet of a video stream;

extract one or more features from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame;

implement one or more concept detectors on the one or more features creating one or more confidence values; and

transmit the confidence values to a display module for filtering of video packets.

18. The apparatus of claim 17, wherein the operation of implementing one or more concept detectors comprises the operation of matching user interests with a confidence output of semantic model vectors.

19. The apparatus of claim 17, wherein the operation of transmitting the confidence values comprises the operations of:

storing the confidence values as metadata; and

transmitting the metadata to the display module for filtering of video packets.

20. The apparatus of claim 17, wherein the operation of extracting one or more features comprises the operations of:

parsing the video packet to attain a specified frame;

calculating a histogram for each of one or more slices of the specified frame using frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms.

21. The apparatus of claim 20, wherein the operation of calculating a histogram comprises the operations of:

determining frequency domain coefficients of the specified frame;

partitioning the specified frame into one or more slices; and

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame.

22. The apparatus of claim 21, further comprising the operation of truncating selected frequency domain coefficients in each of the one or more slices of the specified frame.

23. The apparatus of claim 21, wherein the operation of determining frequency domain coefficients comprises the step of mapping a data compression coding scheme to frequency domain coefficients of the specified frame.

24. The apparatus of claim 23, wherein, in the operation of mapping a data compression coding scheme, the data compression coding scheme comprises variable length codes.

25. The apparatus of claim 23, wherein, in the operation of mapping a data compression coding scheme, the frequency domain coefficients comprise discrete cosine transform coefficients.

26. The apparatus of claim 21, wherein the operation of calculating a histogram comprises the steps of using a DC coefficient and one or more AC coefficients of the discrete cosine transformation coefficients on each color plane.

27. An article of manufacture for filtering video packets for video stream monitoring, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

receiving a video packet of a video stream;

extracting one or more features from a specified frame of the video packet via one or more histograms and frequency domain coefficients of the specified frame;

implementing one or more concept detectors on the one or more features creating one or more confidence values; and

transmitting the confidence values to a display module for filtering of video packets.

28. The article of manufacture of claim 27, wherein the step of extracting one or more features comprises the steps of:

parsing the video packet to attain a specified frame;

calculating a histogram for each of one or more slices of the specified frame using frequency domain coefficients of the specified frame; and

creating a feature vector of the specified frame based on the one or more calculated histograms.

29. The article of manufacture of claim 28, wherein the step of calculating a histogram comprises the steps of:

determining frequency domain coefficients of the specified frame;

partitioning the specified frame into one or more slices; and

calculating a histogram of the frequency domain coefficients for each of the one or more slices of the specified frame.