Descriptor for spatial distribution of motion activity in compressed video

• US 6,600,784 B1
• Filed: 02/02/2000
• Issued: 07/29/2003
• Est. Priority Date: 02/02/2000
• Status: Expired due to Term

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Sun, Huifang, Divakaran, Ajay, Peker, Kadir A.
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): PARSONS, CHARLES E
• Application Number:
  US09/496,707
• Time in Patent Office: 1,273 Days
• Field of Search: 375/240.16, 375/340.24, 375/240.08, 348/416.1, 348/699, 382/239, 382/236, 382/168, 382/170, 382/171
• US Class Current: 375/240.14
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method describes motion activity in a video sequence. A motion activity matrix is determined for the video sequence. A threshold for the motion activity matrix is determined. Connected regions of motion vectors at least equal to the threshold are identified and measured for size. A histogram of the distribution of the sizes of the connected areas is constructed for the entire video sequence. The histogram is normalized to characterize the spatial distribution of the video sequence in a motion activity descriptor.

Claims

1. A method for describing motion activity in a video, comprising the steps of:

determining a motion activity matrix for the video;

determining a threshold for the motion activity matrix;

identifying connected regions with motion vectors at least equal to the threshold;

measuring the size of the connected regions;

constructing a histogram of the connected areas of the video; and

normalizing the histogram to characterize the spatial distribution of the video in a motion activity descriptor.

2. The method of claim 1 wherein each frame of the video includes a plurality of macroblocks, and wherein each macro block includes a motion vector, and wherein the motion vectors of the macroblocks of the frames determine the motion activity matrix.

3. The method of claim 2 wherein the connected areas are immediately adjacent macroblocks having motion vectors at least equal to the threshold, and the size of each area is equal to the number of macroblocks in the areas.

4. The method of claim 3 wherein the threshold is an average motion vector magnitude.

5. The method of claim 4 wherein the threshold is the average motion vector magnitude plus a predetermined constant.

6. The method of claim 3 wherein the threshold is a median of the motion vector magnitudes.

7. The method of claim 3 wherein the threshold is a most common motion vector magnitude.

8. The method of claim 3 wherein the threshold is a boundary between two clusters of motion vector magnitudes.

9. The method of claim 1 wherein the histogram has a plurality of bins, and the bins follow a logarithmic-like scale.

10. The method of claim 2 wherein connected areas larger than half the area of the frame indicate global motion.

11. The method of claim 9 wherein the normalizing further comprises:

dividing the value in each bin by the average size of connected regions of the histogram with rounding; and

applying a square root operation on the values of the bins as a compander.

12. The method of claim 1 further comprising:

describing a first video with a first motion activity descriptor derived from a first histogram;

describing a second video with a second motion activity descriptor derived from a second histogram; and

comparing the first and second motion activity descriptors to match the first and second videos.

13. The method of claim 12 further comprising the steps of:

left shifting the first histogram;

right shifting the first histogram;

no shifting the first histogram;

comparing the left, right, and no shifted first histogram with a second histogram to produce a first, second, and third distance, respectively;

weighting the first, second, and third distances.

14. The method of claim 13 further comprising the step of:

selecting the minimum of the weighted first, second, and third distances as a final distance between the first and second videos.

15. The method of claim 13 further comprising the step of:

summing the first, second, and third weighted distances to determine a final distance between the first and second videos.