Methods and architecture for indexing and editing compressed video over the world wide web

• US 6,735,253 B1
• Filed: 03/14/2000
• Issued: 05/11/2004
• Est. Priority Date: 05/16/1997
• Status: Expired due to Term

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Chang, Shih-Fu, Meng, Horace J.
• Primary Examiner(s): Rao, Andy
• Application Number:
  US09/423,769
• Time in Patent Office: 1,519 Days
• Field of Search: 375/240-1-2
• US Class Current: 375/240.16
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Techniques for detecting moving video objects in a compressed digital bitstream (111) and for tools for editing compressed video are disclosed. Video objects (117) are detected and indexed by analyzing a compressed bitstream to locate scene cuts (112), estimating operating parameters for a camera which initially viewed the video (114), and detecting one or more moving video objects represented in the compressed bitstream by applying global motion compensation which account for the estimated operating parameters. Tools are provided for permitting dissolve, masking, freeze frame, slow and variable speed playback, and strobe motion special effects to compressed video. The tools may be implemented in a system for editing (130) compressed video information over a distributed network.

Claims

1. A method for detecting moving video objects in a compressed digital bitstream which represents a sequence of fields or frames of video information for one or more previously captured scenes of video, comprising the steps of:

a. analyzing said compressed bitstream to locate scene cuts therein, thereby determining at least one sequence of fields or frames of video information which represents a single video scene;

b. estimating one or more operating parameters for a camera which initially captured said video scene by analyzing a portion of said compressed bitstream which corresponds to said video scene; and

c. detecting one or more moving video objects represented in said compressed bitstream by applying global motion compensation with said estimated operating parameters.

2. The method of claim 1, further comprising the step of extracting visual features of said one or more detected moving video objects from said compressed bitstream.

3. The method of claim 1, wherein said compressed bitstream comprises a bitstream compressed in accordance with the MPEG video standard.

4. The method of claim 1, wherein said analyzing step further comprises the steps of:

a. parsing said compressed bitstream into blocks of video information and associated motion vector information for each field or frame of video information which comprises the determined sequence of fields or frames of video information representative of said single scene;

b. performing inverse motion compensation on each of said parsed blocks of video information to derive discrete cosign transform coefficients for each of said parsed blocks of video information;

c. counting said motion vector information associated with each of said parsed blocks of video information; and

d. Determining from said counted motion vector information and said discrete cosign transform coefficient information whether one of said scene cuts has occurred.

5. The method of claim 1, wherein said analyzing step comprises parsing said compressed bitstream into blocks of video information and associated motion vector information for each field or frame of video information which comprises the determined sequence of fields or frames of video information representative of said single scene, and wherein said estimating step comprises the step of estimating any zoom and any pan of said camera by determining a multi-parameter transform model applied to said parsed motion vector information.

6. The method of claim 5, wherein said estimating step comprises the steps of:

a. computing each parameter for a multi-parameter affine transform which represents a transformation from a current frame of video information to a previous frame of video; and

b. computing said multi-parameter affine transform to thereby determine global motion information representative of said zoom and pan of said camera.

7. The method of claim 6, wherein said detecting step comprises computing local object motion for said one or more moving video objects based on said global motion information and on one or more of said motion vectors which correspond to said one or more moving video objects.

8. The method of claim 7, further comprising the steps of:

a. determining whether said local object motion is greater than a predetermined threshold;

b. applying morphological operations to said determined local object motion values to eliminate any erroneously sensed moving objects; and

c. determining border points of said detected moving objects to thereby locate a bounding box for said detected moving object.

9. An apparatus for detecting moving video objects in a compressed digital bitstream which represents a sequence of fields or frames of video information for one or more previously captured scenes of video, comprising:

a. means for analyzing said compressed bitstream to locate scene cuts therein and to determine at least one sequence of fields or frames of video information which represents a single video scene;

b. means, coupled to said analyzing means, for estimating one or more operating parameters for a camera which initially viewed said video scene by analyzing a portion of said compressed bitstream which corresponds to said video scene; and

c. means, coupled to said estimating means, for detecting one or more moving video objects represented in said compressed bitstream by applying global motion compensation with said estimated operating parameters.

10. The apparatus of claim 9, further comprising means, coupled to said detecting means, for extracting visual features of said one or more detected moving video objects from said compressed bitstream.

11. The apparatus of claim 9, wherein said compressed bitstream comprises a bitstream compressed in accordance with the MPEG video standard, and wherein said analyzing means further comprises:

a. parsing means for receiving and parsing said compressed bitstream into blocks of video information and associated motion vector information for each field or frame of video information which comprises the determined sequence of fields or frames of video information representative of said single scene;

b. means, coupled to said parsing means, for performing inverse motion compensation on each of said parsed blocks of video information to derive discrete cosign transform coefficients for each of said parsed blocks of video information;

c. counting means, coupled to said inverse motion compensation means, for counting said motion vector information associated with each of said parsed blocks of vide information; and

d. determining means, coupled to said counting mens, for determining from said counted motion vector information and said discrete cosign transform coefficient information whether one of said scene cuts has occurred.

12. The apparatus of claim 9, wherein said analyzing means further comprises means for parsing said compressed bitstream into blocks of video information and associated motion vector information for each field or frame of video information which comprises the determined sequence of fields or frames of video information representative of said single scene, and wherein said estimating means further comprises means for estimating any zoom and any pan of said camera by determining a multi-parameter transform model applied to said parsed motion vector information.

13. The apparatus of claim 12, wherein said estimating means further comprises:

a. means for computing each parameter for a multi-parameter affine transform which represents a transformation from a current frame of video information to a previous frame of video; and

b. means, coupled to said transform parameter computing means, for computing said multi-parameter affine transform to thereby determine global motion information representative of said zoom and pan of said camera.

14. The apparatus of claim 12, wherein said detecting means further comprises means for computing local object motion for said one or more moving video objects based on said global motion information and on one or more of said motion vectors which correspond to said one or more moving video objects.

15. The apparatus of claim 14, further comprising:

a. comparison means, coupled to said local object motion computing means, for determining whether said local object motion is greater than a predetermined threshold;

b. morphological operation means, coupled to said comparison means, for determined local object motion values to eliminate any erroneously sensed moving objects; and

c. border point determination means, coupled to said morphological operation means, for determining border points of said detected moving objects to thereby locate a bounding box for said detected moving object.