VIDEO SEGMENT MOTION CATEGORIZATION

• US 20080002771A1
• Filed: 06/30/2006
• Published: 01/03/2008
• Est. Priority Date: 06/30/2006
• Status: Abandoned Application

• Current Assignee: Nokia Solutions and Networks US LLC (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Chen, George Qian
• Application Number:
  US11/428,246
• Publication Number:
  US 20080002771A1
• Time in Patent Office: Days
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/527 Global motion vector estimation

Abstract

Analysis of video segments based upon the type of motion displayed in the video segments. A video segment is analyzed to determine if it displays a scene that is stationary or has motion. If a video segment displays a scene with motion, then the segment is further analyzed to determine if the motion resulted from camera movement, or if it resulted from movement of the object that was filmed. If the video segment displays a scene with motion created by camera movement, then the video segment is analyzed to determine if the movement was caused by controlled camera movement or unstable camera movement. These categories of video motion may then be used to determine the perceptual importance of the video segment. If the video segments are in a compressed data format, such as the MPEG-2 or MPEG-4 format, the motion displayed in the video segments can be categorized based upon motion vectors in the compressed data.

Claims

1. A method of categorizing a video segment, comprising:

analyzing a plurality of frames in a video segment to determine, for each analyzed frame, a position change of at least one image portion in the analyzed frame relative to a corresponding image portion in another frame;

if the determined position changes in the video segment have a representative magnitude below a first threshold value, categorizing the video segment as a stationary video segment;

if the representative magnitude of the determined position changes in the video segment is at or above the first threshold value, then, for each analyzed frame, determining differences between at least one second image portion in the analyzed frame relative to at least one second corresponding image portion in another frame;

if the determined differences have a representative discrepancy above a second threshold value, categorizing the video segment as a complex video segment;

if the representative discrepancy of the determined differences is at or below the second threshold value, identifying motion changes of corresponding third image portions in the frames of the video segment in substantially opposite directions;

if the identified motion direction changes occur at a representative frequency above a third threshold value, categorizing the video segment as a shaky video segment; and

if the identified position direction changes occur at a representative frequency at or below a third threshold value, then categorizing the video segment as a moving video segment.

2. The method recited in claim 1, wherein

the video segment is encoded using a compressed digital format;

and

further comprising, for each frame, using a motion vector of the at least one image portion in the frame to determining the position change of the at least one image portion in the frame relative to the corresponding image portion in another frame.

3. The method recited in claim 2, wherein

the motion vector has components dx and dy;

and

further comprising determining a magnitude of determined position changes for each frame to be |dx|+|dy|.

4. The method recited in claim 3, wherein the representative magnitude of the determined position changes in the video segment is an average of the magnitude of determined position changes for each frame in the video segment

5. The method recited in claim 2, wherein

the compressed data format is the MPEG-2 or MPEG-4 format, and

the at least one first image portion is a block.

6. The method recited in claim 1, wherein

the video segment is encoded using a compressed digital format;

and

further comprising using affine modeling to determine the differences between the at least one second image portion in the frame relative to the at least one second corresponding image portion in another frame.

7. The method recited in claim 6, further comprising obtaining the representative discrepancy of the determined differences from a residual of the affine modeling.

8. The method recited in claim 7, wherein the second threshold is ninety percent of the representative magnitude of the determined position changes in the video segment.

9. The method recited in claim 6, wherein the affine modeling employs a four parameter affine model.

10. The method recited in claim 6, wherein

the compressed data format is the MPEG-2 or MPEG-4 format;

and

the at least one first image portion is a block.

11. The method recited in claim 6, further comprising identifying motion direction changes in substantially opposite directions based upon parameters employed in the affine modeling.

12. A video segment analysis tool, comprising:

a position determination module configured to

determine, for frames in a video segment, a position change of at least one first image portion in a frame relative to a first corresponding image portion in another frame, and

if the determined position changes in the video segment have a representative magnitude below a first threshold value, categorize the video segment as a stationary video segment;

a difference determination module configured to

determine, for frame in the video segment, differences between at least one second image portion in the frame relative to at least one second corresponding image portion in another frame; and

if the representative magnitude of the determined position changes in the video segment is at or above the first threshold value and if the determined differences have a representative discrepancy above a second threshold value, categorize the video segment as a complex video segment; and

a motion direction change identification module configured to identify motion changes of corresponding third image portions in the frames of the video segment in substantially opposite directions, and

if the representative magnitude of the determined position changes in the video segment is at or above the first threshold value, if the determined differences have a representative discrepancy at or below the second threshold value, and if the identified motion direction changes have a representative frequency above a third threshold value, categorize the video segment as a shaky video segment; and

if the representative magnitude of the determined position changes in the video segment is at or above the first threshold value, if the determined differences have a representative discrepancy at or below the second threshold value, and if the identified position direction changes occur at a representative frequency at or below a third threshold value, categorize the video segment as a moving video segment.

13. The video segment analysis tool recited in claim 12, wherein

the video segment is encoded using a compressed digital format;

and

the position determination module is configured to use a motion vector of the at least one image portion in the frame to determine the position change of the at least one image portion in the frame relative to the corresponding image portion in another frame.

14. The video segment analysis tool recited in claim 13, wherein

the motion vector has components dx and dy;

and

the position determination module is configured to determine a magnitude of determined position changes for each frame to be |dx|+|dy|.

15. The video segment analysis tool recited in claim 14, wherein the position determination module configured to determine the representative magnitude of the determined position changes in the video segment to be an average of the magnitude of determined position changes for each frame in the video segment

16. The video segment analysis tool recited in claim 13, wherein

the compressed data format is the MPEG-2 or MPEG-4 format, and

the at least one first image portion is a block.

17. The video segment analysis tool recited in claim 12, wherein

the video segment is encoded using a compressed digital format;

and

the difference determination module is configured to use affine modeling to determine the differences between the at least one second image portion in the frame relative to the at least one second corresponding image portion in another frame.

18. The video segment analysis tool recited in claim 17, wherein the difference determination module is configured to obtain the representative discrepancy of the determined differences from a residual of the affine modeling.

19. The video segment analysis tool recited in claim 18, wherein the second threshold is ninety percent of the representative magnitude of the determined position changes in the video segment.

20. The video segment analysis tool recited in claim 17, wherein the difference determination module is configured to employ a four parameter affine model for the affine modeling.

21. The video segment analysis tool recited in claim 17, wherein

the compressed data format is the MPEG-2 or MPEG-4 format;

and

the at least one first image portion is a block.

22. The video segment analysis tool recited in claim 17, wherein the motion direction change identification module is configured to identify motion direction changes in substantially opposite directions based upon parameters employed in the affine modeling