Capture-intention detection for video content analysis

• US 20070101269A1
• Filed: 10/31/2005
• Published: 05/03/2007
• Est. Priority Date: 10/31/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mei, Tao, Hua, Xian-Sheng, Li, Shipeng
• Granted Patent: US 7,773,813 B2
• Time in Patent Office: Days
• US Class Current: 715/723
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

Systems and methods are described for detecting capture-intention in order to analyze video content. In one implementation, a system decomposes video structure into sub-shots, extracts intention-oriented features from the sub-shots, delineates intention units via the extracted features, and classifies the intention units into intention categories via the extracted features. A video library can be organized via the categorized intention units.

Claims

1. A method, comprising:

delineating video data into intention units; and

classifying the intention units into intention categories.

2. The method as recited in claim 1, wherein each of the intention categories represents one of multiple human intentions for capturing images with a camera to create the video data.

3. The method as recited in claim 1, wherein the delineating video data into intention units further includes decomposing a video structure into sub-shots based on camera motion thresholds.

4. The method as recited in claim 1, further comprising extracting features from the video data, wherein each feature is used to estimate one or more of the human intentions.

5. The method as recited in claim 4, wherein the extracting features includes extracting attention-specific features, wherein each attention-specific feature represents one dimension of human attention.

6. The method as recited in claim 5, wherein the extracting attention-specific features includes analyzing four dimensions of attention (DoA):

an attention stability, an attention energy, an attention window, and a camera pattern.

7. The method as recited in claim 4, wherein the extracting features includes extracting content generic features, wherein each content generic feature comprises a low-level visual feature that reflects the capture intention by strengthening a relationship between the capture-intention and low-level features.

8. The method as recited in claim 1, further comprising selecting a number of categories to be the intention categories and defining each of the intention categories according a type of video content characteristic of one of the human intentions.

9. The method as recited in claim 8, wherein each different human intention corresponding to one of the intention categories is capable of being indicated in the video content by intention-oriented features.

10. The method as recited in claim 8, wherein the intention categories include a static scene category, a dynamic event category, a close-up view category, a beautiful scenery category, a switch record category, a longtime record category, and a just record category.

11. The method as recited in claim 1, wherein the delineating video data into intention units includes dividing the video data into temporal sub-shots, wherein each sub-shot comprises a camera motion, and each intention unit comprises one or more sub-shots assignable to the same one or more intention categories.

12. The method as recited in claim 11, wherein the delineating the video data into the intention units includes comparing both attention-specific features and content generic features of the contiguous sub-shots, this comparing performed to determine the number of sub-shots to be included in a given intention unit.

13. The method as recited in claim 12, further comprising classifying each intention unit as belonging to one or more of the intention categories based on the attention-specific features and the content generic features.

14. The method as recited in claim 1, wherein the classifying each intention unit to one or more intention categories includes a learning-based classification of the intention units.

15. The method as recited in claim 14, wherein the learning-based classification includes applying one of a support vector machine (SVM) classification schema or a Boosting classification schema.

16. The method as recited in claim 1, further comprising organizing a video recording or multiple video recordings in a video library according to the classification of the intention units into the intention categories.

17. A system, comprising:

a shot detector to determine temporal segments of video shots in video data;

a sub-shot detector to determine temporal segments of sub-shots in the video shots;

a feature analyzer to determine both attention-specific characteristics and content-generic characteristics for each of multiple features of each sub-shot, wherein the attention characteristic indicates a person'"'"'s attention degree on the scene or object to be captured or having been captured;

an intention unit segmenter to delineate intention units composed of the sub-shots according to the attention characteristics of the features of the sub-shots; and

an intention classifier to assign each intention unit to an intention category, such that the video data is capable of being organized by intention units.

18. The system as recited in claim 17, wherein the multiple features of a sub-shot include attention-specific features and content generic features.

19. The system as recited in claim 17, wherein the intention classifier includes a learning engine to train the classification of intention units into intention categories by applying one of a support vector machine (SVM) classification schema or a Boosting classification schema.

20. A system, comprising:

means for delineating video data into intention units; and

means for classifying the intention units into intention categories.