VIDEO PROCESSING APPARATUS, METHOD AND SYSTEM

• US 20120076357A1
• Filed: 09/22/2011
• Published: 03/29/2012
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Yamamoto, Koji, Hirohata, Makoto
• Granted Patent: US 8,879,788 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

According to one embodiment, a video processing apparatus includes an acquisition unit, a first extraction unit, a generation unit, a second extraction unit, a computation unit and a selection unit. The acquisition unit is configured to acquire video streams. A first extraction unit is configured to analyze at least one of the moving pictures and the sounds for each video stream and to extract feature values. A generation unit is configured to generate segments by dividing each video stream, and to generate associated segment groups. A second extraction unit is configured to extract the associated segment groups that number of associated segments is greater than or equal to threshold as common video segment groups. A computation unit is configured to compute summarization score. A selection unit is configured to select segments used for a summarized video as summarization segments from the common video segment groups based on the summarization score.

Claims

1. A video processing apparatus, comprising:

an acquisition unit configured to acquire a plurality of video streams each including moving picture data items and sound data items;

a first extraction unit configured to analyze at least one of the moving picture data items and the sound data items for each video stream, and to extract a feature value from the analyzed one, the feature value indicating a common feature between the plurality of video streams;

a generation unit configured to generate a plurality of segments by dividing each of the video streams in accordance with change in the feature value, and to generate associated segment groups by associating a plurality of segments between different video streams, each associated segment included in the associated segment groups having a similarity of feature value between the segments greater than or equal to a first threshold value;

a second extraction unit configured to extract, from the associated segment groups, one or more common video segment groups in which number of associated segments is greater than or equal to a second threshold value, the number of the associated segments being number of different video streams each including the associated segment which corresponds each of the associated segment groups;

a computation unit configured to compute a summarization score indicating a degree of suitability for including a segment of the common video segment group in a summarized video created from a part of the video streams, the summarization score varying with time and being based on the feature value extracted at least one of the moving picture data items and the sound data items; and

a selection unit configured to select summarization segments to be used for the summarized video from the common video segment groups based on the summarization score.

2. The apparatus according to claim 1, wherein the generation unit is configured to generate the associated segment groups in accordance with the feature value of the sound data items, and the computation unit is configured to compute the summarization score at least based on the feature value of the moving picture data items.

3. The apparatus according to claim 1, further comprising:

a detection unit configured to compute likelihood of matching the feature value with feature value models of typical shot patterns indicating a combination of shots creating a predetermined scene, and to detect the feature value in which the likelihood is greater than or equal to a third threshold value; and

a correction unit configured to generate correction value for the summarization score computed from the feature value in which the likelihood is greater than or equal to the third threshold value,

wherein the selection unit selects the summarization segments based on the summarized score in which the correction values are added.

4. The apparatus according to claim 1, wherein the video streams include authorization information items indicating whether or not the video streams are authorized to be used, respectively,

the apparatus further comprising a management unit configured to request authorization to users owning the video streams if the selected summarization segments are unauthorized,

wherein the selection unit is configured to eliminate the summarization segments which are unauthorized to be used or select the authorized segments as the summarization segments.

5. A video processing apparatus, comprising:

an acquisition unit configured to acquire a plurality of video streams each including moving picture data items and sound data items;

a first extraction unit configured to analyze at least one of the moving picture data items and sound data items for each video streams, and to extract a feature value from the analyzed one, the feature value indicating a common feature between the plurality of video streams;

a generation unit configured to generate a plurality of segments by dividing each of the video streams in accordance with change in the feature value, and to generate associated segment groups by associating a plurality of segments between the different video streams, each associated segment included in the associated segment groups having a similarity of feature value between the segments greater than or equal to a first threshold value;

a computation unit configured to compute a user relevant score indicating high relevance between users who own the associated segment groups having high similarity for each of the users;

a decision unit configured to decide whether or not the user relevant score is greater than or equal to a fourth threshold value;

a notification unit configured to send a message to the users each having the user relevant score greater than or equal to the fourth threshold value.

6. The apparatus according to claim 5, wherein in a case where a plurality of communities in which a plurality of users join exist, the notification unit is configured to send a message relating to the communities if the user relevant score are greater than or equal to a community relevant score computed from the user relevant score of the users belonging to the communities.

7. A video processing apparatus, comprising:

a receiving unit configured to receive a plurality of sound data items from a plurality of video streams each including moving picture data items and sound data items;

a first extraction unit configured to analyze the sound data items and to extract first feature value, the first feature value representing feature of sound data items and indicating common feature between the plurality of sound data items;

a generation unit configured to generate a plurality of segments by dividing each of the sound data items in the video streams in accordance with change in the first feature value, and to generate associated segment groups by associating a plurality of segments of different sound data items in the video stream, each associated segment included in the associated segment groups having a similarity of first feature value between the segments greater than or equal to a first threshold value;

a second extraction unit configured to extract, from the associated segment groups, one or more common video segment groups in which number of segments is greater than or equal to a second threshold value, the number of the associated segments being number of different sound data items in the video streams each including the associated segment which corresponds each of the associated segment groups,

wherein the receiving unit receives moving picture data items of the common video segment groups, and the first extraction unit extracts second feature value, the second feature value representing features of moving picture data items and indicating common features between the plurality of moving picture data items included in the common video segment groups,

a computation unit configured to compute a summarization score indicating a degree of suitability for including each segment of the common video segment groups in a summarized video created from a part of the video streams, the summarization score varying with time and being based on the second feature value; and

a selection unit configured to select summarization segments to be used for the summarized video from the common video segment groups based on the summarization score.

8. The video processing system comprising at least one client each comprising a first transmission/receiving unit and a server comprising a second transmission/receiving unit and operable to communicate with the client,

the client further comprising:

an acquisition unit configured to acquire a plurality of video streams including moving picture data items and sound data items,

the first transmission/receiving unit of the client transmitting the sound data items;

the second transmission/receiving unit of the server receiving the sound data items;

the server further comprising;

a first extraction unit configured to analyze the sound data items and to extract first feature values, the first feature values representing features of sound data items and indicating common features between the sounds;

a generation unit configured to generate a plurality of segments by dividing each of the video streams in accordance with change in the first feature values, and to generate associated segment groups by associating a plurality of segments between different video streams, each associated segment included in the associated segment groups having a similarity of feature value between the segments greater than or equal to a first threshold value; and

a second extraction unit configured to extract, from the associated segment groups, one or more common video segment groups in which a number of associated segments is greater than or equal to a second threshold value, the number of different video streams each including the associated segment which corresponds each of the associated segment groups,

the second transmission/receiving unit of the server transmitting the common video segment groups,

the first transmission/receiving unit of the client receiving the common video segment groups and transmitting moving picture data items included in the common video segment groups,

the second transmission/receiving unit of the server receiving the moving pictures included in the common video segment groups,

the first extraction unit extracting second feature value, the second feature values representing feature of moving picture data items and indicating common features between the moving picture data items included in the common video segment groups,

the server further comprising;

a computation unit configured to compute a summarization score indicating a degree of suitability for including each segment of the common video segment groups in a summarized video created from a part of the video streams, the summarization score varying with time and being based on the second feature value; and

a selection unit configured to select summarization segments to be used for the summarized video from the common video segment groups based on the summarization score,

the second transmission/receiving unit of the server transmitting the summarization segments.

9. A video processing apparatus, comprising:

an acquisition unit configured to acquire at least one still picture group each including a plurality of still pictures;

a first extraction unit configured to analyze the still picture group and to extract feature values, the feature values indicating a common feature between the plurality of still picture groups;

a generation unit configured to generate associated still picture groups by associating a plurality of still pictures, each associated segment included in the associated still picture groups having a similarity of a feature value between the still pictures within each still picture group greater than or equal to a first threshold value;

a second extraction unit configured to extract, from the associated still picture groups, one or more common still picture groups in which number of associated still pictures is greater than or equal to a second threshold value;

a computation unit configured to compute a summarization score indicating a degree of suitability for including a segment of the common still picture group in a summarized still picture created from a part of the still pictures, the summarization score varying with time and being based on the feature value of the common still picture groups; and

a selection unit configured to select still pictures to be used for the summarized still picture from the common still picture groups based on the summarization score.

10. A video processing method, comprising

acquiring a plurality of video streams each including moving picture data items and sound data items;

analyzing at least one of the moving picture data items and the sound data items for each video stream, extracting a feature value from the analyzed one, the feature value indicating a common feature between the plurality of video streams;

generating a plurality of segments by dividing each of the video streams in accordance with change in the feature value, and to generate associated segment groups by associating a plurality of segments of between different video streams, each associated segment included in the associated segment groups having a similarity of feature value between the segments greater than or equal to a first threshold value;

extracting, from the associated segment groups, one or more common video segment groups in which number of the associated segments is greater than or equal to a second threshold value, the number of the associated segments being number of different video streams each including the associated segment which corresponds each of the associated segment groups;

computing a summarization score indicating a degree of suitability including a segment of the common video segment group in a summarized video created from a part of the video streams, the summarization score varying with time and being based on the feature value extracted at least one of the moving picture data items and the sound data items; and

selecting summarization segments to be used for the summarized video from the common video segment groups based on the summarization score.