AV signal processing apparatus and method as well as recording medium

• US 20020061136A1
• Filed: 07/13/2001
• Published: 05/23/2002
• Est. Priority Date: 07/14/2000
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Shibata, Hiromasa, Walker, Toby
• Granted Patent: US 7,027,508 B2
• Time in Patent Office: Days
• US Class Current: 382/199
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

The invention provides an AV signal processing apparatus and method by which a boundary between scenes is detected so that recorded video data can be played back beginning with an arbitrary scene. First, video data inputted is divided into video segments or audio segments or, if possible, into both of video and audio segments. Then, feature amounts representative of features of the segment are calculated, and then, similarity measurement between segments is performed using the feature amounts. Thereafter, it is discriminated whether or not the segment corresponds to a break of a scene. Thus, the video-audio processing apparatus uses the dissimilarity measurement criterion and the feature amounts calculated as above to determine, regarding each segment as the reference segment at present, in which one of the past and the future with respect to the reference segment the ratio of presence of neighboring similar segments is higher, and investigates the pattern of the change of the ratio to discriminate whether or not the reference segment corresponds to a boundary of a scene.

Claims

1. An AV signal processing apparatus for detecting and analyzing a pattern which reflects a significance structure of contents of an AV signal supplied thereto to detect a scene of a significant break, comprising:

feature amount extraction means for extracting feature amounts of segments each formed from a series of frames which form the AV signal;

calculation means for calculating a measurement criterion to be used for measurement of a similarity of the feature amounts between a reference segment and other segments;

similarity measurement means for using the measurement criterion to measure the similarity between the reference segment and the other segments;

measurement value calculation means for using the similarity measured by said similarity measurement means to calculate a measurement value indicative of a possibility that the reference segment may be a boundary of the scene; and

boundary discrimination means for analyzing a variation of a pattern with respect to time of the measurement value calculated by said measurement value calculation means and discriminating based on a result of the analysis whether or not the reference segment is the boundary of the scene.

2. An AV signal processing apparatus according to claim 1, wherein the AV signal includes at least one of a video signal and an audio signal.

3. An AV signal processing apparatus according to claim 1, further comprising intensity value calculation means for calculating an intensity value indicative of a degree of the variation of the measurement value corresponding to the reference segment.

4. An AV signal processing apparatus according to claim 1, wherein said measurement value calculation means calculates similar segments in a predetermined time area with respect to the reference segment, analyses the time distribution of the similar segments and determines a ratio at which the similar segments are present in the past and in the future to calculate the measurement value.

5. An AV signal processing apparatus according to claim 1, wherein said boundary discrimination means discriminates based on a sum total of the absolute values of the measurement values whether or not the reference segment is the boundary of the scene.

6. An AV signal processing apparatus according to claim 2, further comprising audio segment production means for detecting, when the AV signal includes a video signal, a shot which is a basic unit of a video segment to produce the audio segment.

7. An AV signal processing apparatus according to claim 2, further comprising audio segment production means for using, when the AV signal includes an audio signal, at least one of the feature amount of the audio signal and a no sound period to produce an audio segment.

8. An AV signal processing apparatus according to claim 2, wherein the feature amounts of the video signal at least include a color histogram.

9. An AV signal processing apparatus according to claim 2, wherein the feature amounts of the video signal at least include at least one of a sound volume and a spectrum.

10. An AV signal processing apparatus according to claim 1, wherein said boundary discrimination means compares the measurement value with a preset threshold value to discriminate whether or not the reference segment is a boundary of the scene.

11. An AV signal processing method for an AV signal processing apparatus for detecting and analyzing a pattern which reflects a significance structure of contents of an AV signal supplied thereto to detect a scene of a significant break, comprising:

a feature amount extraction step of extracting feature amounts of segments each formed from a series of frames which form the AV signal;

a calculation step of calculating a measurement criterion to be used for measurement of a similarity of the feature amounts between a reference segment and other segments;

a similarity measurement step of using the measurement criterion to measure the similarity between the reference segment and the other segments;

a measurement value calculation step of using the similarity measured by the processing in the similarity measurement step to calculate a measurement value indicative of a possibility that the reference segment may be a boundary of the scene; and

a boundary discrimination step of analyzing a variation of a pattern with respect to time of the measurement value calculated by the processing in the measurement value calculation step and discriminating based on a result of the analysis whether or not the reference segment is the boundary of the scene.

12. A recording medium on which a computer-readable program for AV signal processing for detecting and analyzing a pattern which reflects a significance structure of contents of a supplied AV signal to detect a scene of a significant break is recorded, the program comprising:

a feature amount extraction step of extracting feature amounts of segments each formed from a series of frames which form the AV signal;

a calculation step of calculating a measurement criterion to be used for measurement of a similarity of the feature amounts between a reference segment and other segments;

a similarity measurement step of using the measurement criterion to measure the similarity between the reference segment and the other segments;

a measurement value calculation step of using the similarity measured by the processing in the similarity measurement step to calculate a measurement value indicative of a possibility that the reference segment may be a boundary of the scene; and

a boundary discrimination step of analyzing a variation of a pattern with respect to time of the measurement value calculated by the processing in the measurement value calculation step and discriminating based on a result of the analysis whether or not the reference segment is the boundary of the scene.