Redundancy elimination in a content-adaptive video preview system

• US 20050200762A1
• Filed: 01/25/2005
• Published: 09/15/2005
• Est. Priority Date: 01/26/2004
• Status: Active Grant

• Current Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Original Assignee: Sony Limited (Sony Group Corp.)
• Inventors: Mayer, Matthias, Moser, Boris, Barletta, Antonio
• Granted Patent: US 8,090,200 B2
• Time in Patent Office: Days
• US Class Current: 348/700
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A content-adaptive video preview system (100) allows to go faster through a video than existing video skimming techniques. Thereby, a user can interactively adapt (S1) the speed of browsing and/or the abstraction level of presentation.

According to one embodiment of the invention, this adaptation procedure (S1) is realized by the following steps: First, differences between precalculated spatial color histograms associated with chronologically subsequent pairs of video frames said video file is composed of are calculated (S1a). Then, these differences and/or a cumulative difference value representing the sum of these differences are compared (S1b) to a predefined redundancy threshold (S(t)). In case differences in the color histograms of particular video frames (302a-c) and/or said cumulative difference value exceed this redundancy threshold (S(t)), these video frames are selected (S1c) for the preview. Intermediate video frames (304a-d) are removed and/or inserted (S1d) between each pair of selected chronologically subsequent video frames depending on the selected abstraction level of presentation. Thereby, said redundancy threshold value (S(t)) can be adapted (S1b′) for changing the speed of browsing and/or the abstraction level of presentation.

Claims

1. A method for previewing a video file,

characterized in that the video frames to be displayed during the preview are selected as follows:

calculating (S1a) differences between histograms associated with subsequent video frames of said video file,

comparing (S1b) these differences and/or a cumulative difference value representing the sum thereof to a predefined redundancy threshold (S(t)), and

in case said differences in the histograms of particular video frames and/or said cumulative difference value exceed this redundancy threshold (S(t)), selecting (S1c) these video frames for the preview.

2. A method according to claim 1,

characterized by the step of

adapting (S1b′

) said redundancy threshold value (S(t)) in order to change the speed of browsing of the preview and/or the abstraction level of presentation.

3. A method according to claim 1,

characterized in that

differences (d(i,j)) between video frames (i and j) to be compared are also detected in the color or audio spectrum of the video file to be previewed, in the spatial configuration of pixels or regarding moving objects and/or motion of complete scenes.

4. A method according to claim 1,

characterized by the steps of

associating (S2a) meta data of any kind allowing users to identify segmented parts of video files to be previewed to said video files and

synchronizing (S2b) said meta data with said video files.

5. A method according to claim 1,

characterized in that

said color histograms are converted to a gray scale or to an adequate subset of the colors contained in said video frames before said differences are calculated.

6. A method according to claim 1,

characterized in that

the step of calculating (S1a) differences (d(i,j)) between precalculated spatial color histograms (H_i and H_j) comprises the steps of

segmenting (S3a) video frames (400a+b) to be compared into a certain number (M) of regions and assigning an index (m) to each region,

calculating (S3b) color histograms (H_im and H_jm) for the particular regions of said video frames (400a+b), and

pairwise comparing (S3c) those regions having the same index and calculating (S3d) the differences between the color histograms (H_im and H_jm) of these regions.

7. A method according to claim 1,

characterized in that

the difference d(i,j) between two frames (i and j) to be compared is calculated as the accumulated differences d_m(i,j) of all (M) regions these frames are composed of.

8. A method according to claim 1,

characterized in that

the difference d(i,j) between two frames (i and j) to be compared is the minimal or maximal difference of all (M) regions these frames are composed of.

9. A method according to claim 1,

characterized in that

the difference d(i,j) between two frames (i and j) to be compared is calculated as the arithmetic mean of the differences of all (M) regions these frames are composed of.

10. A method according to claim 1,

characterized in that

the step of calculating (S1a) said differences (d(i,j)) between precalculated spatial color histograms (H_i and H_j) comprises the steps of

separately calculating (S4a) color histograms (HR_i, HG_i, HB_i and HR_j, HG_j, HB_j) for each color channel (R,G,B) of the video frames (400a+b) to be compared, and

pairwise comparing (S4b) color histograms (HR_i and HR_j, HG_i and HG_j, HB_i and HB_j) associated to the same color channel of said video frames (400a+b) and calculating (S4c) the differences between these color histograms.

11. A method according to claim 1,

characterized in that

the step of calculating (S1a) differences between precalculated spatial color histograms for chronologically subsequent pairs of video frames said video file is composed of refers to all immediately chronologically adjacent pairs of video frames contained in said video file.

12. A method according to claim 1,

characterized in that

the step of calculating (S1a) differences between precalculated spatial color histograms for chronologically subsequent pairs of video frames said video file is composed of refers to video frames with a distance (n) of more than one video frame.

13. A method according to claim 1,

characterized in that the steps of calculating (S1a) and comparing (S1b) said differences for selecting (S1c) specific video frames for the preview are realized by the following steps:

for each selected video frame (i) calculating (S5a) the differences (d(i,i+κ

)) between the color histogram (H1) of the respectively selected video frame (i) and the color histograms (H<sub>i+κ

) of all chronologically subsequent video frames (i+κ

) until a distance (k) is found such that the difference between the selected video frame (i) and the corresponding subsequent video frame (i+k) exceeds a predefined redundancy threshold (S(t)) and</sub>

selecting (S5b) this subsequent video frame (i+k) for the preview.

14. A method according to claim 1,

characterized in that the steps of calculating (S1a) and comparing (S1b) said differences for selecting (S1c) specific video frames for the preview are realized by the following steps to be performed for each selected video frame (i):

for each immediately chronologically adjacent pair of video frames calculating (S6a) differences (d(i+κ

,i+κ

+1)) in the color histogram (H_i) of the respectively selected video frame (i) and the color histograms (H_i+κ

) of the immediately chronologically adjacent video frame (i+κ

),

accumulating (S6b) these differences until a chronologically subsequent video frame (i+k) is found for which the content of the respective difference image exceeds a predefined redundancy threshold (S(t)), and

selecting (S6c) this subsequent video frame (i+k) for the preview.

15. A method according to claim 1,

characterized in that the steps of calculating (S1a) and comparing (Sib) said differences for selecting (S1c) specific video frames for the preview are realized by the following steps to be performed for each selected video frame (i):

defining (S7a) an initial distance (n ;

=n₀) between the selected video frame (i) and a new video frame (j) to be selected for the preview, said initial distance (no) being an integer value greater than one,

in case the difference (d(i,j)|_j=i+n) between these two video frames (i and j) falls below a predefined redundancy threshold (S(t)), multiplying (S7b) said distance (n) with a constant integer factor (l) and defining (S7b′

) the obtained result (n ;

=n·

l) as a new distance between the selected video frame (i) and a new video frame (j) to be selected for the preview,

otherwise dividing (S7c) said distance (n) by said factor (f) and defining (S7c′

) the obtained result (n;

=n/l) rounded down to the nearest integer as a new distance between the selected video frame (i) and a new video frame (j) to be selected for the preview,

iterating (S7d) steps (S7b) and (S7c) until a minimum for said distance (n) is found such that said difference (d(i,j)|_j=i+n) is the smallest difference exceeding said redundancy threshold (S(t)) among all differences compared with said threshold for a given initial distance (no),

in case said initial distance (no) is an integer value greater than one, decrementing (S7e) said initial value by one, and

repeating (S7f) steps (S7b), (S7c) and (S7d) until a global 5 minimum for said distance (n) is found such that said difference (d(i,j)|_j=i+n) is the smallest difference exceeding said redundancy threshold (S(t)) among all differences compared with said threshold for all chosen initial distances (no).

16. A method according to claim 1,

characterized in that

precalculated histograms, said differences and/or predefined redundancy thresholds are temporarily stored in a database of said multimedia server (102).

17. A method according to claim 1,

characterized in that

redundant or slowly changing content in chronologically subsequent video frames said video file is composed of is eliminated while other video frames containing fast changing content are replicated and/or interpolated depending on user commands instructing the video preview system (100) to either quicker or slower browse through the content of said video files such that the degree of presented details is the higher the lower the speed of presentation and vice versa.