Redundancy elimination in a content-adaptive video preview system

• US 8,090,200 B2
• Filed: 01/25/2005
• Issued: 01/03/2012
• Est. Priority Date: 01/26/2004
• Status: Expired due to Fees

• Current Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Original Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Inventors: Barletta, Antonio, Moser, Boris, Mayer, Matthias
• Primary Examiner(s): Ge, Yuzhen
• Application Number:
  US11/042,670
• Publication Number:
  US 20050200762A1
• Time in Patent Office: 2,534 Days
• Field of Search: None
• US Class Current: 382/168
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A content-adaptive video preview system (100) allows to go faster through a video than existing video skimming techniques. Thereby, a user can interactively adapt (S1) the speed of browsing and/or the abstraction level of presentation.

According to one embodiment of the invention, this adaptation procedure (S1) is realized by the following steps: First, differences between precalculated spatial color histograms associated with chronologically subsequent pairs of video frames said video file is composed of are calculated (S1a). Then, these differences and/or a cumulative difference value representing the sum of these differences are compared (S1b) to a predefined redundancy threshold (S(t)). In case differences in the color histograms of particular video frames (302a-c) and/or said cumulative difference value exceed this redundancy threshold (S(t)), these video frames are selected (S1c) for the preview. Intermediate video frames (304a-d) are removed and/or inserted (S1d) between each pair of selected chronologically subsequent video frames depending on the selected abstraction level of presentation. Thereby, said redundancy threshold value (S(t)) can be adapted (S1b′) for changing the speed of browsing and/or the abstraction level of presentation.

Claims

1. A method, implemented on a display system having at least a display device, for previewing a video file on the display device by selecting video frames to be displayed for a preview, comprising:

calculating, at the display system, differences between histograms associated with an active video frame and other frames of the video file, the calculating including determining the neighboring frame which is a frame with a difference between a histogram of the neighboring frame and a histogram of the active video frame that is closest to but greater than a predefined redundancy threshold than all the other frames, determining the neighboring frame being based on the differences previously calculated, the calculating including increasing a frame number of the next frame to be tested by a number greater than 1 when a difference between a histogram of a current frame being tested and the histogram of the active frame is less than the predefined redundancy threshold, and decreasing the frame number of the next frame to be tested when the difference between the histogram of the current frame being tested and the histogram of the active frame is greater than the predefined redundancy threshold;

comparing, at the display system, at least one of the differences or a cumulative difference value representing a sum of the differences to the predefined redundancy threshold, the redundancy threshold being a time dependent variable;

selecting, at the display system, particular video frames for the preview only when the at least one of said differences in the histograms of the particular video frames or said cumulative difference value exceeds the redundancy threshold so as to remove the redundancy of video frame content;

adapting, at the display system, said redundancy threshold to change a speed of browsing of the preview so that fast changing content is previewed slowly with more video frames selected per unit of time in comparison to slow changing content, and slow changing content is previewed fast with fewer video frames selected per unit of time in comparison to fast changing content; and

displaying the particular video frames for the preview on the display device.

2. The method according to claim 1, wherein said histograms associated with said subsequent video frames are calculated based on at least one of a color spectrum, audio spectrum, spatial configuration of pixels, parameters related to moving objects, or parameters related to motion of complete scenes of the video frames.

3. The method according to claim 1, further comprising:

associating meta data to said video files, configured to identify parts of video files to be previewed; and

synchronizing said meta data with said video files.

4. The method according to claim 1, further comprising:

converting color histograms of said histograms to a gray scale histogram or to an adequate subset of colors included in said video frames before said calculating differences.

5. The method according to claim 1, wherein the histograms are spatial color histograms that are pre-calculated, the step of calculating differences further comprising:

segmenting video frames to be compared into a certain number of regions;

assigning an index to each region;

calculating color histograms for at least some of the regions;

pairwise comparing regions having a same index of the frames that are to be compared; and

calculating differences between the color histograms of the regions having the same index.

6. The method according to claim 5, wherein the difference between two frames to be compared is calculated as an accumulated difference of all regions that are included in the two frames.

7. The method according to claim 5, further comprising:

calculating a minimal or a maximal difference of the differences of all regions that are included in the subsequent video frames, before said comparing; and

comparing the minimal or the maximal difference in said comparing.

8. The method according to claim 5, further comprising:

calculating an arithmetic mean of the difference of all regions that are in the subsequent video frames, before said comparing; and

comparing the arithmetic mean in said comparing.

9. The method according to claim 1, wherein the histograms are spatial color histograms that are pre-calculated, and the step of calculating differences further comprises:

separately calculating color histograms for each color channel of the video frames to be compared;

pairwise comparing color histograms associated to a same color channel of said video frames; and

calculating differences between the color histograms of the same color channel.

10. The method according to claim 1, wherein

the histograms are spatial color histograms that are pre-calculated, and

the pre-calculated spatial color histograms are based on the subsequent video frames that are immediately chronologically adjacent pairs of video frames included in said video file.

11. The method according to claim 1, wherein

the histograms are spatial color histograms that are pre-calculated, and

the pre-calculated spatial color histograms are based on the subsequent video frames that are video frames with a distance of more than one video frame included in said video file.

12. The method according to claim 1, further comprising:

calculating differences between a color histogram of one of the selected video frames of said selecting and color histograms of all chronologically subsequent video frames, until a distance is found such that a difference between the one of the selected video frame and a corresponding subsequent video frame exceeds the predefined redundancy threshold, for each selected video frame; and

selecting the subsequent video frame that exceeded the predefined redundancy threshold for the preview.

13. The method according to claim 1, further comprising:

calculating differences between color histograms of the selected video frames of said selecting, and color histograms of the immediately chronologically adjacent video frame, for each immediately chronologically adjacent pair of video frames;

accumulating the differences in the color histograms until a chronologically subsequent video frame is found for which a content of a respective difference image exceeds the predefined redundancy threshold; and

selecting the subsequent video that exceeded the predefined redundancy threshold frame for the preview.

15. The method according to claim 1, wherein the histograms are pre-calculated, and at least one of said differences or predefined redundancy thresholds are temporarily stored in a database of a multimedia server.

16. The method according to claim 1, wherein in said step of displaying the particular video frames, more video frames per unit time are displayed for fast changing content compared to video frames of slow changing content, to decrease the speed of browsing when fast changing content is displayed.

14. A method, implemented on a display system having at least a display device, for previewing a video file by selecting video frames to be displayed for a preview, comprising:

calculating, at the display system, differences between histograms associated with subsequent video frames of the video file; and

comparing, at the display system, at least one of the differences or a cumulative difference value representing a sum of the differences to a predefined redundancy threshold, wherein

when the at least one of said differences in the histograms of particular video frames or said cumulative difference value exceeds the predefined redundancy threshold, selecting, at the display system, the particular video frames for the preview, and

the predefined redundancy threshold is a time-dependent variable that is adjusted depending on histogram differences of subsequent video frames, the method further comprising,

defining, at the display system, an initial distance between the selected video frame of said selecting and a new video frame to be selected for the preview, the initial distance being an integer value greater than one;

when the difference between the selected video frame and the new video frame falls below the predefined redundancy threshold, multiplying, at the display system, said distance with a constant integer factor and defining a result of said multiplying as a new distance between the selected video frame and a new video frame to be selected for the preview;

when the difference between the selected video frame and the new video frame does not fall below the predefined redundancy threshold, dividing, at the display system, said distance by said constant integer factor and defining a result of said dividing rounded down to a nearest integer as a new distance between the selected video frame and a new video frame to be selected for the preview;

iterating, at the display system, at least one of said multiplying or dividing until a minimum for said distance is found such that said difference is a smallest difference exceeding said redundancy threshold among all differences compared with said predefined redundancy threshold for a given initial distance;

when said initial distance is an integer value greater than one, decrementing, at the display system, said initial value by one; and

repeating, at the display system, at least one of said multiplying, dividing, or iterating until a global minimum for said distance is found such that said difference is the smallest difference exceeding said redundancy threshold among all differences compared with said predefined redundancy threshold for all chosen initial distances.