Visual summary for scanning forwards and backwards in video content

• US 7,333,712 B2
• Filed: 02/14/2002
• Issued: 02/19/2008
• Est. Priority Date: 02/14/2002
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Jeannin, Sylvie, Zimmerman, John
• Primary Examiner(s): Tran; Thai Q.
• Assistant Examiner(s): Chowdhury; Nigar
• Application Number:
  US10/076,183
• Publication Number:
  US 20030152363A1
• Time in Patent Office: 2,196 Days
• Field of Search: 386/68, 386/69, 386/70, 386/7, 386/6, 386/8, 348/700, 348/701, 348/702
• US Class Current: 386/343
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/328 on a tape [TTOC]
• CPC Class Codes: G11B 27/329 on a disc [VTOC]
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

A method and system for providing the creation of a visual summary of a video source during fast forward/rewind of the video can be created by using extracting frames either automatically or manually to create an initial visual summary. A series of weights may be assigned to the extracted frames, which are then filtered according to the relative weights to create a modified visual summary. The keyframe display rate is then adjusted according tot he fast forward/rewind speed, which can be either a standard speed or user selected speed, so as to display the keyframes while the video source is being fast forwarded/rewound. The keyframes may be substituted by selected images and audio, so that an advertiser can substitute an image of the product and a brief audio summary while the user is fast forwarding/rewinding past the commercial message.

Claims

1. A method for the automatic creation of a visual summary of video content of a video source, comprising the steps of:

(a) automatic extraction of a plurality of keyframes representing scenes from a video source according to predetermined criteria to produce an initial visual summary;

(b) assigning weights to a particular group of keyframes extracted in step (a) representing a particular scene of the video source in the initial visual summary;

(c) refining the initial visual summary into a modified visual summary by filtering the keyframes having a lower weight assigned in step (b) relative to higher-weighted keyframes from the particular group of keyframes from the particular scene of the video source; and

(d) adjusting a keyframe display rate of the modified visual summary to correspond with a fast forward/rewind speed of the video source so that the keyframes can be displayed during fast forward/rewind of the video source.

2. The method according to claim 1, wherein assignment of weights in step (b) includes assigning the weights based on a relative time each of the keyframes in the particular group represents of the particular scene from the video source.

3. The method according to claim 2, wherein the keyframes in the particular group that represent less than a predetermined threshold of time in the particular scene of the video source are filtered.

4. The method according to claim 1, wherein the keyframe display rate is adjusted to correspond with a user-selected fast forward/rewind speed of the video source.

5. The method according to claim 1, wherein the adjusting of the keyframe display rate in step (d) includes providing audio for the modified visual summary during fast forward/rewind.

6. The method according to claim 1, wherein the adjusting of the keyframe display includes replacing individual keyframes by short sets of frames that capture movement in the particular scene.

7. The method according to claim 1, wherein the weighting of the keyframes is based on user-specified interests.

8. The method according to claim 7, wherein the user-specified interests include images of specific actors in the video source.

9. The method according to claim 7, wherein the user-specified interests include movement of actors in the video source.

10. A method for creation of a visual summary of video content of a video source comprising the steps of:

(a) designating a plurality of frames from a video source as keyframes;

(b) adjusting a display rate of the keyframes designated in step (a) according to a fast forward/rewind speed of the video source so that the keyframes are displayed for a predetermined time during fast forward/rewind of the video source.

11. The method according to claim 10, further including (c) providing an audio portion for the keyframes designated in step (a).

12. The method according to claim 11, where the audio portion is a substitute audio portion customized to correspond to the designated keyframes.

13. The method according to claim 11, wherein step (c) includes providing a plurality of audio portions, wherein a particular audio portion is selected to match the display rate of the keyframes in step (b).

14. The method according to claim 10, where a portion of the designated keyframes in step (a) comprise an advertisement.

15. The method according to claim 14, wherein the substitute audio portion comprises one of:

(1) a description of the product advertised in the portion of the designated keyframes comprising the advertisement, and (2) a pronunciation of the name of the product advertised in the portion of the designated keyframes comprising an advertisement.

16. The method according to claim 10, wherein step (a) includes providing at least one user-created alternative keyframe to the visual summary that did not originate from the video source.

17. The method according to claim 16, wherein said at least one user-created alternative keyframe comprises an advertisement.

18. A system for the automatic creation of a visual summary of video content of a video source, comprising:

means for extraction of a plurality of keyframes representing scenes from a video source according to predetermined criteria to produce an initial visual summary;

means for assigning weights to a particular group of keyframes extracted by the means for extraction representing a particular scene of the video source in the initial visual summary;

means for refining the initial visual summary into a modified visual summary by filtering the keyframes having a lower weight relative to higher-weighted keyframes from the particular group of keyframes from the particular scene of the video source; and

means for adjusting a keyframe display rate of the modified visual summary to correspond with a fast forward/rewind speed of the video source so that the keyframes are displayed during fast forward/rewind.

19. The system according to claim 18, wherein the means for extraction comprises manual designation of keyframes representative of the video source.

20. The system according to claim 18, wherein the means for extraction comprises automatic extraction means.

21. The system according to claim 18, wherein the means for assigning assigns the weights to the particular group of keyframes according to a relative time span represented by each respective keyframe.

22. The system according to claim 21, wherein the means for adjusting the keyframe display rate adjusts the display rate according to a user-selected fast forward/rewind speed of the video source.