Fast video playback with automatic content based variable speed

• US 6,760,536 B1
• Filed: 05/16/2000
• Issued: 07/06/2004
• Est. Priority Date: 05/16/2000
• Status: Expired due to Term

• Current Assignee: Acer, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Amir, Arnon, Petkovic, Dragutin, Ponceleon, Dulce Beatriz
• Primary Examiner(s): NGUYEN, HUY THANH
• Application Number:
  US09/572,136
• Time in Patent Office: 1,512 Days
• Field of Search: 386/46, 386/68, 386/70, 386/71, 386/111, 386/112, 386/109, 348/700
• US Class Current: 386/344
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Browsing of digital video data is performed using a fast forward or fast reverse play mode. The digital video is analyzed and processed to produce a content-based variable-rate video playback sequence for fast browsing. To create the playback sequence, each shot in a video is sped-up at a relatively slow rate at the beginning of the shot by selecting many frames and then the speedup rate is increased as the shot progresses by selecting progressively fewer frames. This method and apparatus of variable-rate frame selection can be used to add and index to a video, play an original video in fast forward/backward mode or to create a new video—a fast forward playback video summary.

Claims

1. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots comprising the steps:

determining a first frame of a first video shot;

for each frame in said first video shot, determining a frame selection rate dependent on a frame'"'"'s distance from said first frame, and

selecting a plurality of frames from said first video shot according to said frame selection rate.

2. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 1, wherein said frame selection rate decreases with distance from said first frame of said first video shot.

3. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 1, wherein said selection rate is substantially 100% at said first frame and decreases linearly in proportion to said frame'"'"'s distance from said first frame.

4. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 3, wherein said frame selection rate stops decreasing upon reaching a minimum selection rate.

5. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 1, further comprising the step:

combining said plurality of frames to produce a fast mode digital video playback sequence.

6. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 5, wherein said fast mode is fast forward.

7. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 5, wherein said fast mode is fast reverse.

8. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 5, wherein said playback sequence is recorded on one of film, tape, computer disk, laser disk, DVD, CD, or semiconductor-based memory.

9. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 5, wherein said playback sequence is stored in one of MPEG, MPEG2, MPEG4, DV (Sony), QuickTime, RealVideo, AVI, H.263 or H.261 format.

10. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 5, wherein said playback sequence is displayed on one of TV, CATV, LCD, CRT, or electronic display.

11. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 1, further comprising the step:

creating at least one index of said plurality of frames, said at least one index comprising a corresponding entry for each frame in said plurality of frames.

12. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 11, wherein said at least one index is stored in one of optical, magnetic, or semiconductor storage medium.

13. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 11, wherein each frame of said plurality of frames is encoded with its said corresponding entry.

14. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 1, which further comprises the step:

determining a set of frames at an end boundary of said first shot.

15. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 14, wherein said frame selection rate is less than substantially 100% prior to said set of frames.

16. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 15, wherein said frame selection rate increases linearly from a first frame of said set of frames to a frame selection rate of substantially 100% at a final frame of said set of frames.

17. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 16, further comprising the step:

combining said plurality of frames to produce a fast mode digital video playback sequence.

18. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 17, wherein said fast mode is fast forward.

19. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 17, wherein said fast mode is fast reverse.

20. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 16, further comprising the step:

creating at least one index of said plurality of frames, said at least one index comprising a corresponding entry for each frame in said plurality of frames.

21. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 20, wherein said at least one index is stored in one of optical, magnetic, or semiconductor storage medium.

22. A method of selecting frames from a sequence of digitally encoded frames containing one or more video shots, as per claim 20, wherein each frame of said plurality of frames is encoded with its said corresponding entry.

23. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence comprising the steps:

identifying a first frame of a first video shot, said first video shot comprising a first sequence of frames;

determining a first frame selection rate for each frame in said first sequence of frames, wherein said first frame selection rate varies over a portion of said first video shot according to a frame'"'"'s distance from said first frame of said first video shot;

identifying a first frame of a second video shot, said second video shot comprising a second sequence of frames;

determining a second frame selection rate for each frame in said second sequence of frames, wherein said second frame selection rate varies over a portion of said second video shot according to a frame'"'"'s distance from said first frame of said second video shot;

selecting a first plurality of frames from said first sequence of frames according to said first frame selection rate;

selecting a second plurality of frames from said second sequence of frames according to said second selection rate;

combining said first and second pluralities of frames to produce a fast mode digital play back sequence, and

performing one of recording, indexing, displaying or storing said play back sequence.

24. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein

said first frame selection rate is substantially 100% at said first frame of said first video shot and decreases linearly until a first specified minimum frame selection rate is reached, and

said second frame selection rate is substantially 100% at said first frame of said second video shot and decreases linearly until a second specified minimum frame selection rate is reached.

25. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 24, wherein said first specified minimum frame selection rate is smaller than said second specified minimum selection rate.

26. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said first video shot has accompanying audio data and said second video shot does not.

27. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said first video shot relates to advertising and said second video shot relates to programming.

28. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said fast mode is fast forward.

29. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said fast mode is fast reverse.

30. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said recording is performed on one of film, tape, computer disk, laser disk, DVD, CD, or semiconductor-based memory.

31. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said storing is performed in one of MPEG, MPEG2, MPEG4, DV (Sony), QuickTime, RealVideo, AVI, H.263 or H.261 format.

32. A method for producing a fast mode digital video playback sequence from a multi-shot video sequence, as per claim 23, wherein said displaying is performed on one of TV, CATV, LCD, CRT, or electronic display.

33. A system for generating a fast mode digital video playback sequence comprising:

a shot analyzer which identifies a first frame of a video shot;

a frame counter which calculates for each frame in said video shot a corresponding distance from said first frame;

a frame-rate selector which determines a frame selection rate for each frame, based on said corresponding distance, and

a frame selector which selects a plurality of frames according to said frame selection rate.

34. A system for generating a fast mode digital video playback sequence, as per claim 33, further comprising a graphical display which displays said plurality of frames.

35. A system for generating a fast mode digital video playback sequence, as per claim 34, wherein said graphical display is one of a TV, CATV, LCD, CRT, or electronic display.

36. A system for generating a fast mode digital video playback sequence, as per claim 33, wherein said playback sequence is recorded on one of film, tape, computer disk, laser disk, DVD, CD, or semiconductor-based memory.

37. A system for generating a fast mode digital video playback sequence, as per claim 36, wherein said plurality of frames is stored in one of MPEG, MPEG2, MPEG4, DV (Sony), QuickTime, RealVideo, AVI, H.263 or H.261 format.

38. A system for generating a fast mode digital video playback sequence, as per claim 33, further comprising a frame indexer which constructs an index of said plurality of frames.

39. A system for generating a fast mode digital video playback sequence, as per claim 38, wherein said index is encoded within said plurality of frames.

40. A system for generating a fast mode digital video playback sequence, as per claim 33, wherein said fast mode is one of fast forward or fast reverse.

41. A system for generating a fast mode digital video playback sequence, as per claim 33, wherein said system is implemented locally or remotely on one or more computer-based systems.

42. A system for generating a fast mode digital video playback sequence, as per claim 33, wherein said system is implemented across networks comprising any of LANs, WANs, cellular, Internet or Web-based networks.

43. A system for generating fast mode digital video playback sequence, as per claim 33, wherein said system is implemented in a set top box, a television, a video recorder, or a video card for a computer.

44. A system for generating fast mode digital video playback sequence, as per claim 33, wherein said system is implemented in video camera, a camcorder, or a video editing device.

45. A system for generating fast mode digital video playback sequence, as per claim 33, wherein said system is implemented in hardware, firmware, or programmable gate array (PGA).

46. An article of manufacture comprising a computer user medium having computer readable program code embodied therein which implements constructing a fast mode digital video playback sequence, said constructing comprising the steps:

determining a first frame of a first video shot;

for each frame in said first video shot, determining a frame selection rate dependent on a frame'"'"'s distance from said first frame, and

selecting a plurality of frames from said first video shot according to said frame selection rate.