Extracting a time-sequence of slides from video

• US 6,646,655 B1
• Filed: 03/09/1999
• Issued: 11/11/2003
• Est. Priority Date: 03/09/1999
• Status: Expired due to Term

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: WebEx Communications Incorporated (Cisco Systems, Inc.)
• Inventors: Brandt, Jonathan Worthen, Chen, Shenchang Eric
• Primary Examiner(s): Cabeca, John
• Assistant Examiner(s): Hailu, Tadesse
• Application Number:
  US09/265,488
• Time in Patent Office: 1,708 Days
• Field of Search: 345/328, 345/330, 345/327, 345/302, 345/333, 345/348, 345/349, 345/329, 345/723, 345/753, 345/726, 345/732, 345/738, 348/13-15, 348/231, 348/232, 348/220, 348/330, 348/526, 704/8
• US Class Current: 715/723
• CPC Class Codes: G09B 5/067 Combinations of audio and projected visual presentation, e.g. film, slides
• CPC Class Codes: H04N 5/06 Generation of synchronising signals
• CPC Class Codes: H04N 5/073 for mutually locking plural sources of synchronising signals, e.g. studios or relay stations
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

An apparatus and method for generating a slide from a video. A slide is automatically identified in video frames of a video input stream. A digitized representation of the slide is generated based on the video frames of the video input stream.

Claims

1. A method of generating a slide from a sequence of video frames, the method comprising:

selecting a still image within at least one frame from the sequence of video frames to be a first slide in a slide presentation, the image being substantially unchanged for a predetermined threshold value;

associating a value with the first slide that indicates a time at which the at least one frame appears in the sequence of video frames;

comparing the first slide against at least one source slide of a set of source slides of the slide presentation used to create a video; and

substituting the at least one source slide for the first slide.

2. The method of claim 1 wherein selecting the image within at least one frame of the video to be the first slide comprises selecting the at least one frame to be the first slide in response to user input.

3. The method of claim 1 wherein selecting the image within at least one frame of the video to be the first slide comprises processing frames of the video in a slide detector that automatically identifies a sequence of frames that each contain an image of the first slide, the sequence of frames including the at least one frame.

4. The method of claim 1 wherein selecting the image within at least one frame of the video to be the first slide comprises generating the first slide based on the image.

5. The method of claim 4 wherein generating the first slide based on the image within at least one frame of the video comprises combining images contained in a sequence of frames of the video, the sequence of frames including the at least one frame of the video.

6. The method of claim 1 wherein substituting the at least one source slide for the first slide comprises automatically selecting one of a plurality of different language versions of the at least one source slide according to a language preference specified by a user.

7. The method of claim 1 further comprising:

displaying a thumbnail representation of the first slide in a user interface of a computer system;

receiving user input in the computer system selecting the thumbnail representation of the first slide; and

displaying the video starting at a frame of the video that contains the image of the first slide in response to receiving the user input selecting the thumbnail representation of the first slide.

8. The method of claim 7 wherein displaying the video starting at the at least one frame of the video containing the image selected to be the first slide comprises displaying the video at the time indicated by the value associated with the first slide.

9. The method of claim 7 wherein displaying the video starting at the frame of the video that contains the image of the first slide comprises displaying the video at a frame of the video that is an initial frame in a sequence of frames that each include the image of the first slide, the sequence of frames including the at least one frame.

10. The method of claim 1 wherein the value includes a timecode component and a duration component that indicate a starting frame and an ending frame, respectively, of a segment of the video input that includes the image of the first slide.

11. The method of claim 10 wherein the timecode component and the duration component specify respective starting and ending frame numbers of the segment of the video input.

12. The method of claim 10 wherein the timecode component and the duration component specify respective starting and ending times of the segment of the video input relative to the start of the video input.

13. The method of claim 10 wherein the duration value indicates the ending frame by specifying a number of frames in the segment of the video input.

14. The method of claim 10 wherein the duration value indicates the ending frame by specifying an elapsed time relative to the starting frame of the segment of the video input.

15. The method of claim 1 further comprising:

presenting the video in a first display area; and

displaying the first slide in a second display area at a time indicated by the value associated with the first slide so that the first slide is displayed concurrently with the at least one frame.

16. The method of claim 15 wherein the first display area and the second display area are located on a single display.

17. A method comprising:

generating a sequence of slides of a slide presentation based on respective segments of a video, each segment including at least a threshold number of frames having a persistent image, the image being substantially unchanged for a predetermined threshold value, wherein each slide is generated based on said persistent image in each respective segment;

associating respective time values with the slides that indicate, for each of the slides, when a corresponding one of the segments appears in the video;

comparing each slide in the sequence of slides to a source slide of a set of source slides of the slide presentation used to create the video; and

substituting the source slide for the slide in the sequence of slides.

18. The method of claim 17 further comprising displaying the sequence of slides in synchronization with presentation of the video based on the time values.

19. A method of extracting slides from a video input, the method comprising:

inspecting frames of the video input to automatically identify sequences of the frames that contain respective still images, the still images being substantially unchanged for a predetermined threshold value;

generating respective slides that include the still images, the respective slides forming a set of output slides of a slide presentation;

comparing slides in the set of output slides to identify a slide that is a duplicate of another slide in the set of output slides;

comparing each slide in the set of output slides against slides in a source slide set of the slide presentation to identify a matching one of the slides in the source slide set, the source slide set used to generate a sequence of video frames in the video input that contain the still images included in the slides; and

substituting the matching source slide for the slide in the set of output slides.

20. The method of claim 19 further comprising:

removing the duplicate from the set of output slides.

21. The method of claim 20 wherein comparing slides to identify a slide that is a duplicate of another slide comprises comparing a first slide in the output set of slides with a second slide in the output set of slides to determine whether a difference between the first and second set of slides is less than a threshold, the second slide being a duplicate of the first slide if the difference is less than the threshold.

22. The method of claim 19 further comprising substituting the source slide for a first slide by selecting one of a plurality of different language versions of the source slide according to a previously entered user-preference value.

23. The method of claim 22 wherein substituting the source slide for a first slide comprises selecting one of a plurality of different descriptor format versions of the source slide according to a previously entered user-preference value.

24. The method of claim 23 wherein selecting one of a plurality of different descriptor format versions of the source slide comprises selecting a markup language version of the source slide.

25. The method of claim 23 wherein selecting one of a plurality of different descriptor format versions of the source slide comprises selecting a version of the source slide in a file format that can be used in a slide generation program of a computer system to render the source slide onto a display of the computer system.

26. The method of claim 19 wherein inspecting frames of the video input to automatically identify sequences of the frames that contain respective still images comprises identifying sequences of the frames for which camera motion effects are less than a predetermined threshold.

27. The method of claim 19 wherein inspecting frames of the video input to automatically identify sequences of the frames that contain respective still images comprises identifying sequences of the frames for which foreground changes in the frames are less than a predetermined threshold.

28. The method of claim 27 wherein identifying sequences of frames for which foreground changes in the frames are less than a predetermined threshold comprises:

generating measures of difference by subtracting frames in the sequences of frames from respective successive frames in the sequences of frames; and

determining if the measures of difference exceed the predetermined threshold.

29. The method of claim 27 further comprising:

automatically identifying frames of the video input that include a video inset; and

masking the video inset before identifying the sequences of the frames for which the foreground changes in the frames are less than the predetermined threshold.

30. The method of claim 29 wherein automatically identifying frames of the video input that include a video inset comprises evaluating pixel values along a border of an insert area within the frames of the video input to determine if changes in pixel intensity along the border exceed a predetermined threshold.

31. The method of claim 30 wherein automatically identifying frames of the video input that include a video inset comprises setting an inset detect attribute to indicate that a video inset is present for each frame of the video input for which the changes in pixel intensity along the border exceed the predetermined threshold.

32. The method of claim 30 wherein the border has a frame location and dimension that are specified by a user.

33. The method of claim 29 further comprising:

displaying a frame of the video input that contains a video inset on a display of a computer system; and

receiving user input specifying a border of the video inset, the border of the video inset being the border of the inset area along which changes in pixel intensity are evaluated.

34. The method of claim 19 wherein inspecting the frames of the video input to automatically identify sequences of the frames that contain respective still images comprises detecting frames within the sequences of frames that include a video inset overlaid on one of the still images, the method further comprising generating a video presentation based on images in the video inset that is different from the video input.

35. The method of claim 34 further comprising:

displaying the slides in a slide presentation window on a display of a computer system; and

displaying the video presentation based on images in the video inset in a video presentation window of the computer system concurrently with displaying the slides in the slide presentation window.

36. The method of claim 35 wherein displaying the slides in the slide presentation window and the video presentation in the video presentation window comprises displaying the slides and the video presentation according to respective times of appearance of subject matter of the slide and the video presentation in the video input.

37. An apparatus comprising:

a video receiving circuitry to receive a video input;

a slide detector to detect a sequence of frames in the video input remaining substantially unchanged for a predetermined threshold value;

a slide generator to generate a first slide of a slide presentation based on at least one frame from the sequence of frames of the video input and to associate a value with the first slide that indicates a time at which the at least one frame appears in the video input; and

a slide coalescing stage to compare slides in the set of slides to determine if one slide of the set of slides is a duplicate of another slide in the set of slides;

a slide substitution stage coupled to receive the first slide from the slide generator, the substitution stage being configured to substitute a source slide for the first slide, the source slide being a slide from the slide presentation used to create the video input.

38. The apparatus of claim 37 wherein the slide generator is configured to generate the first slide in response to user input.

39. The apparatus of claim 37 further comprising a slide detector to automatically identify a sequence of frames in the video input that each contain an image of the first slide and to signal the slide generator to generate the first slide based on the at least one frame, the sequence of frames including the at least one frame.

40. The apparatus of claim 39 wherein the slide generator includes a combining stage coupled to receive the sequence of frames identified by the slide detector and to combine the images contained in the sequence of frames to generate the first slide.

41. The apparatus of claim 40 wherein the combining stage is an averager that combines the images contained in the sequence of frames by averaging the constituent pixel values in the images.

42. The apparatus of claim 40 wherein the slide generator further includes an upsampling stage coupled to receive the sequence of frames identified by the slide detector and to upsample the images from the sequence of frames to higher resolution before the frames are received in the combining stage.

43. The apparatus of claim 39 further comprising an inset detector coupled to receive the video input and configured to automatically identify frames of the video input that include a video inset and to mask the video inset before outputting the frames to the slide detector.

44. The apparatus of claim 37 wherein the slide generator further includes a sharpening stage to amplify high frequency components of the first slide.

45. The apparatus of claim 37 further comprising a slide substitution stage coupled to receive the first slide from the slide generator, the substitution stage being configured to substitute a source slide for the first slide, the source slide being a slide used to create the video input.

46. The apparatus of claim 37 wherein the substitution stage is further configured to automatically select one a plurality of different language versions of the source slide according to a language, preference specified by a user.

47. The apparatus of claim 37 wherein the source slide is formatted in descriptive format that can be used by a computer application program to generate the image of the first slide.

48. The apparatus of claim 47 wherein the descriptive format is a markup language representation of the image of the first file.

49. The apparatus of claim 37 wherein the slide coalescing stage being configured to remove the duplicate from the set of slides.

50. The apparatus of claim 37 further comprising a slide coalescing stage coupled to receive the first slide from the slide generator and to determine if the first slide is a build slide that includes both an image of a second slide output by the slide generator and a build region containing additional information not present in the second slide.

51. The apparatus of claim 50 wherein the slide coalescing stage is configured to associate build information with the first slide, the build information being representative of the build region.

52. The apparatus of claim 51 further comprising:

a network communications device to transmit slides output from the slide coalescing stage to a slide presentation device via a communications network, the slides including the first slide and the second slide; and

a processing unit to determine that the build information is associated with the first slide and to select the build information to be transmitted to the slide presentation device instead of the first slide, the first slide being constructable;

within the slide presentation device, from the build information and the second slide.

53. The apparatus of claim 52 wherein the slide presentation device is a computer system.

54. The apparatus of claim 37 wherein the slide generator is implemented by a programmed processing unit.

55. An article of manufacture including one or more computer-readable media that embody a program of instructions for generating a slide from a video, wherein the program of instructions, when executed by a processing unit in a data processing system, causes the processing unit to:

select at least one image within one frame of the video to be a first slide of a slide presentation, the image being substantially unchanged for a predetermined threshold value;

associate a value with the first slide that indicates a time at which the at least one image within one frame appears in the video; and

compare the first slide against at least one source slide of a set of source slides of the slide presentation used to create the video; and

substitute the at least one source slide for the first slide.

56. An article of manufacture including one or more computer-readable media that embody a program of instructions for extracting slides form a video input, wherein the program of instructions, when executed by a processing unit in a data processing system, causes the processing unit to:

inspect frames of the video input to automatically identify sequences of the frames that contain respective still images, the images being substantially unchanged for a predetermined threshold value;

generate respective slides that include the still images, the respective slides forming a set of output slides of a slide presentation;

compare slides in the set of output slides to identify a slide that is a duplicate of another slide in the set of output slides;

compare each slide in the set of output slides against slides in a source slide set of the slide presentation to identify a matching one of the slides in the source slide set, the source slide set used to generate a sequence of video frames in the video input that contain the still images included in the slides; and

substitute the matching source slide for the slide in the set of output slides.

57. The article of claim 56 wherein the computer-readable media includes a propagated signal having the program of instructions encoded therein.

58. The article of claim 56 wherein the program of instructions, when executed by the processing unit in the data processing system, causes the processing unit further to remove the duplicate from the output set of slides.