Video capture system producing a video summary

• US 8,446,490 B2
• Filed: 05/25/2010
• Issued: 05/21/2013
• Est. Priority Date: 05/25/2010
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Deever, Aaron T.
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): DAGNEW, MEKONNEN D
• Application Number:
  US12/786,501
• Publication Number:
  US 20110292245A1
• Time in Patent Office: 1,092 Days
• Field of Search: 348/231.2, 386/290, 386/95, 375/240.25
• US Class Current: 348/231.2
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

A digital video capture system for capturing a digital video sequence and for producing a corresponding video summary, comprising: an image sensor; an optical system; a processor, a softcopy display and a means for providing a user with an option to view either a stored digital video sequence or a video summary on the softcopy display. The processor performs the steps of: capturing a digital video sequence; automatically analyzing the digital video sequence at the time the digital video sequence is captured to determine one or more feature values; compressing the digital video sequence; storing the compressed digital video sequence; automatically analyzing the feature values to identify one or more key video snippets, assembling the key video snippets to form a video summary; and storing a representation of the video summary.

Claims

1. A digital video capture system for capturing a digital video sequence and for producing a corresponding video summary, comprising:

an image sensor for capturing a video frame;

an optical system for imaging a scene onto the image sensor;

a digital camera that captures a digital video sequence including a plurality of video frames;

a processor that automatically analyzes at least one of the video frames in the digital video sequence at the time the digital video sequence is captured to determine one or more feature values;

the processor compresses the digital video sequence, wherein the analyzed video frames are compressed only after the corresponding feature values have been determined;

stores the compressed digital video sequence in a processor-accessible memory;

the stored compressed digital video sequence automatically gets analyzed for the feature values, without decompressing the stored compressed digital video sequence, to identify one or more key video snippets, each key video snippet corresponding to a set of video frames from the digital video sequence;

assembles the key video snippets to form a video summary; and

stores a representation of the video summary in a processor-accessible memory;

a softcopy display to view a digital video sequence; and

provides a user with an option to view either the stored digital video sequence or the stored representation of the video summary on the softcopy display.

2. The system of claim 1 wherein the digital video capture system automatically analyzes at least one of the audio samples in the digital video sequence at the time the digital video sequence is captured to determine one or more feature values.

3. The system of claim 1 wherein the determined feature values are stored in the processor-accessible memory as metadata associated with the stored compressed digital video sequence.

4. The system of claim 1 wherein the determined feature values are stored in the processor-accessible memory as a separate file associated with the stored compressed digital video sequence.

5. The system of claim 1 wherein the processor automatically analyzes the feature values and user input, without decompressing the stored compressed digital video sequence, identifies one or more key video snippets, each key video snippet corresponding to a set of video frames from the digital video sequence.

6. The system of claim 1 wherein the processor further stores a representation of the video summary in a processor-accessible memory at least part of the stored compressed digital video sequence to extract video frames to the video summary, and compresses the extracted video frames to form a compressed video summary, and the compressed video summary in a processor-accessible memory.

7. The system of claim 6 wherein the processor further decompresses at least part of the stored compressed digital video sequence to extract audio samples that corresponds to the video summary, compresses the extracted audio samples and includes the compressed extracted audio samples in the compressed video summary.

8. The system of claim 6 wherein the extracted video frames are re-sampled to produce extracted video frames at a different spatial resolution.

9. The system of claim 6 wherein the extracted video frames are compressed at a more aggressive compression setting than the stored compressed digital video sequence.

10. The system of claim 1 wherein the stored representation of the video summary is a digital video file stored in a format adapted to be played using a standard video player.

11. The system of claim 1 wherein the processor further stores a representation of the video summary in a processor-accessible memory and includes metadata providing an indication of the video frames in the digital video sequence that corresponds to the video summary, and wherein the metadata is stored in association with the stored compressed digital video sequence.

12. The digital video capture system of claim 1 wherein the image sensor, the optical system, the processor and the softcopy display are all components of a digital video camera, and wherein a user interface control on the digital video camera is used to provide the user with the option to view either the stored compressed digital video sequence or the stored video summary on the softcopy display of the digital video camera.

13. The digital video capture system of claim 1 wherein the image sensor, the optical system, the processor and the softcopy display are all components of a digital video camera which can be connected to an external softcopy display, and wherein a user interface control on the digital video camera is used to provide the user with the option to view either the stored compressed digital video sequence or the stored video summary on the external softcopy display.

14. The digital video capture system of claim 1 wherein the image sensor, the optical system, the processor and the softcopy display are all components of a digital video camera, and wherein the softcopy display is a component of an independent video viewing system capable of accessing the stored digital video sequence and the stored video summary, and wherein a user interface control on the digital video camera is used to provide the user with the option to view either the stored compressed digital video sequence or the stored video summary on the softcopy display.

15. The system of claim 1 further provides a user with an option to request a video summary according to user settings.

16. The system of claim 15 wherein the user settings include the length of the video summary.

17. The system of claim 15 wherein the user settings include the minimum length of a key video snippet in the video summary.

18. The system of claim 1 wherein the assembled video summary includes video transition effects that are applied between the key video snippets.

19. The system of claim 18 wherein the results of the video transition effects are computed at the time of display.

20. The system of claim 18 wherein the video transition effects are automatically selected from a plurality of available transition effects by analyzing the key video snippets.

21. The system of claim 1 wherein the assembled video summary includes audio transition effects that are applied between the key video snippets.

22. The system of claim 21 wherein the results of the audio transition effects are computed at the time of display.

23. A digital video capture system for capturing a digital video sequence and for producing a corresponding video summary, comprising:

an image sensor for capturing a video frame;

an optical system for imaging a scene onto the image sensor;

a digital camera that captures a digital video sequence including a plurality of video frames;

automatically analyzes at least one of the video frames in the digital video sequence at the time the digital video sequence is captured to determine one or more feature values;

the digital video sequence, wherein the analyzed video frames are compressed only after the corresponding feature values have been determined;

stores the compressed digital video sequence in a processor-accessible memory;

after the compressed digital video sequence automatically analyzes the feature values, along with information contained in the stored compressed digital video sequence, to identify one or more key video snippets, each key video snippet corresponding to a set of video frames from the digital video sequence;

assembling the key video snippets to form a video summary; and

storing a representation of the video summary in a processor-accessible memory;

a softcopy display to view a video sequence; and

provides a user with an option to view either the stored compressed digital video sequence or the stored representation of the video summary on the softcopy display;

a means for providing a user with an option to view either the stored compressed digital video sequence or the stored representation of the video summary on the softcopy display.

24. The system of claim 23 wherein a processor is used to automatically analyze the feature values along with audio information contained in the stored compressed digital video sequence, to identify one or more key video snippets, each key video snippet corresponding to a set of video frames from the digital video sequence.

25. The system of claim 23 wherein a processor is used to automatically analyzes the feature values along with video information contained in the stored compressed digital video sequence, to identify one or more key video snippets, each key video snippet corresponding to a set of video frames from the digital video sequence.