Computer user interface for interacting with video cliplets generated from digital video

• US 7,383,508 B2
• Filed: 06/19/2002
• Issued: 06/03/2008
• Est. Priority Date: 06/19/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Toyama, Kentaro, Drucker, Steven M.
• Primary Examiner(s): Cabeca; John
• Assistant Examiner(s): Pillai; Namitha
• Application Number:
  US10/176,827
• Publication Number:
  US 20030234805A1
• Time in Patent Office: 2,176 Days
• Field of Search: 715/721, 715/719, 715/716, 715/720, 715/723, 715/725
• US Class Current: 715/723
• CPC Class Codes: G06F 16/743 a collection of video files or sequences
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4786 e-mailing message switching systems, e.g. electronic mail systems H04L51/00
• CPC Class Codes: H04N 21/8173 End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 5/147 : Scene change detection
• CPC Class Codes: H04N 5/45 : Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

The present invention includes a computer user interface system and a method for displaying short segments of video each representing a single theme or event (called video cliplets or “cliplets”). The computer user interface displays the cliplets and information and properties of each of the cliplets. Through the computer user interface of the invention, a user can quickly and easily determine the contents and interesting portions of the video. The cliplet user interface method includes displaying a plurality of cliplets in a two-dimensional arrangement along with cliplet information. The cliplet user interface system includes a plurality of keyframe windows for displaying cliplets and a keyframe within each keyframe window representing a cliplet therein. Moreover, the cliplet user interface system includes a cliplet property presentation area for displaying a cliplet interest rating. The cliplet interest rating indicates the amount of interesting content within the cliplet for a particular cliplet property.

Claims

1. In a computer system having a user interface including a display device and an input device, a method for displaying short segments of video called video cliplets generated from a digital source video, comprising:

extracting a first set of video cliplets from the digital source video by determining sub-shot boundaries to define a first set of editing points in the source video every N seconds without reference to any type of data, wherein the first set of video cliplets are extracted based on the first set of editing points such that one editing point is a starting point of a video cliplet and another editing point is an ending point of the video cliplet;

displaying the first set of video cliplets on the display device in a two-dimensional arrangement with each of the first set of video cliplets displayed as a single, independent entity without displaying piles or stacks of clipets;

extracting a second set of video cliplets by determining sub-shot boundaries using camera zoom criteria to define a second set of editing points in the digital source video such that a first editing point immediately following a first camera zoom in the source video is a starting point of a first video cliplet and a second editing point immediately following a second camera zoom in the source video is an ending point of the first video cliplet;

determining that a user is unsatisfied with at least one of the starting point and the ending point of the first video cliplet;

merging the first video cliplet with a second video cliplet, the first and the second video cliplets being temporal neighbors, to generate a third video cliplet that is a duration of the first and second video cliplets combined;

displaying a keyframe of a cliplet of the first set and the second set of cliplets in a keyframe window, such that the keyframe is a still frame of the cliplet that represents the cliplet;

previewing the cliplet in the keyframe window, as directed by the user, such that the keyframe is replaced by the cliplet playing in the keyframe window and at the conclusion of the cliplet playing the keyframe returns to the keyframe window; and

displaying a cliplet property for each of the plurality of video cliplets to provide the user with information about a corresponding cliplet.

2. The method as set forth in claim 1, wherein each of the first set and second set of video cliplets is displayed in its own keyframe window.

3. The method as set forth in claim 2, further comprising displaying interest ratings in the keyframe window,

4. The method as set forth in claim 2, wherein the keyframe window is a thumbnail.

5. The method as set forth in claim 1, wherein each of the first set and second set of video cliplets is represented by a keyframe.

6. The method as set forth in claim 1, further comprising representing each of the first set and second set of video cliplets by a plurality of keyframes.

7. The method as set forth in claim 6, further comprising allowing the user to use an input device to rotate through the plurality of keyframes to select a desired keyframe to represent an associated video cliplet.

8. The method as set forth in claim 6, further comprising displaying simultaneously at least two of the plurality of keyframes from a same cliplet.

9. The method as set forth in claim 1, wherein the cliplet property is a cliplet interest rating that is based on any available cliplet information.

10. The method as set forth in claim 1, further comprising organizing the first set and second set of cliplets into visually-separate groups based on at least one of:

(a) the cliplet property;

(b) any other form of metadata available for the first set and second set of cliplets.

11. The method as set forth in claim 1, further comprising organizing the first set and second set of cliplets into visually-separate rows based on at least one of:

(a) the cliplet property;

(b) any other form of metadata available for the first set and second set of cliplets.

12. The method as set forth in claim 1, further comprising organizing the first set and second set of cliplets into visually-separate columns based on at least one of:

(a) the cliplet property;

(b) any other form of metadata available for the first set and second set of cliplets.

13. A method in a computer system for displaying on a display device video cliplets generated from a source video, comprising:

generating the video cliplets such that at least some of the video cliplets overlap in time and any two different video cliplets can contain some identical video frames;

determining sub-shot boundaries using camera zoom criteria to define editing points in the source video such that a first editing point immediately following a first camera zoom in the source video is a starting point of a first video cliplet and a second editing point immediately following a second camera zoom in the source video is an ending point of the first video cliplet;

determining that the user is unsatisfied with at least one of the starting point and the ending point of the first video cliplet;

*

merging the first video cliplet with a second video cliplet, the first and the second video cliplets being temporal neighbors, to generate a third video cliplet that is a duration of the first and second video cliplets combined;

dividing the display area on the display device into a plurality of windows;

displaying each of the video cliplets in each of the plurality of windows as a single, independent entity without displaying piles or stacks of cliplets; and

displaying a video cliplet rating for each of the video cliplets.

14. The method as set forth in claim 13, further comprising arranging the plurality of windows in a two-dimensional arrangement.

15. The method as set forth in claim 13, further comprising displaying the video cliplet rating as at least one of the following:

(a) a graphical representation;

(b) a textual representation.

16. The method as set forth in claim 15, wherein the graphical representation is interest rating bars.

17. The method as set forth in claim 13, further comprising allowing a user to use an input device to scroll the display area and reveal additional windows containing cliplets.

18. The method as set forth in claim 13, further comprising providing drag and drop functionality such that at least one of the video cliplets can be dragged and dropped into an application for processing.

19. The method as set forth in claim 13, further comprising allowing a user to use an input device to merge at least two of the video cliplets such that a first one of the video cliplets merges with a second one of the video cliplets to create a new video cliplet.

20. The method as set forth in claim 19, wherein the first and second ones of the video cliplets are temporal neighbors.