Methods and systems for real-time storyboarding with a web page and graphical user interface for automatic video parsing and browsing

• US 20040054964A1
• Filed: 07/22/2003
• Published: 03/18/2004
• Est. Priority Date: 03/18/1999
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Harrington, Steven J., Yu, Hong Heather, Campanelli, Michael R., Bozdagi, Gozde, Bryll, Robert
• Granted Patent: US 7,313,762 B2
• Time in Patent Office: Days
• US Class Current: 715/500.1
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/8453 by locking or enabling a set of features, e.g. optional functionalities in an executable program
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 7/0885 for the transmission of subtitles
• CPC Class Codes: H04N 7/0887 for the transmission of programme or channel identifying signals
• CPC Class Codes: H04N 7/165 Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

Systems and methods to enable real-time and near real-time storyboarding on the World Wide Web in addition to a graphical user interface for video parsing and browsing the of the storyboard. Specifically, storyboarding can be accomplished on the World Wide Web by parsing an input video into representative or key frames. These frames then can be posted to a web document, or the like, for subsequent viewing by a user. This allows a video to be distilled down to the essential frames thus eliminating storage and bandwidth problems as well as eliminating the need for a user to view the entirety of the video. Furthermore, the graphical user interface allows a user to visually interact with an input video signal to determine the key or representative frames, or to retrieve video segments associated with already determined key frames. Furthermore, the interface allows manipulation of these frames including, but not limited to, playing of the entire segment represented by that key or significant frame as well as actual determining of the cuts between significant segments.

Claims

1. A multimedia image data signal browsing and parsing graphical user interface comprising:

a key frame video display area displaying at least one representative frame located by a find key frame widget from an input multimedia image data signal;

a cuts video display area displaying cuts located by a find cuts widget in the input multimedia image data signal;

a plot video display area displaying plots generated by a plots widget and graphically illustrating statistics relating to the input multimedia image data signal; and

a play video display area displaying a portion of the input multimedia image data signal played by a play widget.

2. The user interface of claim 1 wherein the cuts video display area displays at least one discontinuous cut identified by the find cuts widget.

3. The user interface of claim 1 wherein the plot video display area displays plots resulting from parsing of the input multimedia image data signal, as well as determination of statistics relating to the input multimedia image data signal, by the plots widget.

4. The user interface of claim 1 wherein the key frame video display area displays at least one representative frame between at least two discontinuous cuts of the input multimedia data image having been identified by the find key frame widget.

5. The user interface of claim 4 wherein the at least one representative frame is identified based on detected command data in the input multimedia data image.

6. The user interface of claim 4 wherein the at least one representative frame is identified based on a frame difference determination.

7. The user interface of claim 1 wherein the key frame video display area displays at least one representative frame between an end of the input multimedia data image and the least one discontinuous cut of the input multimedia data image having been identified by the find key frame widget.

8. The user interface of claim 1 further comprising a frame select widget that can select a frame of the input multimedia data image.

9. The user interface of claim 8 further comprising a print command interface element that, when invoked, causes the frame to be printed.

10. The user interface of claim 1 further comprising a help command interface element that, when invoked, causes help information to be displayed.

11. The user interface of claim 1 further comprising an info command interface element that, when invoked, causes information to be displayed.

12. A method comprising:

identifying at least one representative frame of an input multimedia data image;

providing a video display area; and

displaying the at least one representative frame in the video display area.

13. The method of claim 12 further comprising:

identifying at least one cut in an input multimedia data image;

displaying the least one cut in the video display area.

14. The method of claim 12 further comprising:

identifying at least one cut in an input multimedia data image;

displaying the at least one cut in a cut display area.