Media browser using multimodal analysis

• US 6,366,296 B1
• Filed: 09/11/1998
• Issued: 04/02/2002
• Est. Priority Date: 09/11/1998
• Status: Expired due to Term

• Current Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation), Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation), Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Girgensohn, Andreas, Foote, Jonathan T., Boreczky, John S.
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): BAUTISTA, XIOMARA L
• Application Number:
  US09/151,285
• Time in Patent Office: 1,299 Days
• Field of Search: 345/302, 345/326-327, 345/339, 345/968, 707/3-4, 707/6, 707/104
• US Class Current: 715/719
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 715/968 interface for database querying and retrieval

Abstract

A media browser, graphical user interface and method for browsing a media file wherein a user selects at least one feature in a media file and is provided with information regarding the existence of the selected feature in the media file. Based on the information, the user can identify and playback portions of interest in a media file. Features in a media file, such as a speaker'"'"'s identity, applause, silence, motion, or video cuts, are preferably automatically time-wise evaluated in the media file using known methods. Metadata generated based on the time-wise feature evaluation are preferably mapped to confidence score values that represent a probability of a corresponding feature'"'"'s existence in the media file. Confidence score information is preferably presented graphically to a user as part of a graphical user interface, and is used to interactively browse the media file.

Claims

1. A media browser, comprising:

a feature selection module that inputs at least one media feature selected by a user;

a media presentation device that provides an aural and/or visual representation of a media file being browsed; and

a feature indicator that provides information related to a confidence score for at least one selected media feature, the confidence score being related to a presence of a corresponding selected media feature in the media file;

a confidence score mapping module that maps a metadata value representing a time-wise evaluation of a media feature in the media file to a corresponding confidence score,

wherein the selected media feature is not used to perform, and is not related, to a text search of the media file.

2. The media browser of claim 1, wherein the confidence score mapping module performs a non-linear mapping of the metadata value to the confidence score.

3. The media browser of claim 1, wherein mapping of the metadata value to the confidence score is adjusted based on a user-defined control value.

4. The media browser of claim 1, wherein the metadata value is automatically determined.

5. The media browser of claim 1, wherein the metadata value is input by a user.

6. The media browser of claim 1, wherein the feature indicator provides information related to the confidence score based on a value of another confidence score.

7. The media browser of claim 1, wherein the feature indicator combines at least two selected confidence scores and provides information related to the combined confidence scores.

8. The media browser of claim 1, wherein the media presentation device comprises user controls that control presentation of the aural and/or visual representation of the media file, and an effect of the user controls on the presentation is adjusted based on at least one confidence score.

9. A method of browsing a media file, comprising:

selecting a feature of a media file being browsed, the feature not being used to perform and not related to a text search of the media file;

providing information related to a confidence score for at least one selected feature relative to a segment of the media file, the confidence score related to the existence of a corresponding selected feature in the media file;

mapping metadata representing a time-wise evaluation of a feature in the media file to a corresponding confidence score; and

selecting a portion of the media file based on the information related to a confidence score for at least one selected feature.

10. The method of claim 9, wherein the mapping is a non-linear mapping.

11. The method of claim 9, further comprising adjusting the mapping based on a user-defined control value.

12. The method of claim 9, further comprising the step of automatically determining the metadata.

13. The method of claim 9, further comprising the step of receiving metadata input by a user.

14. The method of claim 9, wherein the step of providing information further comprises providing information related to a confidence score based on a value of another confidence score.

15. The method of claim 9, further comprising the steps of:

combining at least two selected confidence scores; and

providing information related to the combined confidence scores.

16. The method of claim 14, further comprising the steps of inputting a user control value for controlling a presentation of an aural and/or visual representation of the media file;

and

adjusting the effect of the user controls based on at least one confidence score.