Method and apparatus for indexing and retrieving audio-video data

• US 5,136,655 A
• Filed: 03/26/1990
• Issued: 08/04/1992
• Est. Priority Date: 03/26/1990
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Company (HP Inc.)
• Original Assignee: HP Inc.
• Inventors: Bronson, Barry S.
• Primary Examiner(s): Shaw, Dale M.
• Assistant Examiner(s): Doerrler, Michelle
• Application Number:
  US07/499,138
• Time in Patent Office: 862 Days
• Field of Search: 381/41-45, 364/513.5, 369/53, 369/83
• US Class Current: 704/270
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/107 of operating tapes
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A technique is disclosed for processing audio-video data in a storage medium where indexes are generated by automatically extracting and indexing words and pictures in the audio-video data with their corresponding timing or other location data. The indexes permit searching and selective retrieval of word and/or scene data in accordance with location data associated therewith.

Claims

1. A system for processing audio-video data with associated location data in a storage medium for retrieval, comprising:

means for extracting the audio-video data in conjunction with its associated location data to produce a first output signal;

voice recognition means connected to the first output signal for automatically extracting words therefrom to produce a second output signal;

means connected tot he second output signal for creating word data of the extracted words with their associated location data; and

means connected to the word data for creating a data index of the words and their associated location data.

2. The system of claim 1 further comprising:

computing-monitoring means connected to the data index for producing one or more words identified to be of interest; and

record-playback means connected to the computing-monitoring means for selectively retrieving the audio-video data corresponding to the location data associated with the identified words.

3. The system of claim 1 further comprising:

pattern recognition means connected to the first output signal for automatically extracting scenes therefrom to produce a third output;

means connected to the third output signal for creating scene data of the extracted scenes with their associated location data;

computing-monitoring means connected to the scene data for producing one or more scenes identified to be of interest;

record-playback means connected to the computer-monitoring means for selectively retrieving the audio-video data corresponding to the location data of the identified scenes; and

means connected to the scene data and word data for creating a combined index in the data index.

4. The system of clam 3 wherein:

the computing-monitoring means is connected to the data index to produce one or more combinations of word and scene identified t be of interest; and

the record-playback means is connected to the computing-monitoring means for selectively retrieving the audio-video data corresponding to the location data associated with the identified word and scene combination.

5. A method of processing stored audio-video data with associated location data for retrieval, comprising the steps of:

extracting from storage, the audio-video data in conjunction with its associated location data;

applying the audio-video data to a voice recognition subsystem to automatically extract words included in the audio-video data; and

creating an index of the extracted words and their respective location data.

6. The method of claim 5 further comprising the steps of:

searching the word index to identify words of interest; and

selectively retrieving from the stored audio-video data, the audio-video data corresponding to the location data associated with the identified words.

7. The method of claim 5 further comprising the steps of:

applying the audio-video data to a pattern recognition subsystem to automatically extract scene data from the audio-video data;

creating a storage for the scene data and their respective location data;

searching the stored scene data to identify one or more scenes of interest;

selectively retrieving from the stored audio-video data, scene data corresponding to the location data of the identified scene; and

creating a combined index of scenes and words with associated location data from the stored scene data and the word index.

8. The method of claim 7 further comprising the steps of:

searching the combined index to identify combinations of word an scene identified to be of interest; and

selectively retrieving the audio-video data corresponding to the location data associated with the identified word and scene combination.