Apparatus and method using speech recognition and scripts to capture author and playback synchronized audio and video

• US 6,336,093 B2
• Filed: 01/16/1998
• Issued: 01/01/2002
• Est. Priority Date: 01/16/1998
• Status: Expired due to Term

• Current Assignee: Avid Technology Incorporated
• Original Assignee: Avid Technology Incorporated
• Inventors: Fasciano, Peter
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US09/007,931
• Publication Number:
  US 20010047266A1
• Time in Patent Office: 1,446 Days
• Field of Search: 704/270, 704/275, 704/278, 345/328, 345/340, 345/348, 348/594
• US Class Current: 704/278
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Audio associated with a video program, such as an audio track or live or recorded commentary, may be analyzed to recognize or detect one or more predetermined sound patterns, such as words or sound effects. The recognized or detected sound patterns may be used to enhance video processing, by controlling video capture and/or delivery during editing, or to facilitate selection of clips or splice points during editing.

Claims

1. A computer system for editing a video program generated from a plurality of video sources, each video source having one or more video clips, each clip associated with a common audio comprising:

means for analyzing the audio to recognize one or more predetermined sound patterns;

means for retrieving clips from the plurality of video sources using the recognized sound patterns;

means for controlling delivery of the retrieved clips to a display during an editing process;

means for facilitating selection from among the retrieved clips during editing; and

means for placing the selected clips in a timeline.

2. The computer system of claim 1, further including means for controlling capture of portions of the video program using the recognized sound patterns.

3. The computer system of claim 2, wherein the means for controlling capture identifies predetermined sound patterns in the audio and further including means for initiating capture of a video clip upon identification of a first sound pattern.

4. The computer system of claim 3, wherein the means for controlling capture further includes means for terminating capture of the video clip upon identification of a second sound pattern.

5. The computer system of claim 1, wherein the audio is an audio track of the video program.

6. The computer system of claim 1, wherein the audio is commentary associated with the video program.

7. The computer system of claim 1, wherein the predetermined sound patterns are words.

8. The computer system of claim 1, wherein the predetermined sound patterns are sound effects.

9. The computer system of claim 1, wherein the means for analyzing uses a script to automatically match clips with portions of the script that they represent.

10. The computer system of claim 9, further comprising:

means for displaying the script on a computer user interface;

means for permitting an editor to select a portion of the script;

means for matching video segments having sound patterns matching the selected portion of the script;

means for displaying the matched video segment as option for selection by the editor.

11. The computer system of claim 1, wherein the means for analyzing identifies predetermined sound patterns in the audio and further including means for indicating possible splice points to a user.

12. The computer system of claim 1, wherein the audio is a live audio input from an individual providing commentary for a video segment, and the means for analyzing identifies desired words in the audio, and further including means for logging the video clip according to desired words identified in the audio.

13. The computer system of claim 1, further comprising means for receiving an audio track from a live audio input.

14. The computer system of claim 13, wherein the audio track is commentary used to log clips of the audio program.

15. The computer system of claim 1, further comprising means for determining a first a first location in a selected clip and a second location in a selected clip, said first and second locations determining the beginning of a selected clip and the end of the selected clip, respectively.

16. A computer system for digitally editing a video program based upon a script associated with the video program using one or more video sources, each of the video sources including one or more video clips and a common audio portion, the system comprising:

means for analyzing the audio portion to recognize one or more predetermined sound patterns;

means for selecting a range of the script;

means for retrieving one or more clips from the one or more video sources that contain sound patterns that match the selected range of the script;

means for displaying the one or more clips that contain sound patterns that match the selected range;

means for enabling the selection of at least one of the clips;

means for placing the selected clips in a timeline.

17. The computer system of claim 16, further comprising means for capturing portions of the video sources using the sound patterns in the selected range.

18. The computer system of claim 16, wherein the audio portions are audio tracks.

19. The computer system of claim 16, wherein the audio portions are commentary.

20. The computer system of claim 16, wherein the sound patterns are words.

21. The computer system of claim 16, wherein the sound patterns are sound effects.

22. The computer system of claim 16, wherein the means for analyzing identifies predetermined words in the audio portions.

23. The computer system of claim 16, further comprising means for indicating possible splice points to a user.

24. The computer system of claim 16, further comprising means for initiating capture of a video segment upon identification of a first sound pattern.

25. The computer system of claim 24, further comprising means for terminating capture of the video segment upon identification of a second sound pattern.

26. The computer system of claim 16, wherein the audio portions are live audio inputs from an individual providing commentary, and the means for analyzing identifies desired words in the audio portions and includes means for logging the video source according to desired words identified in the audio.

27. The computer system of claim 16, further comprising means for receiving an audio track from a live audio input.

28. The computer system of claim 27, wherein the audio track is commentary used to log clips of the audio program.

29. The computer system of claim 16, further comprising means for determining a first a first location in a selected clip and a second location in a selected clip, said first and second locations determining the beginning of a selected clip and the end of the selected clip, respectively.