ASSOCIATING SOURCE INFORMATION WITH PHONETIC INDICES

• US 20100094630A1
• Filed: 10/10/2008
• Published: 04/15/2010
• Est. Priority Date: 10/10/2008
• Status: Active Grant

• Current Assignee: Arlington Technologies, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Nortel Networks Limited (Nortel Networks Corporation)
• Inventors: Yoakum, John H.
• Granted Patent: US 8,301,447 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

The present invention relates to creating a phonetic index of phonemes from an audio segment that includes speech content from multiple sources. The phonemes in the phonetic index are directly or indirectly associated with the corresponding source of the speech from which the phonemes were derived. By associating the phonemes with a corresponding source, the phonetic index of speech content from multiple sources may be searched based on phonetic content as well as the corresponding source.

Claims

1. A method comprising:

accessing an audio segment comprising speech content from a plurality of sources, wherein different speech segments of the speech content correspond to different ones of the plurality of sources;

parsing the speech content into a sequence of phonemes wherein each phoneme corresponds to a basic unit of speech of the speech content;

generating a phonetic index from the sequence of phonemes; and

associating a corresponding one of the plurality of sources with phonemes in the sequence of phonemes of the phonetic index such that a source associated with a given phoneme or group of phonemes in the sequence of phonemes of the phonetic index is ascertainable.

2. The method of claim 1 wherein source information identifying the corresponding ones of the plurality of sources with the phonemes in the sequence of phonemes is provided in the phonetic index.

3. The method of claim 2 wherein the phonetic index comprises synchronization information sufficient to align the phonemes in the sequence of phonemes with corresponding basic units of speech in the speech content of the audio segment.

4. The method of claim 3 wherein the synchronization information is a time reference that aligns the phonemes of the sequence of phonemes in time with the corresponding units of speech in the speech content of the audio segment.

5. The method of claim 2 wherein the phonetic index is maintained in a data entity that does not include the audio segment.

6. The method of claim 2 wherein the phonetic index is maintained in a media item that also includes the audio segment or a version of the audio segment.

7. The method of claim 6 wherein the phonemes in the sequence of phonemes and the associated source information is time aligned with the audio segment, such that the phonemes of the sequence of phonemes are aligned in time with the corresponding units of speech in the speech content of the audio segment.

8. The method of claim 2 wherein the source information identifies one of the plurality of sources for each of the phonemes in the sequence of phonemes of the phonetic index.

9. The method of claim 2 wherein different groups of phonemes of the phonetic index correspond to the different speech segments, and the source information identifies one of the plurality of sources for each group of the different groups of phonemes.

10. The method of claim 1 wherein source information identifying the corresponding ones of the plurality of sources with the phonemes in the sequence of phonemes is provided in a separate file or stream than the phonetic index.

11. The method of claim 1 further comprising receiving a media item containing the audio segment and receiving source indicia identifying a source from the plurality sources for each of the different speech segments of the speech content in the audio segment, wherein the source indicia is used to associate the corresponding one of the plurality of sources with phonemes in the sequence of phonemes of the phonetic index.

12. The method of claim 11 wherein the media item is a media stream.

13. The method of claim 1 wherein the audio segment is a telephony audio signal and the different speech segments correspond to speech from different telephony sources or telephony parties.

14. The method of claim 13 wherein the telephony audio signal is an output of an audio conference bridge and the different speech segments correspond to speech from the different telephony sources or the telephony parties who are connected by the audio conference bridge.

15. The method of claim 14 further comprising:

providing the audio conference bridge to support audio conferencing of the different telephony sources or the telephony parties; and

providing the audio segment as the output of the conference bridge.

16. The method of claim 1 further comprising:

providing at least one phonetic search criterion in association with a search query wherein the phonetic search criterion corresponds to phonetic content;

identifying a matching portion of the phonetic index that meets the at least one phonetic search criterion; and

taking an action in response to identifying the matching portion of the phonetic index.

17. The method of claim 16 further comprising providing source criteria identifying at least one source of the plurality of sources in association with the search query, and wherein identifying the matching portion of the phonetic index comprises identifying the matching portion of the phonetic index that meets the at least one phonetic search criterion and the source criteria in light of the association of the corresponding one of the plurality of sources with the phonemes in the sequence of phonemes, such that the matching portion of the phonetic index corresponds to a source identified by the source criteria.

18. The method of claim 16 wherein the action comprises providing a notification that identifies the matching portion of the phonetic index.

19. The method of claim 1 further comprising:

providing source criteria identifying at least one source of the plurality of sources;

identifying a matching portion of the phonetic index that meets the source criteria in light of the association of the corresponding one of the plurality of sources with the phonemes in the sequence of phonemes; and

taking an action in response to identifying the matching portion of the phonetic index.

20. A method comprising:

accessing a phonetic index of a sequence of phonemes and associated source information wherein;

each phoneme in the phonetic index corresponds to a basic unit of speech content of an audio segment comprising speech content from a plurality of sources, such that different speech segments of the speech content correspond to different ones of the plurality of sources; and

the source information associates a corresponding one of the plurality of sources with the phonemes in the sequence of phonemes of the phonetic index;

determining at least one phonetic search criterion in association with a search query wherein the phonetic search criterion corresponds to phonetic content;

identifying a matching portion of the phonetic index that meets the at least one phonetic search criterion; and

taking an action in response to identifying the matching portion of the phonetic index.

21. The method of claim 20 further comprising determining source criteria identifying at least one source of the plurality of sources in association with the search query, and wherein identifying the matching portion of the phonetic index comprises identifying the matching portion of the phonetic index that meets the at least one phonetic search criterion and the source criteria in light of the source information, such that the matching portion of the phonetic index corresponds to a source identified by the source criteria.

22. The method of claim 20 wherein the action comprises providing a notification that identifies the matching portion of the phonetic index.

23. The method of claim 20 wherein the phonetic search criterion comprises a string of phonemes.

24. The method of claim 20 further comprising receiving a search query comprising at least one keyword, and wherein determining the at least one phonetic search criterion comprises translating the at least one keyword into a string of phonemes that is phonetically equivalent to the at least one keyword to provide the at least one phonetic search criterion.

25. The method of claim 20 wherein the audio segment is a telephony audio signal of a conference call, and the different speech segments correspond to speech from different telephony sources or telephony parties associated with the conference call.

26. The method of claim 20 wherein the audio segment is a telephony audio signal and the different speech segments correspond to speech from different telephony sources or telephony parties.

27. The method of claim 26 wherein the telephony audio signal is an output of an audio conference bridge and the different speech segments correspond to speech from the different telephony sources or the telephony parties who are connected by the audio conference bridge.

28. The method of claim 26 wherein taking the action comprises providing an alert to at least one of the different telephony sources or the telephony parties.

29. The method of claim 26 wherein taking the action comprises providing an alert to a person or entity other than the different telephony sources or the telephony parties.

30. The method of claim 20 wherein taking the action comprises identifying a portion of the audio content that corresponds to the matching portion of the phonetic index.

31. A system comprising:

at least one communication interface; and

a control system associated with the at least one communication interface and adapted to;

access an audio segment comprising speech content from a plurality of sources wherein different speech segments of the speech content correspond to different ones of the plurality of sources;

parse the speech content into a sequence of phonemes wherein each phoneme corresponds to a basic unit of speech of the speech content;

generate a phonetic index from the sequence of phonemes; and

associate a corresponding one of the plurality of sources with phonemes in the sequence of phonemes of the phonetic index such that a source associated with a given phoneme or group of phonemes in the sequence of phonemes of the phonetic index is ascertainable.