Methods and apparatus to present a video program to a visually impaired person

• US 8,229,748 B2
• Filed: 04/14/2008
• Issued: 07/24/2012
• Est. Priority Date: 04/14/2008
• Status: Expired due to Fees

• Current Assignee: AT&T Knowledge Ventures, L.P. (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Chang, Hisao M., Schroeter, Horst
• Primary Examiner(s): Godbold, Douglas
• Assistant Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US12/102,310
• Publication Number:
  US 20090259473A1
• Time in Patent Office: 1,562 Days
• Field of Search: 704/260, 704/261, 704/270, 704/270.1, 704/275, 704/276, 704/277, 704/278, 704/258
• US Class Current: 704/260
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence

Abstract

Methods and apparatus to present a video program to a visually impaired person are disclosed. An example method comprises receiving a video stream and an associated audio stream of a video program, detecting a portion of the video program that is not readily consumable by a visually impaired person, obtaining text associated with the portion of the video program, converting the text to a second audio stream, and combining the second audio stream with the associated audio stream.

Claims

1. A method comprising:

detecting, with a processor, a text portion of a program from a program stream having a video stream and a first audio stream, the text portion of the program not being consumable by a blind person;

obtaining, with the processor, text associated with the text portion of the program;

converting the text to a second audio stream;

generating pronunciation and prosody annotations based on the second audio stream and a language type associated with the program; and

combining the second audio stream with the first audio stream to form a combined audio stream.

2. A method as defined in claim 1, wherein the text portion of the program is associated with a portion of the first audio stream that does not contain any spoken words.

3. A method as defined in claim 1, further comprising presenting the combined audio stream and the text portion of the program to the blind person.

4. A method as defined in claim 1, wherein obtaining the text associated with the text portion of the program comprises:

detecting that a first video frame of the video stream is substantially unchanged from a second video frame of the video stream; and

performing optical character recognition to extract the text from the first video frame, the text representing alphanumeric characters presented in the first and second video frames.

5. A method as defined in claim 1, wherein obtaining the text associated with the text portion of the program comprises:

detecting that the video stream contains scrolling text; and

performing optical character recognition to extract the text from the video stream.

6. A method as defined in claim 1, wherein obtaining the text associated with the text portion of the program comprises:

detecting a link associated with the video program; and

obtaining the text from a text server associated with the link, the text representing a text-based description of a corresponding portion of the video stream.

7. A method as defined in claim 6, wherein the text obtained from the text server comprises markup language data having a timestamp to synchronize the second audio stream to the first audio stream.

8. A method as defined in claim 6, wherein the link comprises at least one of an Internet protocol address or a uniform resource locator.

9. A method as defined in claim 1, wherein converting the text to the second audio stream comprises:

querying a text-to-speech database to determine whether first audio data corresponding to a word to be converted exists; and

when the first audio data does not exist;

performing text-to-speech synthesis to convert the word to second audio data; and

adding the second audio data to the second audio stream.

10. A method as defined in claim 9, further comprising adding the first audio data to the second audio stream when the first audio data exists.

11. A method as defined in claim 9, wherein performing the text-to-speech synthesis to convert the word to second audio data comprises sending the word to a network-based text-to-speech synthesis server to obtain the second audio data.

12. A method as defined in claim 1, wherein converting the text to the second audio stream comprises performing text-to-speech synthesis.

13. A method as defined in claim 12, wherein the text-to-speech synthesis is performed at a set-top box.

14. A method as defined in claim 12, wherein a first portion of the text-to-speech synthesis is performed at a set-top box, and a second portion of the text-to-speech synthesis is performed at a network-based text-to-speech synthesis server.

15. A method as defined in claim 12, wherein the text-to-speech synthesis is performed based on at least one of the language type or a speaker persona.

16. A method as defined in claim 1, wherein the first audio stream comprises a first audio signal associated with a front speaker and a second audio signal associated with a back speaker, and combining the second audio stream with the first audio stream comprises combining the first audio signal and the second audio stream.

17. A set-top box comprising:

a network interface to receive a video stream and an associated audio stream;

a detector to identify a portion of the video stream that is not consumable by a blind person;

a text locator to obtain text associated with the portion of the video stream;

a text-to-speech module to convert the text to a second audio stream;

a pronunciation and prosody analyzer to generate verbal annotations of the second audio stream based on a language type associated with the video stream; and

a mixer to combine the second audio stream with the associated audio stream.

18. A set-top box as defined in claim 17, wherein the detector comprises a non-spoken audio segment detector, and the portion of the video stream that is not consumable by the blind person is associated with a portion of the associated audio stream that does not contain any spoken words.

19. A set-top box as defined in claim 17, further comprising a media device interface to present the combined audio stream to the blind person.

20. A set-top box as defined in claim 17, wherein the detector comprises a text frame detector to identify a first video frame of the video stream that is substantially unchanged from a second video frame of the video stream, and the text locator comprises an optical character recognizer to extract the text from the first video frame, the text representing written characters presented in the first and second video frames.

21. A set-top box as defined in claim 17, wherein the detector comprises a text frame detector to detect that the video stream contains scrolling text, and the text locator comprises an optical character recognizer to extract the scrolling text.

22. A set-top box as defined in claim 17, wherein the text locator comprises:

a metadata handler to detect a link associated with the portion of the video stream; and

a text acquirer to request the text from a text server associated with the link, the text representing a text-based description of at least a portion of the video stream.

23. A set-top box as defined in claim 17, wherein the text-to-speech module comprises:

a text-to-speech database querier to determine whether first audio data corresponding to a word to be converted exists; and

a text-to-speech engine to perform text-to-speech synthesis to convert the word to second audio data when the first audio data does not exist, and to add the second audio data to the second audio stream.

24. A set-top box as defined in claim 23, wherein the text-to-speech engine is to add the first audio data to the second audio stream when the first audio data exists.

25. A set-top box as defined in claim 23, wherein the text-to-speech engine is to perform the text-to-speech synthesis by sending the word to a network-based text-to-speech synthesis server to obtain the second audio data.

26. A set-top box as defined in claim 17, wherein the text-to-speech module comprises:

an abbreviation analyzer to convert a first character string into a pronounceable second character string, wherein the pronunciation analyzer is to add the verbal annotations to the pronounceable second character string; and

a text-to-speech engine to convert the annotated pronounceable second character string to second audio data.

27. A set-top box as defined in claim 17, wherein the text-to-speech module is to convert the text to the second audio stream based on at least one of a language or a speaker persona.

28. A set-top box as defined in claim 17, wherein the associated audio stream comprises a first audio signal associated with a front speaker and a second audio signal associated with a back speaker, and the mixer is to combine the second audio stream with the associated audio stream by:

converting the second audio stream to a third audio signal; and

combining the first and third audio signals.

29. A tangible machine readable medium storing instructions which, when executed, cause a machine to perform a method comprising:

detecting, with a processor, a text portion of a program from a program stream having a video stream and a first audio stream, the text portion of the program not being consumable by a blind person;

obtaining, with the processor, text associated with the text portion of the program;

converting the text to a second audio stream;

generating pronunciation and prosody annotations based on the second audio stream and a language type associated with the program; and

combining the second audio stream with the first audio stream to form a combined audio stream.

30. A machine readable medium as described in claim 29, wherein the instructions, when executed, cause the machine to associate the text portion of the program with a portion of the first audio stream when the first audio stream does not contain any spoken words.

31. A machine readable medium as described in claim 29, wherein the instructions, when executed, cause the machine to present the combined audio stream and the text portion of the program to the blind person.

32. A machine readable medium as described in claim 29, wherein the instructions, when executed, cause the machine to:

detect that a first video frame of the video stream is unchanged from a second video frame of the video stream; and

perform optical character recognition to extract the text from the first video frame, the text representing alphanumeric characters presented in the first and second video frames.

33. A machine readable medium as described in claim 29, wherein the instructions, when executed, cause the machine to:

detect that the video stream contains scrolling text; and

perform optical character recognition to extract the text from the video stream.

34. A machine readable medium as described in claim 29, wherein the instructions, when executed, cause the machine to:

detect a link associated with the video program; and

obtain the text from a text server associated with the link, the text representing a text-based description of a corresponding portion of the video stream.

35. A machine readable medium as described in claim 34, wherein the instructions, when executed, cause the machine to synchronize the second audio stream to the first audio stream with markup language data having a timestamp.

36. A machine readable medium as described in claim 29, wherein the instructions, when executed, cause the machine to:

query a text-to-speech database to determine whether first audio data corresponding to a word to be converted exists; and

when the first audio data does not exist;

perform text-to-speech synthesis to convert the word to second audio data; and

add the second audio data to the second audio stream.

37. A machine readable medium as described in claim 36, wherein the instructions, when executed, cause the machine to add the first audio data to the second audio stream when the first audio data exists.

38. A machine readable medium as described in claim 36, wherein the instructions, when executed, cause the machine to send the word to a network-based text-to-speech synthesis server to obtain the second audio data.