System and Method for Digital Video Retrieval Involving Speech Recognition

• US 20130300845A1
• Filed: 07/16/2013
• Published: 11/14/2013
• Est. Priority Date: 01/25/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Bangalore, Srinivas
• Granted Patent: US 9,135,336 B2
• Time in Patent Office: Days
• US Class Current: 348/62
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

Disclosed are systems, methods, and computer readable media for retrieving digital images. The method embodiment includes converting a descriptive audio stream of a digital video that is provided for the visually impaired to text and then aligning that text to the appropriate segment of the digital video. The system then indexes the converted text from the descriptive audio stream with the text'"'"'s relationship to the digital video. The system enables queries using action words describing a desired scene from a digital video.

Claims

1. A method comprising:

converting a descriptive audio stream associated with a digital video to text;

performing a non-textual optical analysis of a frame on the digital video, to yield an analysis; and

aligning the text to frames in the digital video based on the analysis, a first bit rate associated with the digital video, and a second bit rate associated with the descriptive audio stream.

2. The method of claim 1, further comprising creating a searchable file, wherein the searchable file comprises an indexed file, the indexed filed comprising the digital video aligned with the text.

3. The method of claim 2, further comprising:

receiving a query for a portion associated with the digital video; and

retrieving and presenting the portion associated with the digital video based on the searchable file.

4. The method of claim 3, wherein the query indexes an action word describing a desired portion associated with the digital video.

5. The method of claim 1, wherein the descriptive audio stream is an audio stream provided for visually impaired audiences in a digital storage format.

6. The method of claim 1, wherein converting of the audio stream to text is further based on an analysis of images in the digital video.

7. The method of claim 6, wherein the analysis of images uses optical character recognition.

8. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, result in the processor performing operations comprising;

converting a descriptive audio stream associated with a digital video to text;

performing a non-textual optical analysis of a frame on the digital video, to yield an analysis; and

aligning the text to frames in the digital video based on the analysis, a first bit rate associated with the digital video, and a second bit rate associated with the descriptive audio stream.

9. The system of claim 8, the computer-readable storage medium having additional instructions stored which result in the operations further comprising creating a searchable file, wherein the searchable file comprises an indexed file, the indexed filed comprising the digital video aligned with the text.

10. The system of claim 9, the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

receiving a query for a portion associated with the digital video; and

retrieving and presenting the portion associated with the digital video based on the searchable file.

11. The system of claim 10, wherein the query indexes an action word describing a desired portion associated with the digital video.

12. The system of claim 8, wherein the descriptive audio stream is an audio stream provided for visually impaired audiences in a digital storage format.

13. The system of claim 8, wherein converting of the audio stream to text is further based on an analysis of images in the digital video.

14. The system of claim 13, wherein the analysis of images uses optical character recognition.

15. A computer-readable storage device having instructions stored which, when executed by a computing device, result in the computing device performing operations comprising:

converting a descriptive audio stream associated with a digital video to text;

performing a non-textual optical analysis of a frame on the digital video, to yield an analysis; and

aligning the text to frames in the digital video based on the analysis, a first bit rate associated with the digital video, and a second bit rate associated with the descriptive audio stream.

16. The computer-readable storage device of claim 15, the computer-readable storage device having additional instructions stored which result in the operations further comprising creating a searchable file, wherein the searchable file comprises an indexed file, the indexed filed comprising the digital video aligned with the text.

17. The computer-readable storage device of claim 16, the computer-readable storage device having additional instructions stored which result in the operations further comprising:

receiving a query for a portion associated with the digital video; and

retrieving and presenting the portion associated with the digital video based on the searchable file.

18. The computer-readable storage device of claim 17, wherein the query indexes an action word describing a desired portion associated with the digital video.

19. The computer-readable storage device of claim 15, wherein the descriptive audio stream is an audio stream provided for visually impaired audiences in a digital storage format.

20. The computer-readable storage device of claim 15, wherein converting of the audio stream to text is further based on an analysis of images in the digital video.