Supplementing audio recorded in a media file

• US 9,318,100 B2
• Filed: 01/03/2007
• Issued: 04/19/2016
• Est. Priority Date: 01/03/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bodin, William K., Jaramillo, David, Redman, Jesse W., Thorson, Derral C.
• Primary Examiner(s): MCCORD, PAUL C
• Application Number:
  US11/619,216
• Publication Number:
  US 20080161948A1
• Time in Patent Office: 3,394 Days
• Field of Search: 700/94, 715200-204, 704/235
• US Class Current: 1/1
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 3/5307 for recording messages comprising any combination of audio and non-audio components

Abstract

Methods, systems, and computer program products are provided for supplementing audio recorded in a media file. Embodiments include receiving a media file; identifying the subject matter of audio portion of the media file; identifying supplemental content for supplementing the subject matter recorded in the audio portion of the media file; and inserting in the media file markup for rendering the supplemental content.

Claims

1. A method for supplementing audio recorded in a media file, the method comprising:

receiving a media file;

extracting keywords from text of an audio portion of the media file, and determining a subject matter from text based on the keywords;

identifying, from the audio portion of the media file, the subject matter of the audio portion of the media file;

searching a content source for supplemental content, wherein the content source is a managed library comprising a database indexed by subject matter, wherein the supplemental content relates to and supplements the identified subject matter of the audio portion of the media file;

identifying, automatically without user intervention, the supplemental content for supplementing the subject matter recorded in the audio portion of the media file;

inserting, in the media file, markup for rendering the supplemental content, wherein the supplemental content comprises one or more image files, video files or text files; and

rendering supplemental content in dependence upon the markup.

2. The method of claim 1 wherein identifying the subject matter of audio portion of the media file further comprises converting the audio portion to text.

3. The method of claim 1 further comprising:

extracting the markup from the media file including extracting synchronization markup from the media file; and

wherein the rendering supplemental content in dependence upon the markup includes synchronizing the rendering of the supplemental content with the playing of the audio portion of the media file in dependence upon the synchronization markup.

4. The method of claim 1 further comprising:

rendering supplemental content in dependence upon the markup including rendering the supplemental content on a browser installed on a handheld recording device; and

playing the audio portion of the media file including playing the audio portion of a media file by the handheld recording device.

5. The method of claim 1 wherein the audio portion of the media file contains speech recorded on a handheld recording device for asynchronous communications between users.

6. The method of claim 1, the method further comprising:

receiving the media file;

extracting the markup from the media file;

searching the database for database markup that matches the markup from the media file, the database markup being associated with said supplemental content from the database;

retrieving the database markup and said supplemental content from said database;

rendering the supplemental content in dependence upon the markup and the database markup; and

playing the audio portion of the media file.

19. The method of claim 1, wherein the audio portion of the media file is a voice message recorded on a handheld device configured to communicate over a wireless network.

7. A system for supplementing audio recorded in a media file, the system comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions, wherein the processor, upon executing the computer program instructions, is capable of:

receiving a media file;

extracting keywords from text of an audio portion of the media file, and determining a subject matter from text based on the keywords;

identifying, from the audio portion of the media file, the subject matter of audio portion of the media file;

searching a content source for supplemental content, wherein the content source is a managed library comprising a database indexed by subject matter, wherein the supplemental content relates to and supplements the identified subject matter of the audio portion of the media file;

identifying, automatically without user intervention, the supplemental content for supplementing the subject matter recorded in the audio portion of the media file;

inserting, in the media file, markup for rendering the supplemental content, wherein the supplemental content comprises one or more image files, video files or text files; and

rendering supplemental content in dependence upon the markup.

8. The system of claim 7 wherein identifying the subject matter of audio portion of the media file further comprises converting the audio portion to text.

9. The system of claim 7 wherein the processor is capable of:

extracting the markup from the media file including extracting synchronization markup from the media file; and

wherein the rendering supplemental content in dependence upon the markup includes synchronizing the rendering of the supplemental content with the playing of the audio portion of the media file in dependence upon the synchronization markup.

10. The system of claim 7 wherein the processor is capable of:

rendering supplemental content in dependence upon the markup including rendering the supplemental content on a browser installed on a handheld recording device; and

playing the audio portion of the media file including playing the audio portion of a media file by the handheld recording device.

11. The system of claim 7 wherein the audio portion of the media file contains speech recorded on a handheld recording device for asynchronous communications between users.

12. The system of claim 7, the system further comprising another computer processor, another computer memory operatively coupled to the another computer processor, the another computer memory having disposed within it other computer program instructions, wherein the another computer processor, upon executing the other computer program instructions, is capable of:

receiving the media file;

extracting the markup from the media file;

searching the database for any database markup that matches the markup from the media file, the database markup being associated with said supplemental content from the database;

retrieving the database markup and said supplemental content from said database;

rendering the supplemental content in dependence upon the markup and the database markup; and

playing the audio portion of the media file.

20. The system of claim 7, wherein the audio portion of the media file is a voice message recorded on a handheld device configured to communicate over a wireless network.

13. A computer program product for supplementing audio recorded in a media file, the computer program product embodied on a non-transitory computer-readable recordable medium, the computer program product comprising:

computer program instructions for receiving a media file;

computer program instructions for extracting keywords from text of an audio portion of the media file, and determining a subject matter from text based on the keywords;

computer program instructions for identifying, from the audio portion of the media file, the subject matter of audio portion of the media file;

computer program instructions for searching a content source for supplemental content, wherein the content source is a managed library comprising a database indexed by subject matter, wherein the supplemental content relates to and supplements the identified subject matter of the audio portion of the media file;

computer program instructions for identifying, automatically without user intervention, supplemental content for supplementing the subject matter recorded in the audio portion of the media file;

computer program instructions for inserting, in the media file, markup for rendering the supplemental content, wherein the supplemental content comprises one or more image files, video files or text files; and

rendering supplemental content in dependence upon the markup.

14. The computer program product of claim 13 wherein computer program instructions for identifying the subject matter of audio portion of the media file further comprise computer program instructions for converting the audio portion to text.

15. The computer program product of claim 13 further comprising:

computer program instructions for extracting the markup from the media file including computer program instructions for extracting synchronization markup from the media file; and

wherein the computer program instructions for rendering supplemental content in dependence upon the markup includes computer program instructions for synchronizing the rendering of the supplemental content with the playing of the audio portion of the media file in dependence upon the synchronization markup.

16. The computer program product of claim 13 further comprising:

computer program instructions for rendering supplemental content in dependence upon the markup including computer program instructions for rendering the supplemental content on a browser installed on a handheld recording device; and

computer program instructions for playing the audio portion of the media file including computer program instructions for playing the audio portion of a media file by the handheld recording device.

17. The computer program product of claim 13 wherein the audio portion of the media file contains speech recorded on a handheld recording device for asynchronous communications between users.

18. The computer program product of claim 13, the computer program product further comprising:

computer program instructions for receiving the media file;

computer program instructions for extracting the markup from the media file;

computer program instructions for searching the database for any database markup that matches the markup from the media file, the database markup being associated with said supplemental content from the database;

computer program instructions for retrieving the database markup and said supplemental content from said database;

computer program instructions for rendering supplemental content in dependence upon the markup and the database markup; and

computer program instructions for playing the audio portion of the media file.

21. The computer program product of claim 13, wherein the audio portion of the media file is a voice message recorded on a handheld device configured to communicate over a wireless network.