System and method for enhancing live speech with information accessed from the World Wide Web

• US 7,505,907 B2
• Filed: 07/23/2003
• Issued: 03/17/2009
• Est. Priority Date: 09/27/2002
• Status: Expired due to Fees

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Carro, Fernando Incertis
• Primary Examiner(s): Chawan; Vijay B
• Application Number:
  US10/528,175
• Publication Number:
  US 20050251394A1
• Time in Patent Office: 2,064 Days
• Field of Search: 704/270, 704/270.1, 704/257, 725/51, 386/96
• US Class Current: 704/270.1
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links

Abstract

The present invention is directed to a system, method and computer program for enabling an auditor of a live speech to access immediately after or at a later time, complementary information related with terms pronounced during this speech. The system associates hyperlinks (i.e., URLs) with selected terms or words likely to be pronounced by the speaker in the course of the speech. A speech recognition system operating on a speaker device (i.e., a computing system a with a microphone connected to it) recognizes during the speech (i.e., word spotting) the pronunciation by the speaker of anyone of said hyperlinked terms, and records the time at which each recognized hyperlinked term has been pronounced. The system is also based on the synchronization of the speaker device with several auditors devices (e.g., workstations, portable computers, personal digital assistants—PDAs, smart phones, or any other type of handheld computing devices) according to a same universal time, so that the flow of information transmitted by the speaker and received by the auditors is always synchronized, independently of the relative positions of the speaker and auditors. Each time the auditor perceives an interesting topic during the speech, he immediately selects the topic simply by pressing a reserved key on the auditor'"'"'s device. Universal times at which topics are selected by the auditor are stored in the auditor device.

Claims

1. A method for generating a Speech Hyperlink-Time table in conjunction with a system of universal time that provides a same absolute time for every geographical location on Earth, said method comprising:

creating a Speech Hyperlink table structured with columns and rows, wherein the columns of the Speech Hyperlink table consist of a column for storing hyperlinked terms, a column for storing universal times at which corresponding hyperlinked terms are recognized, and a column for storing network addresses linking corresponding hyperlinked term and a corresponding network address of a network, wherein the Speech Hyperlink table comprises a plurality of entries, wherein each entry of the plurality of entries corresponding to a row of the Speech Hyperlink table and comprises a hyperlinked term and a corresponding network address of the network, wherein said network address links the hyperlinked term to information relating to the hyperlinked term, wherein said information is on a server of the network; and

while a speech is being spoken by a speaker, recognizing each hyperlinked term of the Speech Hyperlink table being spoken by the speaker, said recognizing being performed by a speech recognition system on a computing device, and for each recognized hyperlinked term;

determining a universal time at which the hyperlinked term was recognized;

ascertaining, from the Speech Hyperlink table, a network address that corresponds to the recognized hyperlinked term; and

generating a row in the Speech Hyperlink-Time table, wherein said row comprises the universal time, the recognized hyperlinked term, and the network address that corresponds to the recognized hyperlinked term.

2. The method of claim 1, wherein the system of universal time is selected from the group consisting of a system of Global Positioning System (GPS) time, a system of Universal Time Co-ordinated (UTC) time, a system of Greenwich Mean Time (GMT), and a system of time derived from a free-running atomic clock of a GPS satellite.

3. The method of claim 1, said determining the universal time comprising determining the universal time by a Global Positioning System (GPS) receiver.

4. The method of claim 1, said providing the Speech Hyperlink table comprising:

selecting the hyperlinked terms from a text of the speech; and

for each selected hyperlinked term;

identifying a network address corresponding to the selected hyperlinked term and storing, in the Speech Hyperlink table, an entry comprising the selected hyperlinked term and its identified corresponding network address.

5. The method of claim 1, wherein the network is an Internet, wherein the network address is a Universal Resource Locator (URL), wherein the information is a web page, and wherein the server is a web server.

6. The method of claim 1, said method further comprising prior to said recognizing:

training the speech recognition system to recognize the hyperlinked terms as the hyperlinked terms are pronounced during the speech.

7. The method of claim 1, wherein the speech is comprised by a radio program or a television program.

8. The method of claim 1, said determining, ascertaining, and generating being performed by the computing device.

9. A method for processing a speech in conjunction with a system of universal time that provides a same absolute time for every geographical location on Earth, said method comprising:

entering at least one selection command on an auditor device, said auditor device being a computing device, each selection command entered in real-time response to a spoken hyperlinked term of the speech, each spoken hyperlinked term appearing in a record of a plurality of records of a Speech Hyperlink-Time table comprised by a speech server, each record of the Speech Hyperlink-Time table comprising a hyperlinked term of the speech, a universal time at which the hyperlinked term was spoken during the speech, and a network address linking the hyperlinked term to information relating to the hyperlinked term, said information being on a server of a network; and

for each selection command entered;

determining a universal time at which the selection command was entered and recording the determined universal time in a record of a Selections Hyperlink-Time table comprised by the auditor device.

10. The method of claim 9, said auditor device being coupled to the speech server through the network, said method further comprising selecting at least one universal time from the Selections Hyperlink-Time table and for each selected universal time:

sending the selected universal time to the speech server;

receiving from the speech server a hyperlinked term and its associated network address appearing in a record of the Speech Hyperlink-Time table whose included universal time is closest to the selected universal time and does not exceed the selected universal time; and

storing the received hyperlinked term and its associated network address in a record of the Selections Hyperlink-Time table that comprises to the selected universal time.

11. The method of claim 10, wherein the Speech Hyperlink table is structured with columns and rows, wherein the columns of the Speech Hyperlink table consist of a column for storing hyperlinked terms, a column for storing universal times at which corresponding hyperlinked terms are recognized, and a column for storing network addresses linking corresponding hyperlinked term and a corresponding network address of the network, and wherein each record of the Speech Hyperlink-Time table consists of a row of the Speech Hyperlink-Time table.

12. The method of claim 10, said method further comprising:

selecting a record of the Selections Hyperlink-Time table;

using a network address in the selected record to link to information relating to a hyperlinked term in the selected record;

retrieving the information from a server of the network that comprises the information; and

displaying the retrieved information on the auditor device.

13. The method of claim 9, wherein the system of universal time is selected from the group consisting of a system of Global Positioning System (GPS) time, a system of Universal Time Co-ordinated (UTC) time, a system of Greenwich Mean Time (GMT), and a system of time derived from a free-running atomic clock of a GPS satellite.

14. The method of claim 9, said determining the universal time comprising determining the universal time by a Global Positioning System (GPS) receiver at the auditor device.

15. The method of claim 9, wherein the network is an Internet, wherein the network address is a Universal Resource Locator (URL), wherein the information is a web page, and wherein the server is a web server.

16. The method of claim 9, wherein the speech is comprised by a radio program or a television program.

17. The method of claim 9, said entering, determining, and recording being performed the auditor device.

18. The method of claim 9, wherein the auditor device is selected from the group consisting of a workstation, a portable computer, a personal digital assistant (PDA), a smart phone, and any other type of hand held computing device.

19. The method of claim 9, wherein the Speech Hyperlink table is structured with columns and rows, wherein the columns of the Speech Hyperlink table consist of a column for storing hyperlinked terms, a column for storing universal times at which corresponding hyperlinked terms are recognized, and a column for storing network addresses linking corresponding hyperlinked term and a corresponding network address of the network, and wherein each record of the Speech Hyperlink-Time table consists of a row of the Speech Hyperlink-Time table.