Processing and cross reference of realtime natural language dialog for live annotations

• US 9,484,033 B2
• Filed: 12/11/2014
• Issued: 11/01/2016
• Est. Priority Date: 12/11/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bufe, John P., Byron, Donna K., Pikovsky, Alexander, Winkler, Timothy
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Le, Thuykhanh
• Application Number:
  US14/566,900
• Publication Number:
  US 20160171983A1
• Time in Patent Office: 691 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An approach is provided to receive audible speech and convert the received speech to text while the audible speech is being delivered to a user. An annotation candidate is identified in the text and an annotation reference relating to the identified annotation candidate is retrieved and presented to the user.

Claims

1. A method implemented by an information handling system that includes a memory and a processor, the method comprising:

converting audible speech received by a microphone to text, wherein the audible speech is being delivered to a user;

identifying an annotation candidate included in the text;

combining a rarity score pertaining to the identified annotation candidate, a phrase matching score pertaining to the identified annotation candidate, and a signal strength score pertaining to a context of the identified annotation candidate into an overall score;

in response to the overall score exceeding a threshold, retrieving an annotation reference relating to the identified annotation candidate; and

presenting the annotation reference to the user.

2. The method of claim 1 wherein the annotation reference is presented to the user while the audible speech is still being delivered.

3. The method of claim 1 further comprising:

parsing the text into a plurality of parts;

identifying a locatable entity as one of the parts; and

filtering the locatable entity, wherein the filtering results in the annotation candidate.

4. The method of claim 3 wherein the filtering further comprises:

identifying that the locatable entity is an uncommon entity.

5. The method of claim 4 further comprising:

generating the rarity score pertaining to the uncommon entity;

generating the phrase matching score pertaining to the uncommon entity; and

generating the signal strength score pertaining to a context of the uncommon entity.

6. The method of claim 1 further comprising:

determining whether to present the annotation reference based on a number of a plurality of annotation references previously presented to the user in a given amount of time;

presenting the annotation reference in response to the number of the plurality of annotation references not exceeding a threshold; and

refraining from presenting the annotation reference in response to the number of the plurality of annotation references exceeding the threshold.

7. The method of claim 1 further comprising:

retrieving a delivery preference pertaining to the user, wherein the delivery preference is selected from the group consisting of a link and a text;

presenting an annotation link to the annotation reference in response to the delivery preference being the link; and

presenting a descriptive text of the annotation reference in response to the delivery preference being the text.

8. An information handling system comprising:

one or more processors;

one or more data stores accessible by at least one of the processors;

a microphone;

a memory coupled to at least one of the processors; and

a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of;

converting audible speech received by the microphone to text, wherein the audible speech is being delivered to a user;

identifying an annotation candidate included in the text;

combining a rarity score pertaining to the identified annotation candidate, a phrase matching score pertaining to the identified annotation candidate, and a signal strength score pertaining to a context of the identified annotation candidate into an overall score;

in response to the overall score exceeding a threshold, retrieving an annotation reference relating to the identified annotation candidate; and

presenting the annotation reference to the user.

9. The information handling system of claim 8 wherein the annotation reference is presented to the user while the audible speech is still being delivered.

10. The information handling system of claim 8 wherein the actions further comprise:

parsing the text into a plurality of parts;

identifying a locatable entity as one of the parts; and

filtering the locatable entity, wherein the filtering results in the annotation candidate.

11. The information handling system of claim 10 wherein the filtering further comprises:

identifying that the locatable entity is an uncommon entity.

12. The information handling system of claim 11 wherein the actions further comprise:

generating the rarity score pertaining to the uncommon entity;

generating the phrase matching score pertaining to the uncommon entity; and

generating the signal strength score pertaining to a context of the uncommon entity.

13. The information handling system of claim 8 wherein the actions further comprise:

determining whether to present the annotation reference based on a number of a plurality of annotation references previously presented to the user in a given amount of time;

presenting the annotation reference in response to the number of the plurality of annotation references not exceeding a threshold; and

refraining from presenting the annotation reference in response to the number of the plurality of annotation references exceeding the threshold.

14. The information handling system of claim 8 wherein the actions further comprise:

retrieving a delivery preference pertaining to the user, wherein the delivery preference is selected from the group consisting of a link and a text;

presenting an annotation link to the annotation reference in response to the delivery preference being the link; and

presenting a descriptive text of the annotation reference in response to the delivery preference being the text.

15. A computer program product stored in a non-transitory computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to perform actions comprising:

converting audible speech received by a microphone to text, wherein the audible speech is being delivered to a user;

identifying an annotation candidate included in the text;

combining a rarity score pertaining to the identified annotation candidate, a phrase matching score pertaining to the identified annotation candidate, and a signal strength score pertaining to a context of the identified annotation candidate into an overall score;

in response to the overall score exceeding a threshold, retrieving an annotation reference relating to the identified annotation candidate; and

presenting the annotation reference to the user.

16. The computer program product of claim 15 wherein the annotation reference is presented to the user while the audible speech is still being delivered.

17. The computer program product of claim 15 wherein the actions further comprise:

parsing the text into a plurality of parts;

identifying a locatable entity as one of the parts; and

filtering the locatable entity, wherein the filtering results in the annotation candidate.

18. The computer program product of claim 17 wherein the filtering further comprises:

identifying that the locatable entity is an uncommon entity.

19. The computer program product of claim 18 wherein the actions further comprise:

generating the rarity score pertaining to the uncommon entity;

generating the phrase matching score pertaining to the uncommon entity; and

generating the signal strength score pertaining to a context of the uncommon entity.

20. The computer program product of claim 15 wherein the actions further comprise:

determining whether to present the annotation reference based on a number of a plurality of annotation references previously presented to the user in a given amount of time;

presenting the annotation reference in response to the number of the plurality of annotation references not exceeding a threshold; and

refraining from presenting the annotation reference in response to the number of the plurality of annotation references exceeding the threshold.