Automated intelligent personal representative

• US 10,334,109 B1
• Filed: 12/14/2016
• Issued: 06/25/2019
• Est. Priority Date: 10/10/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: McCobb, Gerald
• Primary Examiner(s): Teshale, Akelaw
• Application Number:
  US15/378,559
• Time in Patent Office: 923 Days
• Field of Search: 37926502, 37921102, 379224, 37926509, 379 8801, 37914205, 37920101, 37920103, 379 2702, 379 8804, 379 8814, 379 8818, 379 9301, 37914201, 37914214, 37920701, 37920714, 37921001, 37921101, 37921501, 37921801, 37922001, 37922101, 37926504, 37926505, 37926507, 37926513, 37926601, 37926607, 379 2601, 379 671, 379 71, 379 8816, 379 8821, 379 9317
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/41 using speaker recognition speaker recognition per se G10L17/00
• CPC Class Codes: H04M 2203/2061 Language aspects
• CPC Class Codes: H04M 2203/2072 Schedules, e.g. personal calendars
• CPC Class Codes: H04M 2203/357 Autocues for dialog assistance
• CPC Class Codes: H04M 2203/5009 Adding a party to an existing conference conducting a computer conference, e.g. admission detection H04L12/1822
• CPC Class Codes: H04M 2203/551 Call history recording a computer conference H04L12/1831
• CPC Class Codes: H04M 2203/6045 Identity confirmation
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/5166 in combination with interactive voice response systems or voice portals, e.g. as front-ends
• CPC Class Codes: H04M 3/5191 : interacting with the Internet
• CPC Class Codes: H04M 3/527 : Centralised call answering arrangements not requiring operator intervention
• CPC Class Codes: H04M 3/53333 : Message receiving aspects
• CPC Class Codes: H04M 3/541 : based on information specified by the calling party
• CPC Class Codes: H04M 3/58 : Arrangements for transferring received calls from one subscriber to another; Arrangements affording interim conversations between either the calling or the called party and a third party substation line holding circuits H04M1/80
• CPC Class Codes: H04M 7/0048 : where the text-based messaging service is a Short Message Service
• CPC Class Codes: H04M 7/0051 : where the data service is a multimedia messaging service
• CPC Class Codes: H04M 7/0054 : where the data service is an electronic mail service

Abstract

A virtual assistant application, system, and service that interacts with a caller over a telephone, cellular, or VoIP network as the called party'"'"'s representative when the called party is unavailable, providing answers to queries and performing commands that were provisioned by the called party. The application includes a telephony interface, speech recognition and synthesis, a dialog manager, and knowledge domain. The application can also be configured to interact with a sender of SMS, MMS, or IM messages.

Claims

1. A method, comprising:

receiving a call at a device;

when the call is unanswerable, forwarding, by the device, the call to a personal representative device;

performing, by a communication programming interface;

directing a network interface to answer the call;

starting an instance of a dialog manager; and

directing the dialog manager to load an application and knowledge domain associated with the answered call;

forwarding, to a natural language processing module, a called party'"'"'s knowledge domain associated with the answered call;

joining the call to a conference call by the personal representative device based on a conference attendee list if the call is not identified as an emergency call;

updating the knowledge domain to participate in the conference call;

transferring the call by the personal representative device to an emergency phone number provided by the called and stored in the called party'"'"'s knowledge domain, wherein the transfer is performed if the call is identified as the emergency call; and

sending an email to a communication device known to the called and stored in the called party'"'"'s knowledge domain as requested by a caller, wherein the personal representative device directs an email server to send the email to or from a called party'"'"'s email account.

2. The method of claim 1, wherein the call is unanswerable the personal representative device performs one of at least answer the call, screen the caller and transfer the call.

3. The method of claim 1, wherein the call is unanswerable the personal representative device performs at least one of asking questions and answering questions.

4. The method of claim 1, wherein the call is unanswerable the personal representative device performs at least one of making appointments and cancelling appointments.

5. The method of claim 1, wherein the call is unanswerable the personal representative device answers an online chat request.

6. The method of claim 1, wherein the call is unanswerable the personal representative device responds to an online chat request.

7. The method of claim 1, wherein the call is unanswerable the personal representative device answers an SMS message.

8. The method of claim 1, wherein the call is unanswerable the personal representative device responds to an SMS message.

9. The method of claim 1, wherein the call is unanswerable the personal representative device participates in an online meeting.

10. The method of claim 1, wherein the call is unanswerable the personal representative device participates in an online discussion.

11. The method of claim 1, wherein the call is unanswerable the personal representative device responds to user commands.

12. The method of claim 1, wherein the call is unanswerable the communication programming interface authenticates a user voice.

13. The method of claim 1, wherein the call is unanswerable the communication programming interface accesses a user calendar.

14. The method of claim 1, wherein the call is unanswerable the communication programming interface accesses contact information.

15. The method of claim 1, wherein the call is unanswerable the communication programming interface presents caller history.

16. The method of claim 1, wherein the call is unanswerable the communication programming interface presents usage statistics.

17. The method of claim 1, further comprising:

forwarding, to the communication programming interface, a language model and grammars to be installed with an automatic speech recognition module;

outputting, from the natural language processing module, a list of intents having entities and confidence scores associated with the list of intents; and

providing a best match to the dialog manager based on a dialog context and on the knowledge domain.

18. A non-transitory computer readable medium comprising instructions that, when read by a processor, cause the processor to perform:

receiving a call at a device;

when the call is unanswerable, forwarding, by the device, the call to a personal representative device;

performing, by a communication programming interface;

directing a network interface to answer the call;

starting an instance of a dialog manager; and

directing the dialog manager to load an application and knowledge domain associated with the answered call;

forwarding, to a natural language processing module, a called party'"'"'s knowledge domain associated with the answered call;

joining the call to a conference call by the personal representative device based on a conference attendee list if the call is not identified as an emergency call;

updating the knowledge domain to participate in the conference call;

transferring the call by the personal representative device to an emergency phone number provided by the called and stored in the called party'"'"'s knowledge domain, if the call is identified as the emergency call; and

sending an email to a communication device known to the called and stored in the called party'"'"'s knowledge domain as requested by a caller, wherein the personal representative device directs an email server to send the email to or from a called party'"'"'s email account.

19. The non-transitory computer readable medium of claim 18, further comprising:

forwarding, to the communication programming interface, a language model and grammars to be installed with an automatic speech recognition module;

outputting, from the natural language processing module, a list of intents having entities and confidence scores associated with the list of intents; and

providing a best match to the dialog manager based on a dialog context and on the knowledge domain.

20. A system, comprising:

a device configured to receive a call;

when the call is unanswerable, the device is configured to forward the call to a personal representative device;

a communication programming interface configured to;

direct a network interface to answer the call;

start an instance of a dialog manager; and

direct the dialog manager to load an application and knowledge domain associated with the answered call;

wherein a natural language processing model forwards a called party'"'"'s knowledge domain associated with the answered call;

wherein the call to a conference call based on a conference attendee list is joined via the personal representative device if the call is not identified as an emergency call;

wherein the knowledge domain is updated to participate in the conference call;

wherein the call is transferred by the personal representative device to an emergency phone number provided by the called and stored in the called party'"'"'s knowledge domain, if the call is identified as the emergency call; and

wherein an email is sent to a communication device known to the called and stored in the called party'"'"'s knowledge domain as requested by a caller wherein the personal representative device directs an email server to send the email to or from a called party'"'"'s email account.