Automated intelligent personal representative

• US 10,171,675 B1
• Filed: 10/10/2016
• Issued: 01/01/2019
• Est. Priority Date: 10/10/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: McCobb, Gerald
• Primary Examiner(s): Teshale, Akelaw
• Application Number:
  US15/289,665
• Time in Patent Office: 813 Days
• Field of Search: 37926509, 37921801, 37920101, 37926501, 37926601, 37926607, 379309, 379 38, 37914206, 37920715, 37926513, 37926606, 379 40, 379 671, 37911415, 379 102, 37920102, 37921001, 37921102, 379223, 379242, 37926502, 37926511, 379 3202, 379 3204, 379 41, 37943306, 379 51, 379 68, 379 70, 379 8801, 379 8816, 379 8818, 379 8823
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/41 using speaker recognition speaker recognition per se G10L17/00
• CPC Class Codes: H04M 2203/2061 Language aspects
• CPC Class Codes: H04M 2203/2072 Schedules, e.g. personal calendars
• CPC Class Codes: H04M 2203/357 Autocues for dialog assistance
• CPC Class Codes: H04M 2203/5009 Adding a party to an existing conference conducting a computer conference, e.g. admission detection H04L12/1822
• CPC Class Codes: H04M 2203/551 Call history recording a computer conference H04L12/1831
• CPC Class Codes: H04M 2203/6045 Identity confirmation
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/5166 in combination with interactive voice response systems or voice portals, e.g. as front-ends
• CPC Class Codes: H04M 3/5191 : interacting with the Internet
• CPC Class Codes: H04M 3/527 : Centralised call answering arrangements not requiring operator intervention
• CPC Class Codes: H04M 3/53333 : Message receiving aspects
• CPC Class Codes: H04M 3/541 : based on information specified by the calling party
• CPC Class Codes: H04M 3/58 : Arrangements for transferring received calls from one subscriber to another; Arrangements affording interim conversations between either the calling or the called party and a third party substation line holding circuits H04M1/80
• CPC Class Codes: H04M 7/0048 : where the text-based messaging service is a Short Message Service
• CPC Class Codes: H04M 7/0051 : where the data service is a multimedia messaging service
• CPC Class Codes: H04M 7/0054 : where the data service is an electronic mail service

Abstract

A virtual assistant application, system, and service that interacts with a caller over a telephone, cellular, or VoIP network as the called party'"'"'s representative when the called party is unavailable, providing answers to queries and performing commands that were provisioned by the called party. The application includes a telephony interface, speech recognition and synthesis, a dialog manager, and knowledge domain. The application can also be configured to interact with a sender of SMS, MMS, or IM messages.

Claims

1. A method, comprising:

receiving a call at a device;

when the call is unanswerable, forwarding, by the device, the call to a personal representative device;

detecting, by a telephony network interface of the personal representative device, ringing;

performing, by a communication programming interface;

directing the network interface to answer the call;

starting an instance of a dialog manager; and

directing the dialog manager to load an application and knowledge domain associated with the answered call;

forwarding, to the communication programming interface, a language model and grammars to be installed with an automatic speech recognition module;

outputting, from the automatic speech recognition module, a list of matches with a corresponding confidence score;

providing a best match to the dialog manager;

updating, by the dialog manager, a dialog context;

engaging in a dialog, by the personal representative device, with a caller device based on the dialog context; and

recognizing, by the personal representative device, an emergency and providing the caller with emergency information available in the application.

2. The method of claim 1, wherein the call is unanswerable when at least one of the device or a user of the device is at least one of busy or not available.

3. The method of claim 1, wherein the application and the knowledge domain is provided to the dialog manager by at least one of:

an application server and a database server.

4. The method of claim 1, wherein when the call is unanswerable, the dialog manager plays an initial prompt for a caller of the call.

5. The method of claim 4, wherein the prompt is provided to the communication programming interface to be played by a text-to-speech module.

6. The method of claim 5, wherein the automatic speech recognition module receives input from the caller and processes the input against at least one of the grammars and semantic analysis.

7. The method of claim 1, wherein the list is given to the communication programming interface and forwarded to a grammar matching module.

8. The method of claim 7, wherein at least one of the grammar matching module and a semantic analysis analyzes each match for accuracy against a conversation context maintained by the context manager and selects the best one.

9. The method of claim 1, comprising identifying the caller and providing, by the personal representative device, the caller with caller-specific information available in the application.

10. The method of claim 1, comprising providing secure access to calendar and contact information of the user of the device, the information stored in a cloud-based storage environment.

11. The method of claim 1, wherein the dialog between the caller and the personal representative device continues until at least one of the following occurs:

the call is terminated;

the call is transferred; and

the call is added to another call.

12. The method of claim 9, wherein, after one of the following occurs, the dialog manager requests the application server to unload the application.

13. A non-transitory computer readable medium comprising instructions that, when read by a processor, cause the processor to perform:

receiving a call at a device;

when the call is unanswerable, forwarding, by the device, the call to a personal representative device;

detecting, by a telephony network interface of the personal representative, ringing;

performing, by a communication programming interface;

directing the network interface to answer the call;

starting an instance of a dialog manager; and

directing the dialog manager to load an application and knowledge domain associated with the answered call;

forwarding, to the communication programming interface, a language model and grammars to be installed with an automatic speech recognition module;

outputting, from the automatic speech recognition module, a list of matches with a corresponding confidence score;

providing a best match to the dialog manager;

updating, by the dialog manager, a dialog context;

engaging in a dialog, by the personal representative device, with a caller device based on the dialog context; and

recognizing, by the personal representative device, an emergency and providing the caller with emergency information available in the application.

14. The non-transitory computer readable medium of claim 13, wherein the call is unanswerable when at least one of the device or a user of the device is at least one of busy or not available.

15. The non-transitory computer readable medium of claim 13, wherein the application and the knowledge domain is provided to the dialog manager by at least one of:

an application server and a database server.

16. The non-transitory computer readable medium of claim 13, wherein when the call is unanswerable, the dialog manager plays an initial prompt for a caller of the call.

17. The non-transitory computer readable medium of claim 16, wherein the prompt is provided to the communication programming interface to be played by a text-to-speech module.

18. The non-transitory computer readable medium of claim 17, wherein the automatic speech recognition module receives input from the caller and processes the input against at least one of the grammars and semantic analysis.

19. The non-transitory computer readable medium of claim 13, wherein the list is given to the communication programming interface and forwarded to a grammar matching module.

20. A system, comprising:

a device configured to receive a call;

when the call is unanswerable, the device is configured to forward the call to a personal representative device;

a telephony network interface of the personal representative device configured to detect ringing;

a communication programming interface configured to;

direct the network interface to answer the call;

start an instance of a dialog manager; and

direct the dialog manager to load an application and knowledge domain associated with the answered call;

wherein a language model and grammars to be installed with an automatic speech recognition module are forwarded to the communication programming interface;

wherein a list of matches with a corresponding confidence score are output from the automatic speech recognition module;

wherein a best match is provided to the dialog manager;

wherein the dialog manager updates a dialog context;

wherein the personal representative device engages in a dialog with a caller device based on the dialog context; and

wherein the personal representative device recognizes an emergency and provides the caller with emergency information available in the application.