Interactive voice-activated bot with visual cue

• US 10,861,454 B2
• Filed: 06/12/2018
• Issued: 12/08/2020
• Est. Priority Date: 06/12/2018
• Status: Active Grant

• Current Assignee: Mastercard Asia/Pacific Pte. Ltd (MasterCard Incorporated)
• Original Assignee: Mastercard Asia/Pacific Pte. Ltd (MasterCard Incorporated)
• Inventors: Wang, Zunhua, Fang, Hui, Lian, Shiying
• Primary Examiner(s): Sharma, Neeraj
• Application Number:
  US16/006,214
• Publication Number:
  US 20190378505A1
• Time in Patent Office: 910 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A method includes a voice-activated device establishing a communication channel with a mobile device through a communication interface, receiving a voice command of a user to perform an action, determining, in response to the voice command, the action based at least in part on the voice command, and outputting an audible response corresponding to the determined action. During outputting of the audible response, visual data that includes a representation of the determined action is displayed on the mobile device. The user is enabled to validate or modify the visual data via a user interface of the mobile device.

Claims

1. An interactive voice-activated method comprising:

establishing, by a voice-activated device located in a commercial space, a communication channel with a mobile device of a user through a communication interface;

obtaining, by the voice-activated device from the user mobile device, user device identity information;

transmitting, by the voice-activated device to a backend system, the user device identity information, wherein the user device identity information is associated with past interaction data between the user mobile device and the voice-activated device, and wherein the user device identity information is stored by the backend system;

receiving, by the voice-activated device, a voice command of the user to perform an action associated with shopping tasks;

determining, by the voice-activated device in response to the voice command, the action based at least in part on the voice command;

outputting, by the voice-activated device, an audible response corresponding to the determined action; and

transmitting, by the voice-activated device while outputting the audible response, data concerning the action to the backend system causing the backend system to utilize the user device identity information to provide visual data that includes a representation of the determined action to the user mobile device for display on a display screen enabling the user to at least one of validate, modify, replace or add to the visual data via a user interface of the user mobile device.

2. The method of claim 1, wherein establishing the communication channel further comprises the voice-activated device searching for and connecting to the mobile device using Bluetooth Low Energy (BLE) communication.

3. The method of claim 1, wherein the modifying is accomplished via a web page accessed from the mobile device.

4. The method of claim 1, wherein modifying the visual data modifies the determined action and finalizes the determined action, and further comprising transmitting the modified visual data to the backend system.

5. The method of claim 1, wherein the voice command includes a request to add a product to a shopping list.

6. The method of claim 1, wherein the visual data comprises a text data.

7. The method of claim 1, wherein the user mobile device is a smartphone.

8. The method of claim 1, further comprising:

receiving, by the voice-activated device, additional voice commands of the user to perform additional actions; and

processing the additional voice commands of the user including;

determining, in response to an additional voice command, an additional action based at least in part on the additional voice command;

outputting, by the voice-activated device, a subsequent audible response corresponding to the determined additional action; and

while outputting the subsequent audible response, updating, on the mobile device, the visual data to reflect a representation of the initial action and the additional actions.

9. An interactive voice-activated system comprising:

a voice-activated device processor; and

a memory in communication with the voice-activated device processor, the memory storing program instructions, the voice-activated device processor operative with the program instructions to perform functions as follows;

establish a communication channel between a voice-activated device located in a commercial space and a user mobile device through a communication interface;

obtain user device identity information from the user mobile device wherein the user device identity information is associated with past interaction data between the user mobile device and the voice-activated device, and wherein the user device identity information is stored by the backend system;

transmit the user device identity information to a backend system;

receive a voice command of the user to perform an action associated with shopping tasks;

determine, in response to the voice command, the action based at least in part on the voice command;

output an audible response corresponding to the determined action; and

transmit while outputting the audible response, data concerning the action to the backend system causing the backend system to utilize the user device identity information to provide visual data that includes a representation of the determined action to the user mobile device for display on a display screen to enable the user to at least one of validate, modify, replace or add to the visual data via a user interface of the user mobile device.

10. The system of claim 9, wherein establishing the communication channel further comprises the voice-activated device searching for and connecting to the user mobile device using Bluetooth Low Energy (BLE) communication.

11. The system of claim 9, wherein the modifying is accomplished via a web page accessed from the user mobile device.

12. The system of claim 9, wherein modifying the visual data modifies the determined action and finalizes the determined action, and further comprising transmitting the modified visual data to a server computer.

13. The system of claim 9, wherein the voice command includes a request to add a product to a shopping list.

14. The system of claim 9, wherein the visual data comprises a text data.

15. The system of claim 9, wherein the user mobile device is a smartphone.

16. The system of claim 9, wherein the processor is further configured to:

receive additional voice commands of the user to perform additional actions; and

process the additional voice commands of the user including;

determining, in response to an additional voice command, an additional action based at least in part on the additional voice command;

outputting a subsequent audible response corresponding to the determined additional action; and

while outputting the subsequent audible response, updating the visual data on the mobile device to reflect a representation of the initial action and the additional actions.

17. A non-transitory computer readable medium having stored therein instructions that when executed cause a computer to perform an interactive voice-activated method comprising:

establishing, by a voice-activated device located in a commercial space, a communication channel with a mobile device of a user through a communication interface;

obtaining, by the voice-activated device from the user mobile device, user device identity information, wherein the user device identity information is associated with past interaction data between the user mobile device and the voice-activated device, and wherein the use device identity information is stored by the backend system;

transmitting, by the voice-activated device to a backend system, the user device identity information;

receiving, by the voice-activated device, a voice command of the user to perform an action associated with shopping tasks;

determining, by the voice-activated device in response to the voice command, the action based at least in part on the voice command;

outputting, by the voice-activated device, an audible response corresponding to the determined action; and

transmitting, by the voice-activated device while outputting the audible response, data concerning the action to the backend system causing the backend system to utilize the user device identity information to provide visual data that includes a representation of the determined action to the user mobile device for display on a display screen enabling the user to at least one of validate, modify, replace or add to the visual data via a user interface of the user mobile device.

18. The non-transitory computer readable medium of claim 17, wherein establishing the communication channel further comprises the voice-activated device searching for and connecting to the mobile device using Bluetooth Low Energy (BLE) communication.