Distributed, predictive, dichotomous decision engine for an electronic personal assistant

• US 9,842,299 B2
• Filed: 01/25/2012
• Issued: 12/12/2017
• Est. Priority Date: 01/25/2011
• Status: Active Grant

• Current Assignee: Telepathy Labs, Inc.
• Original Assignee: Telepathy Labs, Inc.
• Inventors: Stolarz, Damien Phelan, Diaz, David Joseph, Rossfeld, James, Raven, Scott, O'Malley, Chris, Kurpinski, Christopher
• Primary Examiner(s): Sadio, Insa
• Application Number:
  US13/358,004
• Publication Number:
  US 20130278492A1
• Time in Patent Office: 2,148 Days
• Field of Search: 345156, 345173-184
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system, method and user interface are described for providing a personal assistant functionality using a predictive, adaptive, dichotomous (two choices) decision engine that proactively prompts the user for decisions on matters deemed relevant by the decision engine based on past user decisions and activities.

Claims

1. A method of operating a personal assistant comprising:

(a) accepting a user input command as an audio signal from a user on a personal electronic device configured to be able to receive user input commands in audio form without requiring further initialization or wake-up;

(b) determining whether the user input command can be fully processed on the personal electronic device;

(c) if it is determined that the user input command cannot be fully processed on the personal electronic device, establishing a communication link between the personal electronic device and a remote device, and using the communication link to send user input command information to the remote device for processing only on the remote device by streaming the user input command to the remote device while an audio threshold for the audio signal is exceeded; and

(d) if is determined that the user input command can be fully processed on the personal electronic device, fully processing the command only on the personal electronic device,

wherein the remote device is a remote server or a smartphone.

2. The method of claim 1 where the user input command is interpreted using speech recognition.

3. The method of claim 1 wherein the remote device is a remote server and the remote server sends back a command response which is delivered or performed by the personal electronic device.

4. The method of claim 1, further comprising:

attempting to establish the communication link; and

if the attempting step fails, buffering the user input command in a memory of the personal electronic device, such that the user input command can be later delivered.

5. The method of claim 1 wherein a software program operating on the personal electronic device is configured to connect to one or more user data stores via a communication link and the software program is configured to retrieve data therefrom.

6. The method of claim 1 wherein a software program operating on the personal electronic device is connected to one or more online data sources and the software program retrieves data therefrom.

7. The method of claim 1, wherein the personal electronic device is a headset.

8. The method of claim 1, wherein the personal electronic device is a neural interface device.

9. The method of claim 1, wherein the remote device is the smartphone, and the processing of step (d) is performed by another remote server.

10. The method of claim 1, wherein the user input command information is audio information.

11. The method of claim 1, wherein the user input command information is parsed command information.

12. The method of claim 1, wherein the user input command information is user interface device information.

13. The method of claim 1, wherein the user input command information is neural input information.

14. The method of claim 1, wherein the personal electronic device is a wristwatch.

15. An electronic personal assistant system comprising:

(a) a first personal electronic device configured to be able to receive user input commands as an audio signal without requiring further initialization or wake-up, in communication with one or more remote devices wherein the one remote device is a remote server or a smartphone;

(b) a software program running on the first personal electronic device that is configured to;

accept an input command from the user in audio form, determine whether the input command can be fully processed on the first personal electronic device, if it is determined that the user input command cannot be fully processed on the first personal electronic device, transfer input command information to the remote device via a communications link for processing only on the remote device by streaming the user input command to the remote device while an audio threshold for the audio signal is exceeded, and if is determined that the user input command can be fully processed on the first personal electronic device, fully process the command only on the personal electronic device.

16. The system of claim 15 where the software program buffers user commands for later delivery when a needed connection is not immediately available.

17. The system of claim 15 where the user'"'"'s interactions with the system are stored in an historical behavioral database.

18. The system of claim 15 where the remote device delivers data to the user via the first personal electronic device.

19. The system of claim 15 where the software program is connected to the user'"'"'s various data stores via a communications link and retrieves data therefrom.

20. The system of claim 15 where the first personal electronic device is a smartphone.

21. The system of claim 15 where the first personal electronic device is a Bluetooth device, and one or more of the remote devices is a smartphone.

22. The method of claim 15 where the remote server sends back a command response which is delivered or performed by the first personal electronic device.

23. A system of operating a personal assistant, the system comprising:

a remote server operating a software program configured to receive input voice command information from a personal electronic device, the personal electronic device configured to be able to receive an input voice command in audio form without requiring further initialization or wake-up and to stream the input voice command to the remote server while an audio threshold for the input voice command is exceeded, the input voice command information partially processed by the personal electronics device, the software program configured to process the input voice command information, the software program configured to perform commands or retrieve information based upon input voice command information, the software program configured to deliver the results of performing the commands or retrieving information to the personal electronic device, wherein the input voice command information is processed using both the personal electronic device and the remote server.

24. The system of claim 23 where interactions with the system are stored in an historical behavioral database.

25. The system of claim 23 where the one or more remote servers access information from an online data source.

26. The method of claim 25 where the online data source is one of the following:

(a) a remote online database, (b) a social network, (c) a microblogging site, (d) a website, (e) an historical behavioral database.

27. A method of operating a personal assistant comprising:

establishing and maintaining a communication link between a personal electronics device and a remote device;

accepting a voice input command from a user on a personal electronics device configured to be able to receive voice input commands without requiring further initialization or wake-up;

streaming voice input command information based on the voice input command from the personal electronics device to the remote device over the communication link while an audio threshold for the voice input command is exceeded;

processing the voice input command information at the remote device to generate a result; and

communicating the result to the personal electronic device.

28. The method according to claim 27 further comprising processing a portion of the voice input command on the personal electronic device.

29. The method according to claim 27 wherein the communication link to stream a voice input command from the personal electronic device to the remote device is continuously maintained or re-established.

30. The method according to claim 27 wherein the personal electronic device is a smartphone.

31. The method according to claim 27 further comprising operating an API that allows voices services to be utilized on the personal electronic device.

32. The method according to claim 27 wherein the remote device is a remote server.

33. The method of claim 27, wherein the personal electronic device is a wristwatch.