Spoken control for user construction of complex behaviors

• US 9,026,441 B2
• Filed: 02/28/2013
• Issued: 05/05/2015
• Est. Priority Date: 02/29/2012
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Ehsani, Farzad, Witt-Ehsani, Silke Maren, Master, Demitrios
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US13/780,645
• Publication Number:
  US 20130226580A1
• Time in Patent Office: 796 Days
• Field of Search: 704/244, 704/251, 704/270, 704/275, 709/220, 463/30
• US Class Current: 704/244
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: H04M 1/72448 with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72451 according to schedules, e.g. using calendar applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/74 with voice recognition means

Abstract

A device interface system is presented. Contemplated device interfaces allow for construction of complex device behaviors by aggregating device functions. The behaviors are triggered based on conditions derived from environmental data about the device.

Claims

1. A device interface comprising:

a dialog interface module disposed within a device and configured to accept a signal comprising a representation of a spoken utterance;

a data source connection configured to acquire environment data from a plurality of data sources and representative of a device environment;

a device function database storing primitive device functions indexed by device state attributes;

an interaction history database storing previous interactions indexed by environment data attributes; and

a triggering module coupled with the dialog interface, data source interface, and the interaction history database and configured to;

obtain previous interaction from the interaction history database by submitting a query to the interacting history database, the query instantiated based on environment data attributes derived from the environment data;

derive a device state from at least one of the environment data and previous interactions;

obtain a set of primitive device functions from the device function database based on the device state;

instantiate a future device behavior constructed from the set of primitive functions and the dialog signal;

create a trigger as a function of the future device behavior and the device state; and

configure the device to exhibit the future device behavior upon satisfaction of the trigger.

2. The interface of claim 1, wherein the device state comprises a current device state.

3. The interface of claim 1, wherein the device state comprises a previous device state.

4. The interface of claim 1, wherein the device state comprises a future device state.

5. The interface of claim 1, wherein the device state comprises a functional state of the device.

6. The interface of claim 1, wherein the device state comprises a hardware state of the device.

7. The interface of claim 1, wherein at least one of the data sources is internal to the device.

8. The interface of claim 7, wherein the at least one of the data sources comprises an application on the device.

9. The interface of claim 7, wherein the at least one of the data sources comprises an operating system on the device.

10. The interface of claim 1, wherein at least one of the data sources is an external data source that is external to the device.

11. The interface of claim 10, wherein the external data source comprises a network site.

12. The interface of claim 1, wherein the previous interactions comprise user preferences and where the future device behavior depends on a user preference.

13. The interface of claim 1, wherein the previous interactions comprise previously constructed future device behaviors including multiple device function primitives.

14. The interface of claim 13, wherein the future device behavior is derived based on the dialog signal and on at least one of the device function primitives.

15. The interface of claim 1, wherein the device function primitive database is configured to identify device functions as a function of signal characteristics derived from the dialog signal.

16. The interface of claim 1, wherein the trigger comprises optional conditions.

17. The interface of claim 1, wherein the trigger comprises a time dependency.

18. The interface of claim 17, wherein the trigger is satisfied substantially in real-time.

19. The interface of claim 17, wherein the trigger depends on a future device state.

20. The interface of claim 1, wherein the trigger is not satisfied at the time of creation.

21. The interface of claim 1, wherein the dialog signal further comprises a representation of communication modalities other than spoken utterances.

22. The interface of claim 1, wherein the future device behavior in a member of a path of device states.