Spoken control for user construction of complex behaviors

• US 9,324,327 B2
• Filed: 04/23/2015
• Issued: 04/26/2016
• Est. Priority Date: 02/29/2012
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Ehsani, Farzad, Witt-Ehsani, Silke Maren, Master, Demitrios
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US14/694,878
• Publication Number:
  US 20150228283A1
• Time in Patent Office: 369 Days
• Field of Search: 704/275, 726/22, 726/23, 726/24, 726/25, 705/39, 707/723, 706/14
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: H04M 1/72448 with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72451 according to schedules, e.g. using calendar applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

A device interface system is presented. Contemplated device interfaces allow for construction of complex device behaviors by aggregating device functions. The behaviors are triggered based on conditions derived from environmental data about the device.

Claims

1. A virtual assistant device comprising:

a memory;

at least one processor;

a dialog interface including at least one sensor and configured to;

accept a digital representation of a dialog interaction with a user via the at least sensor; and

derive an interpretation of the dialog interaction from signal characteristics of the digital representation; and

a behavior triggering module comprising software instruction stored in the memory and that configures the at least one processor to;

receive the interpretation from the dialog interface;

determine if the interpretation from dialog interface has sufficient information to construct a future device behavior;

instruct the dialog interface to construct a query to be presented to the user to collect additional information when it is determined that the interpretation does not have sufficient information;

identify a set of primitive device functions from a device function database based on the interpretation;

construct a future device behavior from the set of primitive device functions and the interpretation;

generate a trigger for the future device behavior as a function of the future device behavior and a device state associated with a target device;

query the device state associated with a target device to determine satisfaction of the trigger; and

configure the target device to exhibit the future device behavior upon satisfaction of the trigger.

2. The device of claim 1, wherein the digital representation comprises a representation of a spoken utterance.

3. The device of claim 2, wherein the digital representation comprise at least one of the following representations from the spoken utterance:

a word, a token, a syntax, a modulation, a frequency, a tone, and an accent.

4. The device of claim 1, wherein the digital representation comprises a representation of at least one of the following input signals:

haptic data, tactile data, motion data, gesture data, and location data.

5. The device of claim 1, wherein the interpretation comprises a set of key-value pairs.

6. The device of claim 1, wherein the interpretation comprises a concept identifier.

7. The device of claim 1, wherein the behavior triggering module further configures the processor to:

identify missing pieces of information with respect to the future device behavior; and

cause the dialog interface to construct a query for the missing pieces of information from the user.

8. The device of claim 1, wherein the trigger comprises a primary trigger.

9. The device of claim 1, wherein the trigger comprises an optional condition.

10. The device of claim 1, wherein the behavior triggering module further configures the processor to query the device state at regular time intervals.

11. The device of claim 1, wherein the device state comprises a past device state.

12. The device of claim 1, wherein the device state comprises a further device state.

13. The device of claim 1, wherein the device state includes at least one of the following at attributes:

a location, a time, a battery level, a ringer mode, a ringer volume, a device environment attribute, a temperature, and a change in location.

14. The device of claim 1, wherein the future device behavior is a member of a queue of device behaviors.

15. The device of claim 1, further comprising a cell phone that includes the memory, the at least one processor, the dialog interface, and the behavior triggering module.

16. The device of claim 1, further comprising a server that includes the memory, the at least one processor, the dialog interface, and the behavior triggering module.

17. The device of claim 1, wherein the future device behavior comprises at least one of the following:

a device command, a reservation, a temperature change, and a multimodal interaction.

18. The device of claim 1, wherein the behavior triggering module further configures the processor to resolve ambiguities of a future device behavior based on an interaction history stored in an interaction history database.

19. The device of claim 1, wherein the target device comprises the virtual assistant device.