Contextual Prediction of User Words and User Actions

• US 20060247915A1
• Filed: 04/21/2006
• Published: 11/02/2006
• Est. Priority Date: 12/04/1998
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Tegic Communications, Inc. (Microsoft Corporation)
• Inventors: Bradford, Ethan R., Kay, David J., Longe, Michael, Van Meurs, Pim, Tandon, Gaurav
• Granted Patent: US 8,938,688 B2
• Time in Patent Office: Days
• US Class Current: 704/1
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 9/45512 Command shells
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: H04M 1/72451 according to schedules, e.g. using calendar applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver

Abstract

An operating sequence for a handheld computing device manages the device to automatically detect cues describing the device'"'"'s environmental and user actions performed with the device, learn which cues and cue combinations are relevant to predict user actions, and then in response to occurrence of the relevant cues, predictively implementing the appropriate user action or configuring the device in anticipation of user action.

Claims

1. A handheld computing device, comprising;

a user interface;

a digital data processor coupled to the user interface and programmed to perform operations comprising;

over time, observing cues describing environment of the device and observing user performed actions of configuring the device and operating its application programs;

learning which if any of the cues and combinations of cues are relevant predictors of which of the user actions;

for each user action having a learned relevant predictor, planning one or more of the following types of machine-executed actions;

configuring the device in anticipation of the user action;

causing the device to perform the user action;

responsive to occurrence of learned relevant predictors of user actions, performing the machine-executed actions planned for the learned relevant predictors that occurred.

2. The device of claim 1, where the operation of observing user performed actions of configuring the device and operating its application programs includes observing activity including at least the following:

configuring hardware of the device;

starting an application program or system function;

operating menus including navigating said menus and selecting entries of said menus;

entering a hardware or software mode.

3. The device of claim 1, where the operation of configuring of the device in anticipation of the user action includes at least the following operations:

configuring hardware of the device;

starting an application program or system function of the device;

starting an operation whose initiation is represented upon one or more menus provided by the device;

bypassing or prioritizing one or more components of one or more menus provided by the device;

entering a hardware or software mode of the device.

4. The device of claim 1 the cues comprising:

while in a messaging application, one or more of the following;

a message recipient, sender, subject, priority, delivery method, delivery channel;

time including one or more of the following;

day of week, month, year, time of day;

geographic location;

characteristics of ambient noise detected by the device;

software state of the device;

application context of the device.

5. The device of claim 1, the operation of observing cues describing environment of the device comprises operations of observing at least the following:

physical state detected by one or more sensors;

application program state;

changes in application program state.

6. The device of claim 1, where the planning operation comprises:

for a cue comprising arrival of a start time for an entry of an calendar application of the device, planning a configuration operation comprising invoking a silent mode of operation of the device;

for a cue comprising presence of a given character of ambient noise, planning one of the following configuration operations;

switching the device to a hands free mode, or increasing an output volume of the device;

for a cue comprising user selection of a prescribed menu entry, planning a configuration operation comprising configuring the menu to streamline an expected user-invoked follow up action;

for a cue comprising change in application state of a first application, planning a configuration operation comprising one of the following;

automatically opening a second application, streamlining opening of the second application, prioritizing access to functions offered by the second application.

7. The device of claim 1 where the observing and learning operations are performed repeatedly.

8. The device of claim 1, the types of machine-executed actions further including:

prioritizing interpretation of inherently ambiguous user-entered text input by favoring a disambiguated interpretation consistent with the relevant predictor.

9. The device of claim 1, the operations further comprising:

prioritizing interpretation of inherently ambiguous user-entered text input by favoring a disambiguated interpretation consistent with present environment of the device.

10. The device of claim 9, where the prioritizing operation comprises:

where the present environment of the device includes the device having a messaging application open, planning a disambiguation operation comprising promoting a user identifier as best interpretation of user-entered text input;

where the present environment of the device includes detected geographic location, planning a disambiguation operation comprising favoring interpretations of user-entered text input consistent with said geographic location;

where the present environment of the device includes a detected chronological indication, planning a disambiguation operation comprising favoring interpretations of user-entered text input consistent with the detected chronological indication;

where the present environment of the device includes a software application being open with a user-positioned cursor residing in a form field, planning a disambiguation operation comprising favoring interpretations of user-entered text input consistent with a data type associated with the form field.

11. The device of claim 1, the operations further comprising:

responsive to receiving inherently ambiguous user-entered text input, prioritizing interpretation of inherently ambiguous user-entered text input by favoring a disambiguated interpretation consistent with a present environment of the device, where said consistency is according to a predetermined logic independent of learned association between environment and interpretation of user-entered text input.

12. A handheld computing device, comprising:

first means for interfacing with a user;

coupled to the first means, second means for digital data processing including performing operations comprising;

over time, observing cues describing environment of the device and observing user performed actions of configuring the device and operating its application programs;

learning which if any of the cues and combinations of cues are relevant predictors of which of the user actions;

for each user action having a learned relevant predictor, planning one or more of the following types of machine-executed actions;

configuring the device in anticipation of the user action;

causing the device to perform the user action;

responsive to occurrence of learned relevant predictors of user actions, performing the machine-executed actions planned for the learned relevant predictors that occurred.

13. At least one computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform operations to operate a computing device, said operations comprising:

over time, observing cues describing environment of the device and observing user performed actions of configuring the device and operating its application programs;

learning which if any of the cues and combinations of cues are relevant predictors of which of the user actions;

for each user action having a learned relevant predictors planning one or more of the following types of machine-executed actions;

configuring the device in anticipation of the user action;

causing the device to perform the user action;

responsive to occurrence of learned relevant predictors of user actions, performing the machine-executed actions planned for the learned relevant predictors that occurred.

14. At least one computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform operations comprising:

altering programming of a handheld computing device to include added functionality, the added functionality including performance of machine-executed operations comprising;

over time, observing cues describing environment of the device and observing user performed actions of configuring the device and operating its application programs;

learning which if any of the cues and combinations of cues are relevant predictors of which of the user actions;

for each user action having a learned relevant predictor, planning one or more of the following types of machine-executed actions;

configuring the device in anticipation of the user action;

causing the device to perform the user action;

responsive to occurrence of learned relevant predictors of user actions, performing the machine-executed actions planned for the learned relevant predictors that occurred.

15. Circuitry of multiple interconnected electrically conductive elements configured to perform operations to operate a computing device, the operations comprising:

over time, observing cues describing environment of the device and observing user performed actions of configuring the device and operating its application programs;

learning which if any of the cues and combinations of cues are relevant predictors of which of the user actions;

for each user action having a learned relevant predictor, planning one or more of the following types of machine-executed actions;

configuring the device in anticipation of the user action;

causing the device to perform the user action;

responsive to occurrence of learned relevant predictors of user actions, performing the machine-executed actions planned for the learned relevant predictors that occurred.

16. A handheld computing device including a user interface and also including a digital data processor programmed to perform operations comprising detecting cues describing the device'"'"'s context including at least application program state and changes in application program state, detecting user actions of configuring the device and operating application programs of the device, learning which detected cues and cue combinations are relevant to which detected user actions, in response to occurrence of the cues or cue combinations learned to be relevant to a given user action, executing at least part of the given user action or configuring the device in anticipation of the given user action.