Contextual prediction of user words and user actions

• US 9,626,355 B2
• Filed: 08/28/2014
• Issued: 04/18/2017
• Est. Priority Date: 12/04/1998
• Status: Expired due to Term

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Bradford, Ethan R., Kay, David J., Longe, Michael, Van Meurs, Pim, Tandon, Gaurav
• Primary Examiner(s): Pan, Yongjia
• Application Number:
  US14/472,279
• Publication Number:
  US 20140372345A1
• Time in Patent Office: 964 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 9/45512 Command shells
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: H04M 1/72451 according to schedules, e.g. using calendar applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver

Abstract

An operating sequence for a handheld computing device manages the device to automatically detect cues describing the device'"'"'s environmental and user actions performed with the device, learn which cues and cue combinations are relevant to predict user actions, and then in response to occurrence of the relevant cues, predictively implementing the appropriate user action or configuring the device in anticipation of user action.

Claims

1. A handheld computing device, comprising:

a processor programmed for learning user action predictors by monitoring and recording both of cues describing the device environment and user performed actions of configuring the device and operating its application programs;

in response to occurrence of one or more of said learned user action predictors, said processor performing any of configuring said device in anticipation of said user action, causing said device carry out a next user action, and prioritizing certain interpretations of inherently ambiguous user-entered text input; and

said processor favoring a disambiguated interpretation consistent with a present environment of the device, where the present environment of the device includes detected geographic location and the prioritizing operation comprises planning a disambiguation operation comprising favoring interpretations of user-entered text input consistent with said geographic location.

2. The device of claim 1, further comprising:

said processor learning which, if any, of the recorded cues and combinations of recorded cues are relevant predictors of which of the recorded user actions;

for each user action having a learned relevant predictor, planning one or more of the following types of machine-executed actions;

configuring the device to streamline user performance of the user action; and

causing the device to perform the user action.

3. The device of claim 1, further comprising:

said processor monitoring and recording activity including at least the following;

user performed actions of configuring hardware of the device;

user performed actions of starting an application program or system function;

user performed actions of operating menus including navigating said menus and selecting entries of said menus; and

user performed actions of entering a hardware or software mode.

4. The device of claim 1, further comprising:

said processor streamlining user performance of the user action by at least the following operations;

configuring hardware of the device;

starting an application program or system function of the device;

starting an operation whose initiation is represented upon one or more menus provided by the device;

bypassing or prioritizing one or more components of one or more menus provided by the device; and

entering a hardware or software mode of the device.

5. The device of claim 1, wherein, while in a messaging application, said cues comprise one or more of the following:

a message recipient, sender, subject, priority, delivery method, delivery channel;

time including one or more of the following;

day of week, month, year, time of day;

geographic location;

characteristics of ambient noise detected by the device;

software state of the device; and

application context of the device.

6. The device of claim 1, further comprising:

said processor monitoring and recording at least the following;

physical state detected by one or more sensors;

application program state; and

changes in application program state.

7. The device of claim 1, further comprising:

said processor prioritizing interpretation of inherently ambiguous user-entered text input by favoring a disambiguated interpretation consistent with the relevant predictor.

8. The device of claim 1, wherein said processor prioritizing further comprises:

where the present environment of the device includes the device having a messaging application open, promoting a user identifier as best interpretation of user-entered text input;

where the present environment of the device includes a detected chronological indication, favoring interpretations of user-entered text input consistent with the detected chronological indication;

where the present environment of the device includes a software application being open with a user-positioned cursor residing in a form field, favoring interpretations of user-entered text input consistent with a data type associated with the form field.

9. The device of claim 1, further comprising:

responsive to receiving inherently ambiguous user-entered text input, said processor prioritizing interpretation of inherently ambiguous user-entered text input by favoring a disambiguated interpretation consistent with a present environment of the device, where said consistency is according to a predetermined logic independent of a learned association between the environment and an interpretation of user-entered text input.

10. A handheld computing device, comprising:

a processor programmed for learning user action predictors by monitoring and recording both of cues describing the device environment and user performed actions of configuring the device and operating its application programs;

in response to occurrence of one or more of said learned user action predictors, said processor performing any of configuring said device in anticipation of said user action, causing said device carry out a next user action, and prioritizing certain interpretations of inherently ambiguous user-entered text input;

said processor favoring a disambiguated interpretation consistent with a present environment of the device, where the present environment of the device includes detected geographic location and the prioritizing operation comprises planning a disambiguation operation comprising favoring interpretations of user-entered text input consistent with said geographic location;

for a cue comprising arrival of a start time for an entry of an calendar application of the device, invoking a silent mode of operation of the device;

for a cue comprising presence of a given character of ambient noise, switching the device to a hands free mode, or increasing an output volume of the device;

for a cue comprising user selection of a prescribed menu entry, configuring the menu to streamline an expected user-invoked follow up action; and

for a cure comprising change in application state of a first application, automatically opening a second application, streamlining opening of the second application, prioritizing access to functions offered by the second application.

11. A processing method for a handheld computing device, comprising:

providing a processor programmed for learning user action predictors by monitoring and recording both of cues describing the device environment and user performed actions of configuring the device and operating its application programs; and

in response to occurrence of one or more of said learned user action predictors, said processor performing any of configuring said device in anticipation of said user action, causing said device carry out a next user action, and prioritizing certain interpretations of inherently ambiguous user-entered text input; and

said processor favoring a disambiguated interpretation consistent with a present environment of the device, where the present environment of the device includes detected geographic location and the prioritizing operation comprises planning a disambiguation operation comprising favoring interpretations of user-entered text input consistent with said geographic location.