Method, system and device for inferring a mobile user's current context and proactively providing assistance

• US 10,163,058 B2
• Filed: 08/14/2012
• Issued: 12/25/2018
• Est. Priority Date: 08/14/2012
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Nitz, Kenneth C., Lincoln, Patrick D., Myers, Karen L., Bui, Hung H., Senanayake, Rukman, Denker, Grit, Mark, William S., Winarsky, Norman D., Weiner, Steven S.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Pellett, Daniel T
• Application Number:
  US13/585,003
• Publication Number:
  US 20140052680A1
• Time in Patent Office: 2,324 Days
• Field of Search: None
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

A device, method and system for automatically inferring a mobile user'"'"'s current context includes applying a user activity knowledge base to real-time inputs and stored user-specific information to determine a current situation. Automated reasoning is used to infer a user-specific context of the current situation. Automated candidate actions may be generated and performed in accordance with the current situation and user-specific context.

Claims

1. A computer-implemented method comprising:

executing systems software to receive real-time inputs by a mobile electronic device;

monitoring stored information accessible to the mobile electronic device;

executing framework/middleware that is interposed between the systems software and applications software to interface with the systems software to apply one or more learning-based algorithms to the real-time inputs and the stored information, wherein the framework/middleware is programmed to;

determine, based on the applying of the learning-based algorithms to the real-time inputs and the stored information, a user-specific usual temporal pattern of activity that occurs in a physical environment, detect a change to a part of the user-specific usual temporal pattern of activity, and

determine that the change corresponds to a user-specific unusual temporal pattern of user activity;

executing the applications software to interface with the framework/middleware to, when the user-specific unusual temporal pattern of user activity is detected, cause the mobile electronic device to execute an action that is associated by a stored model with the user-specific unusual temporal pattern of user activity and is not associated by the stored model with the user-specific usual temporal pattern of activity;

executing the applications software to interface with the framework/middleware to, when the user-specific usual temporal pattern of user activity is detected, cause the mobile electronic device to execute an action that is associated by the stored model with the user-specific usual temporal pattern of activity and is not associated by the stored model with the user-specific unusual temporal pattern of user activity.

2. The method of claim 1, comprising automatically generating a suggestion, wherein the suggestion comprises at least one of a notification, a recommended action, a prompt for user input, and presenting the suggestion at the mobile electronic device.

3. The method of claim 2, comprising presenting the suggestion in at least one of a textual, a graphical, a visual, an audible form.

4. The method of claim 3, comprising selecting a method of presenting the suggestion based on the user-specific unusual temporal pattern of user activity.

5. The method of claim 2, comprising communicating the suggestion to another electronic device.

6. The method of claim 1, comprising comparing a current set of the real-time inputs to stored information associated with a plurality of instances of an activity over time, generate an inference based on the comparison, and using the inference, determine a set of candidate actions that can be performed by the mobile electronic device.

7. The method of claim 1, comprising determining a set of candidate actions by applying at least one rule to the real-time inputs.

8. The method of claim 1, comprising using a policy to determine a confidence level.

9. The method of claim 6, wherein the set of candidate actions comprises at least one of sending a text message, sending a meeting notification, generating a natural language dialog, autonomously executing an instruction, storing the automated suggestion.

10. The method of claim 1, comprising receiving the real-time inputs from a plurality of different sources.

11. The method of claim 10, wherein at least one of the sources of the real-time inputs comprises at least one of a sensor, a peripheral device, a software application, a communication network, an embedded system.

12. The method of claim 11, wherein the at least one sensor comprises at least one of:

a sensor that is integrated with the mobile electronic device, a sensor that is not integrated with the mobile electronic device.

13. The method of claim 11, wherein at least one of the sources of real-time inputs comprises at least one of a social-media software application, an instant messaging system, a voice communication system, and a real-time information feed.

14. The method of claim 11, comprising determining, based on the at least one real-time input, a location-related aspect of a current situation, wherein the location-related aspect relates to a geographic location or a change in geographic location detected by the source of the real-time input.

15. The method of claim 11, wherein at least one of the real-time inputs comprises publicly available information, and the method comprises determining a degree of urgency associated with the publicly available information and generating the automated suggestion based on the degree of urgency associated with the publicly available information.

16. The method of claim 11, wherein at least one of the real-time inputs comprises a digital image, and the method comprises deriving information relating to the current location-related situation from the digital image.

17. The method of claim 1, further comprising canceling a generated suggestion based on at least one user-specified rule.

18. The method of claim 6, comprising communicating the candidate action over a network to another electronic device.

19. The method of claim 2, comprising receiving feedback in response to execution of the automated suggestion and incorporating the feedback into the predictive model.

20. The method of claim 1, comprising arranging interactive icons on a display device based on data associated with the user-specific unusual temporal pattern of activity.

21. The method of claim 1, comprising determining a confidence level associated with the classification of the current combination of the real-time inputs and causing the mobile electronic device to activate a source of real-time inputs to increase the confidence level that is associated with the classification of the current combination of the real-time inputs.

22. A computing device comprising at least one processor and computer circuitry coupled to the at least one processor, wherein the computer circuitry is arranged to cause the at least one processor to perform the method of claim 1.

23. At least one machine-accessible storage medium comprising a plurality of instructions that in response to being executed result in a computing device performing the method of claim 1.