User interaction pattern extraction for device personalization

• US 9,378,467 B1
• Filed: 01/14/2015
• Issued: 06/28/2016
• Est. Priority Date: 01/14/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Jiang, Lin, Nanjundaiah, Shriram, Chaiyochlarb, Chan, Chory, Susan
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US14/596,881
• Publication Number:
  US 20160203415A1
• Time in Patent Office: 531 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks

Abstract

A context-aware personalization system operable with a device such as a smartphone, tablet, personal computer (PC), game console, etc. extracts user interaction patterns from a graph, which is generated using telemetry data points, of a device user'"'"'s behaviors and interactions. The telemetry data is mined from instrumented applications, operating system, and other components executing on the device. A machine learning pattern recognition algorithm is applied to the behavior and interaction graph to generate a dataset that can include a prioritized list of activities. The list is used to automatically implement personalization of the local device that are tailored to the user while also enabling background agents and processes associated with lower priority applications to be suspended to preserve device resources such as processor cycles, memory, battery power, etc. and increase device performance.

Claims

1. A device, comprising:

one or more processors;

a user interface (UI) for interacting with a user of the device; and

a memory device storing code associated with one or more applications and computer-readable instructions which, when executed by the one or more processors, perform a method comprising the steps of;

collecting telemetry data from the one or more applications, the telemetry data at least describing user interactions with the one or more applications,

obtaining a prioritized list of activities that are associated with the one or more applications, the prioritized list being generated by application of a pattern recognition methodology to extract user interaction patterns from the telemetry data,

personalizing the device responsively to a prioritized list of activities, and

suspending operations of an application associated with activities in the list having a priority below a predetermined threshold.

2. The device of claim 1 further including personalizing the device by composing and rendering a screen on the UI, the screen using user-controllable objects that are emphasized on the UI responsively to the prioritized list, the objects functioning at least as launch controls for the one or more applications.

3. The device of claim 1 further including personalizing the device by composing and rendering a lock screen on the UI, the lock screen showing content or notifications responsively to the prioritized list.

4. The device of claim 1 further including operating a digital assistant on the device responsively to the prioritized list, the digital assistant operation including rendering user experiences on the UI, the UI being adapted for user interaction using audio.

5. The device of claim 4 further including configuring the digital assistant, responsively to voice input, gesture input, or manual input for performing at least one of sharing contact information, sharing a stored contact, scheduling a meeting, looking at a user'"'"'s calendar, scheduling a reminder, making a call, operating a device, playing games, making purchases, taking notes, scheduling an alarm or wake-up reminder, sending a message, checking social media for updates, scraping a website, interacting with a search service, sharing or showing files, sending a link to a website, or sending a link to a resource, the digital assistant further being configured to implement the performing using one or more machine-learning techniques applied to the telemetry data.

6. The device of claim 1 in which the prioritized list is correlated to day and time and further including personalizing the device in response to the correlation to day and time.

7. The device of claim 1 in which the prioritized list is correlated to contextual data further including personalizing the device in response to the correlation to contextual data.

8. The device of claim 7 in which the contextual data comprises one or more of time/date, location of the user or device, language, schedule, applications installed on the device, user preferences, user behaviors, user activities, stored contacts, call history, messaging history, browsing history, device type, device capabilities, or communication network type.

9. The device of claim 1 further including obtaining the prioritized list of activities from a remote service.

10. The device of claim 1 in which the telemetry data describes one or more of device, product, application, activity, activity state, activity duration, date, or time.

11. A method of operating a digital assistant on a device having a user interface (UI) supporting graphics and audio and supporting applications, the method comprising:

exposing the digital assistant on the device for maintaining context-awareness for a device user by collecting telemetry data through monitoring user behaviors and interactions with the device and applications, the digital assistant further interacting with the device user using voice interactions through the UI;

using the context-awareness, performing a task or providing a service in response to a prioritized activity list that is generated from the monitored user behaviors and interactions; and

controlling operation of the device to manage resource utilization by activities having a priority below a predetermined threshold.

12. The method of claim 11 in which the monitoring uses instrumentation instantiated in one of application, operating system, or digital assistant and the context-awareness includes identifying one-off or non-recurring events associated with the monitored user behaviors and interactions.

13. The method of claim 11 in which the performing includes providing a suggestion through the UI for a method of communicating with user of other devices.

14. The method of claim 11 further including, in response to a prioritized activity list that is generated from the monitored user behaviors and interactions and using the context-awareness, personalizing the device to the user so that

i) applications associated with relatively higher priority activities are emphasized on the UI compared with applications associated with relatively lower priority activities, the application emphasis being implemented using one of configuring a size of application icons or configuring a display order for the application icons, and

ii) notifications associated with higher priority applications are emphasized on the UI compared with notifications associated with relatively lower priority, the notifications emphasis being implemented by configuring a display order for the notification icons.

15. The method of claim 14 further including changing the application emphasis or notification emphasis based on time of day, device type, device location, or other context.

16. The method of claim 11 further including generating the prioritized activity list locally on the device according to availability of device resources including network bandwidth.

17. One or more computer-readable memory devices storing instructions which, when executed by one or more processors disposed in a computer server, perform a method comprising the steps of:

interoperating with a personalization client on a local device, the personalization client exposing an application programming interface (API) to one or more applications being executable on the device to collect telemetry data from instrumentation disposed in the one or more applications, the telemetry data describing interactions between a device user and the one or more applications;

obtaining the telemetry data over a network;

applying a pattern recognition process to the telemetry data to extract user interaction patterns;

generating a user interaction pattern dataset by transforming results of the pattern recognition process; and

providing the user interaction pattern dataset to the personalization client on the local device, the local device using the user interaction pattern dataset to personalize the device for the user and allocate resources on the device.

18. The one or more computer-readable memory devices of claim 17 further including implementing a virtuous learning loop with the personalization client and iteratively generating or adjusting the user interaction pattern dataset or pattern recognition process based on a data stream received from the virtuous learning loop.

19. The one or more computer-readable memory devices of claim 17 further including configuring the pattern recognition process to utilize one of a clustering algorithm or naï

ve Bayes classifier.

20. The one or more computer-readable memory devices of claim 17 in which the personalization includes displaying application icons or notifications according to a priority established by the user interaction pattern dataset.