SYSTEM AND METHOD FOR EXTRACTING AND SHARING APPLICATION-RELATED USER DATA

• US 20160335316A1
• Filed: 06/09/2015
• Published: 11/17/2016
• Est. Priority Date: 05/15/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Fernandes, Earlence Tezroyd, Riva, Oriana, Nath, Suman Kumar, Burger, Douglas Christopher
• Granted Patent: US 10,324,926 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/2448 for particular applications; for extensibility, e.g. user defined types
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files

Abstract

Systems and methods for extracting and sharing application-related user data are disclosed. A method may include extracting in-app data for at least one of the plurality of apps running on a computing device, the in-app data including content consumed by a user while the at least one app is running, and/or at least one user action taken in connection with the content. Using an entity template associated with the app, a plurality of text strings within the in-app data are classified into at least one of a plurality of data types specified by the template. At least one user data item (UDI) may be generated by combining at least a portion of the classified plurality of text strings, the at least one UDI being accessible by a second app, an operating system running on the, a service of the operating system, and/or a service running on at least another device.

Claims

1. A computing device, comprising:

a processing unit; and

a memory storing a plurality of applications (apps);

the computing device configured to perform operations for extracting and sharing application-related user data, the operations comprising;

extracting in-app data for at least one of the plurality of apps running on the computing device, the in-app data comprising content consumed by a user while the at least one app is running, and/or at least one user action taken in connection with the content;

using an entity template associated with the at least one app, classifying a plurality of text strings within the in-app data into at least one of a plurality of data types specified by the entity template; and

generating at least one user data item (UDI) by combining at least a portion of the classified plurality of text strings, the at least one UDI being accessible by at least one of the following;

a second app of the plurality of apps, an operating system running on the computing device, a service of the operating system, and/or a service running on at least another device.

2. The computing device according to claim 1, wherein the entity template comprises a plurality of templates for a corresponding plurality of page classes of the at least one app, each template of the plurality of templates comprising a user interface (UI) tree with a plurality of UI elements, each UI element annotated with an entity type and an entity group identifier, or with an action type.

3. The computing device according to claim 2, wherein the at least a portion of the classified plurality of text strings within the UDI are related by have the same group identifier.

4. The computing device according to claim 2, wherein the action type comprises action semantic data indicative of whether a user action is associated with user interest or dis-interest in the content.

5. A method, implemented at least in part by a computing device for extracting and sharing application-related user data, the method comprising:

using a tracer component within the operating system of the computing device;

while an application is running on the computing device, detecting a user interface (UI) event triggered by a user action in connection with a page class of the application, the UI event associated with a UI tree of the page class; and

using an analyzer component within the operating system of the computing device;

receiving a plurality of templates associated with the application running on the computing device, each of the plurality of templates comprising a plurality of entity types and action types;

matching a plurality of text strings within the UI tree with at least one of the plurality of templates to classify the plurality of text strings in the UI tree with at least an entity type of the plurality of entity types or an action type of the plurality of action types;

generating a behavioral data item (BDI) by combining at least a portion of the classified plurality of text strings within the UI tree; and

providing the generated BDI to at least one of the following;

another application running on the client device, one or more services running on the client device, and/or another device.

6. The method according to claim 5, wherein the UI event comprises entering the plurality of text strings while the application is running.

7. The method according to claim 5, wherein the classified plurality of texts within the UI tree that are further associated with the at least one user action.

8. The method according to claim 5, wherein the tracer component and the analyzer component are part of a stand-alone application running on the computing device.

9. The method according to claim 5, wherein the tracer component and the analyzer component are implemented as part as part of the operating system.

10. The method according to claim 5, wherein the tracer component and the analyzer component are implemented as part of an application running on the computing device.

11. The method according to claim 10, wherein the application is a digital personal assistant, and the method further comprises:

receiving by the digital personal assistant, the BDI from the analyzer component using an application programming interface (API); and

generating a response by the digital personal assistant based on the received BDI.

12. The method according to claim 5, wherein each entity type within the templates comprises an entity group identification, and the method further comprises:

classifying the plurality of text strings in the UI tree further based on the group identification; and

generating the BDI by combining the at least a portion of the classified plurality of text strings that have the same group identification.

13. The method according to claim 9, further comprising appending the BDI with one or more of the following:

an identification number of the BDI;

a time stamp associated with the user action;

an entity type for the at least a portion of the classified plurality of texts in the BDI;

an entity name for the at least a portion of the classified plurality of texts in the BDI;

one or more other entities that have an associated relationship based at least on the same group identification;

an identification of possible user actions associated with one or multiple entities of the BDI;

a number of UI taps associated with the user interaction with the plurality of entities in the BDI; and

an amount of time spent at the page class the user interaction took place on.

14. A system for extracting and sharing application-related user data in a client device, the system comprising:

a tracer service operable to;

detect a user interface (UI) event triggered by a user action in connection with a page class of an application, the UI event associated with a UI tree of the page class while the application is running;

an analyzer service operable to;

match a plurality of texts within the UI tree with at least one of a plurality of templates for the application, to classify the plurality of texts in the UI tree with at least a name, a data type, and a group identification;

generate at least one user data item (UDI) by combining at least a portion of the classified plurality of entities within the UI tree that are associated with the at least one user action and have the same group identification; and

a store service operable to;

store the generated at least one UDI in network storage; and

provide access to the stored at least one UDI to at least another application running on the client device.

15. The system according to claim 14, wherein the network storage is at a server computer communicatively coupled to the client device, and the store service is further operable to:

store a plurality of UDIs generated at the client device and at least another device, the client device and the at least another device associated with the same user.

16. The system according to claim 14, wherein:

the tracer service and the analyzer service are implemented as part of the operating system running on the client device.

17. The system according to claim 14, wherein:

the tracer service and the analyzer service are implemented as part of a stand-alone application running on the client device.

18. The system according to claim 14, wherein the analyzer service is further operable to:

receive the plurality of templates associated with the application from a network device communicatively coupled to the client device, each of the plurality of templates comprising a plurality of classified entities, each entity comprising at least the name, the data type, and the group identification.

19. The system according to claim 14, wherein the store service is further operable to:

receive a request for the at least one UDI from an operating system of the device, at least another application running on the device, and/or a service of the operating system.

20. The system according to claim 19, wherein the store service is further operable to:

authenticate the request; and

upon successful authentication of the request, provide the at least one UDI using an application programming interface (API) to the operating system, the at least another application, and/or the service of the operating system.