Automated data transfer from mobile application silos to authorized third-party applications

• US 10,176,336 B2
• Filed: 09/11/2015
• Issued: 01/08/2019
• Est. Priority Date: 07/27/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Navda, Vishnu, Ramjee, Ramachandran, Vageesh, Devarayasamudra Chandramouli, Guha, Saikat, Padmanabhan, Venkata N.
• Primary Examiner(s): Hirl, Joseph P
• Assistant Examiner(s): Nguy, Chi
• Application Number:
  US14/852,451
• Publication Number:
  US 20170032138A1
• Time in Patent Office: 1,215 Days
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events

Abstract

A “Data Transfer Tool” extracts, labels and stores user data or information that may be confined within application silos during user interaction with arbitrary apps. The Data Transfer Tool enables sharing of this otherwise siloed data shared across and between authorized apps. The Data Transfer Tool provides a task-centric approach to define and extract structured and semantically meaningful information from source applications by providing multiple semantic models that are individually tailored to particular source apps. The Data Transfer Tool applies an accessibility API or the like of the OS in combination with the semantic model for the source app to scrape user entered or selected data or information from the source app. The Data Transfer Tool enables and creates new user experiences and increases user efficiency when interacting with various apps by making the scraped data or information available to subscribing destination apps approved to receive that data or information.

Claims

1. A computer-implemented process, comprising:

applying an application programming interface (API) to extract a plurality of data items from a presentation layer of an operating system executing on a computing device, the presentation layer causing text to be rendered on a display device in response to user interaction with a source application executing under the operating system on the computing device;

applying a predefined application content model corresponding to the source application to identify and label the extracted data items;

applying a subscription process to authorize and subscribe one or more destination applications to access the identified and labeled extracted data items; and

automatically applying one or more of the identified and labeled extracted data items to populate one or more corresponding data fields of one or more of the authorized destination applications.

2. The computer-implemented process of claim 1 further comprising:

perform an automated search to retrieve information relevant to one or more of the extracted data items; and

present a notification in an assistant app that is unrelated to either the source application or the destination applications to present the retrieved information relevant to one or more of the extracted data items.

3. The computer-implemented process of claim 2 further comprising presenting the notification in the assistant app as an overlay on the source application.

4. The computer-implemented process of claim 2 further comprising presenting the notification in the assistant app as an overlay on one or more of the destination applications.

5. The computer-implemented process of claim 2 further comprising presenting sponsored content in the notification.

6. The computer-implemented process of claim 1 further comprising automatically updating the application content model when an analysis of the source application indicates changes to the source application binaries.

7. The computer-implemented process of claim 1 further comprising:

modeling user behaviors with respect to the source application; and

automatically initiating one or more destination applications in response to the modeled user behaviors and one or more of the extracted data items.

8. The computer-implemented process of claim 1 further comprising:

modeling user behaviors with respect to one or more of the source application and the destination applications; and

applying the modeled user behaviors in combination with and one or more of the extracted data items to customize a digital assistant.

9. The computer-implemented process of claim 1 further comprising providing at least a portion of the identified and labeled extracted data items to one or more destination apps running on one or more different computing devices of the user.

10. The computer-implemented process of claim 1 further comprising providing at least a portion of the identified and labeled extracted data items to one or more destination apps running on one or more of different computing devices of different users.

11. The computer-implemented process of claim 1 further comprising providing anonymized versions of at least a portion of the identified and labeled extracted data items to one or more computing devices of other users.

12. A computer-readable storage device having computer executable instructions stored therein, said instructions causing a computing device to execute a method comprising:

applying an application programming interface (API) to extract a plurality actions and corresponding attributes from a presentation layer of an operating system, the presentation layer causing content and user interface (UI) controls of a source app executing under the operating system to be rendered on a display device;

applying a content model of the source app to format the extracted actions and corresponding attributes as app-independent tasks; and

applying one or more of the app-independent tasks to populate one or more variables of one or more of destination apps that are subscribed to receive those app-independent tasks.

13. The computer-readable storage device of claim 12 further comprising a presenting a user interface for setting an authorization state for subscriptions to receive app-independent tasks.

14. The computer-readable storage device of claim 12 further comprising:

performing an automated search to retrieve information relevant to one or more of the app-independent tasks; and

applying an assistant app to present the retrieved information as a user selectable popup on the source application.

15. The computer-readable storage device of claim 12 further comprising automatically updating the content model when an analysis of the source app indicates that the source app has been updated.

16. The computer-readable storage device of claim 12 further comprising:

modeling user behaviors with respect to the source app; and

automatically initiating one or more destination apps in response to the modeled user behaviors and one or more of the app-independent tasks.

17. The computer-readable storage device of claim 12 further comprising providing at least a portion of the app-independent tasks to one or more destination apps running on one or more different computing devices of the user.