Providing contextual actions for mobile onscreen content

• US 10,535,005 B1
• Filed: 12/21/2016
• Issued: 01/14/2020
• Est. Priority Date: 10/26/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Badr, Ibrahim, Zuluaga, Mauricio, Okonkwo, Aneto, Bakir, Gokhan
• Primary Examiner(s): Roni, Syed A
• Application Number:
  US15/386,771
• Time in Patent Office: 1,119 Days
• Field of Search: None
• CPC Class Codes: G06F 21/44 Program or device authentication
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 8/61 Installation
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04W 88/02 Terminal devices

Abstract

Systems and methods provide an application programming interface to offer action suggestions to third-party applications using context data associated with the third-party. An example method includes receiving content information and context information from a source mobile application, the content information representing information to be displayed on a mobile device as part of a source mobile application administered by a third party, the context information being information specific to the third party and unavailable to a screen scraper. The method also includes predicting an action based on the content information and the context information, the action representing a deep link for a target mobile application. The method further includes providing the action to the source mobile application with a title and a thumbnail, the source mobile application using the title and thumbnail to display a selectable control that, when selected, causes the mobile device to initiate the action.

Claims

1. A system comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the system to;

receive a request for a suggested action from a source mobile application operated by a third party, the request including content information and context information, the content information representing information to be displayed on a mobile device by the source mobile application and the context information including information specific to the source mobile application and unavailable to a screen scraper,

predict at least one action for the content information based on analysis of the content information in view of the context information, and

provide the at least one action, a title, and an image to the source mobile application, the action identifying a target mobile application that differs from the source mobile application, the source mobile application initiating display of the title and the image as a selectable control that is displayed with the content information on the mobile device and, responsive to receiving selection of the selectable control, initiating the action, including transferring focus to a target application associated with the action.

2. The system of claim 1, wherein the analysis of the content information in view of the context information includes using the context information to disambiguate entities in the content information.

3. The system of claim 1, wherein context information includes user interests.

4. The system of claim 1, wherein the context information further includes location information.

5. The system of claim 1, wherein the request includes authentication information and the memory further stores instructions that cause the system to use the authentication information to verify the source mobile application prior to predicting the at least one action.

6. The system of claim 1, wherein the action is an intent for the target application.

7. The system of claim 1, wherein the action is a deep link into the target application.

8. The system of claim 1, wherein the system provides sub-actions based on the content information and the context information.

9. A method comprising:

receiving content information and context information from a source mobile application administered by a third party, the content information representing information to be displayed on a mobile device by the source mobile application, the context information being information specific to the third party and unavailable to a screen scraper;

predicting an action based on the content information and the context information, the action representing a deep link for a target mobile application; and

providing the action to the source mobile application with a title and a thumbnail, the source mobile application using the title and the thumbnail to display a selectable control that, when selected, causes the mobile device to initiate the action, including transferring focus to a target application associated with the action.

10. The method of claim 9, wherein the source mobile application provides authentication information with the content information and the context information and the method further comprises:

determining that the source mobile application is authenticated based on the authentication information; and

performing the predicting and providing responsive to the determining.

11. The method of claim 10, further comprising:

using the authentication information to select a model that is used in the predicting.

12. The method of claim 9, further comprising:

providing a sub-action with the action, the sub-action representing a deep link for downloading and installing the target mobile application.

13. The method of claim 9, wherein the context information includes a user history stored by the third party.

14. The method of claim 9, wherein the context information includes session information for the source mobile application.

15. The method of claim 9, further comprising:

using the context information to rank the action and a second predicted action, the second predicted action also being based on the content information and the context information.

16. The method of claim 9, wherein the context information includes information proprietary to the source mobile application.

17. A method comprising:

providing content information and context information from a source mobile application administered by a third party to an action suggestion application program interface, the content information representing information to be displayed on a mobile device by the source mobile application, the context information including information specific to the third party that is unavailable to a screen scraper;

receiving at least one semantic record, the semantic record identifying an action predicted using the content information and the context information, the semantic record including a title and a thumbnail for the action;

initiating display of the title and the thumbnail as a selectable control that is displayed with the content information on the mobile device;

receiving selection of the selectable control; and

initiating the action, including transferring focus to a target application associated with the action.

18. The method of claim 17, further comprising:

providing authentication information with the content information, wherein receiving the semantic record is dependent on verifying the authentication information.

19. The method of claim 18, wherein the action is predicted using a model selected based on the authentication information.

20. The method of claim 17, further comprising:

providing data identifying the action as selected to the action suggestion application.