SIMULATED HYPERLINKS ON A MOBILE DEVICE

• US 20170139879A1
• Filed: 11/18/2015
• Published: 05/18/2017
• Est. Priority Date: 11/18/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: SHARIFI, Matthew, PETROU, David
• Granted Patent: US 10,055,390 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system

Abstract

Systems and methods simulate a hyperlink in regular content displayed on a screen. An example method can include generating, responsive to detecting a simulated hyperlink indication, a centered selection from content displayed on a display of a computing device, providing the centered selection to a simulated hyperlink model that predicts an operation given the centered selection, and initiating the operation using an intent associated with a mobile application. The simulated hyperlink model may also provide, from the centered selection, an intelligent selection used the intent'"'"'s parameter. Another method includes identifying documents having a hyperlink whitelisted websites, generating positive training examples for a simulated hyperlink model using the documents, each positive training example having a centered selection, a website, and a mobile application mapped to the website, and training the simulated hyperlink model, using the positive training examples, to predict an operation for the mobile application given the centered selection.

Claims

1. A mobile device comprising:

a display device;

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the mobile device to;

generate a centered selection responsive to detecting an action on content displayed on the display device,

provide the centered selection to a simulated hyperlink model, the simulated hyperlink model being trained to predict an operation given a centered selection, the operation being associated with an intent for a mobile application, and

initiating at least one of;

a) display of a user interface permitting selection of the operation, which uses the intent, or

b) performance of the operation using the intent.

2. The mobile device of claim 1, wherein the simulated hyperlink model is further trained to provide an intelligent selection from the centered selection and performance of the operation occurs using the intelligent selection as a parameter for the intent.

3. The mobile device of claim 2, wherein initiating display of the user interface includes displaying a menu with an icon for the mobile application and with an option to copy the intelligent selection.

4. The mobile device of claim 2, wherein the memory further stores instructions that, when executed by the at least one processor, causes the mobile device to:

modify the appearance of text displayed on the display that corresponds to the intelligent selection.

5. The mobile device of claim 1, wherein the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:

initiate, responsive to receiving a selection of the operation from the user interface, performance of the operation using the intelligent selection.

6. The mobile device of claim 1, wherein initiating display of the user interface includes displaying an icon for the mobile application.

7. The mobile device of claim 1, wherein the centered selection includes a word nearest a location for the action and an equal quantity of words from the content that appears prior to the word nearest the location and following the word nearest the location.

8. The mobile device of claim 7, wherein the words prior to and following the word nearest the location exclude words in different paragraphs and generating the centered selection includes:

determining that the quantity of words prior to the word nearest the location is less than the equal quantity; and

adding padding tokens to the centered selection, the padding tokens being counted as words prior to the word nearest the location, so that the quantity of words prior to the word nearest the location s the equal quantity.

9. The mobile device of claim 1, wherein the memory further stores instructions that, when executed by the at least one processor, causes the mobile device to:

receive the simulated hyperlink model from a server;

generate training examples from hyperlinks in a screen capture index for the user of the mobile device; and

using the training examples to train the simulated hyperlink model, resulting in a personalized simulated hyperlink model.

10. A method comprising:

identifying, from a document corpus, one or more documents having a hyperlink to a whitelisted website, the whitelisted website being one of a plurality of websites in a mapping of websites to mobile applications;

generating positive training examples for a simulated hyperlink model using the one or more documents, each positive training example having a centered selection generated from the respective document, a website, a weight corresponding to a page rank of the website, and a mobile application mapped to the website; and

training the simulated hyperlink model, using the positive training examples, to predict an operation for the mobile application given the centered selection.

11. The method of claim 10, further comprising:

identifying a website in a manifest of a first mobile application; and

adding the website and the first mobile application to the mapping.

12. The method of claim 11, wherein the manifest specifies an intent and the operation for the mobile application is the intent.

13. The method of claim 10, wherein the mobile application is a first mobile application and the method further comprises:

clustering similar mobile applications in a web store, the first mobile application appearing in a first cluster;

identifying a document with a hyperlink to the website;

determining an intelligent selection based on the hyperlink;

verifying that a result generated by the first mobile application for the intelligent selection is similar to a result generated by a second mobile application for the intelligent selection, the second mobile application being in the first cluster; and

adding, responsive to the verifying, the website and the second mobile application to the mapping.

14. The method of claim 10, further comprising:

personalizing the model based on a screen capture index for a user of a mobile device.

15. The method of claim 10, wherein the mapping includes records mapping a non-http uniform resource indicator (URI) to a mobile application and the one ore more documents include a hyperlink using the non-http URI.

16. The method of claim 10, wherein the centered selection includes anchor text for the hyperlink and the model is further trained to predict the anchor text as an intelligent selection.

17. A method comprising:

generating, responsive to detecting a simulated hyperlink indication, a centered selection from content displayed on a display of a mobile computing device;

providing the centered selection to a simulated hyperlink model, the simulated hyperlink model being trained to predict an operation given the centered selection, the operation being associated with an intent for a mobile application; and

initiating at least one of;

a) display of a user interface permitting selection of the operation, which uses the intent, or

b) performance of the operation using the intent the operation using the intent.

18. The method of claim 17, wherein generating the centered selection includes:

performing text and image recognition on a screen capture of the content to determine words represented in the content; and

determining a word closest to a location of the simulated hyperlink model as a center word in the centered selection.

19. The method of claim 17, wherein generating the centered selection further includes:

using padding tokens in place of words in the centered selection to avoid including words from different sections.

20. The method of claim 17, wherein the simulated hyperlink model is also trained to make an intelligent selection from the centered selection, the intelligent selection representing a portion of the centered selection and using the intelligent selection as a parameter for the intent.

21. The method of claim 17, wherein the operation is a first operation and the method further comprises:

receiving the first operation and a second operation from the simulated hyperlink model;

determining a probability score for the first operation and a probability score for the second operation are similar;

providing the user interface, which enables the user of the mobile computing device to select the first operation or the second operation;

receiving a selection of the first operation;

generating a positive training example from the centered selection and the first operation; and

initiating the first operation using the intent.

22. The method of claim 17, further comprising:

obtaining, responsive to detecting the simulated hyperlink indication, a partial screen capture image,

wherein the centered selection is obtained from the screen capture image.