Providing automatic actions for mobile onscreen content
First Claim
1. A method comprising:
- performing recognition on content captured from a display of a mobile device;
identifying a plurality of entities in the content;
issuing a respective query for each of the plurality of entities;
ranking the plurality of entities based on search results returned for the respective queries;
determining that the search results for a first entity of the highest ranked entities includes a knowledge panel;
extracting information from the knowledge panel for the first entity;
generating a respective action card for at least one of the highest ranked entities, including the first entity, wherein each of the respective action cards includes at least three actions per entity and the actions for the respective action card for the first entity are generated from the information extracted from the knowledge panel and the action card for the first entity includes text extracted from the knowledge panel; and
providing the respective action cards for display to a user of the mobile device, wherein the mobile device displays the action cards.
3 Assignments
0 Petitions
Accused Products
Abstract
Implementations provide an interface for initiating actions related to content the user is viewing on a mobile device. A method includes performing recognition on content captured from a display of a mobile device and identifying a plurality of entities in the content. The method may include issuing a query for each of the plurality of entities, ranking the plurality of entities based on the search results, generating an action card for at least some of the highest ranked entities, and providing the action cards for display on the mobile device. Alternatively or additionally, the method may include determining at least one contact in a contacts data store that corresponds to an entity and generating an action card for the entity, the action card having a first action that uses first information from the contacts data store and a second action that uses second information from the contacts data store.
-
Citations
21 Claims
-
1. A method comprising:
-
performing recognition on content captured from a display of a mobile device; identifying a plurality of entities in the content; issuing a respective query for each of the plurality of entities; ranking the plurality of entities based on search results returned for the respective queries; determining that the search results for a first entity of the highest ranked entities includes a knowledge panel; extracting information from the knowledge panel for the first entity; generating a respective action card for at least one of the highest ranked entities, including the first entity, wherein each of the respective action cards includes at least three actions per entity and the actions for the respective action card for the first entity are generated from the information extracted from the knowledge panel and the action card for the first entity includes text extracted from the knowledge panel; and providing the respective action cards for display to a user of the mobile device, wherein the mobile device displays the action cards. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system comprising:
-
at least one processor; an indexed document corpus; a graph-based data store; and memory storing instructions that, when executed by the at least one processor cause the system to perform operations comprising; performing recognition on content captured from a display of a mobile device, identifying a plurality of entities in the content, for each of the plurality of entities, issuing a respective query to a search engine for the entity, the search engine searching the graph-based data store and the indexed document corpus and generating search results for the entity, ranking the plurality of entities based on the search results, the search results for a first entity including a knowledge panel, extracting information from the knowledge panel for the first entity, and providing the plurality of entities with respective ranks and search results to the mobile device, the mobile device generating action cards for at least one of the highest ranked entities using the respective search results, wherein the action cards include at least three actions per entity and wherein the actions for the action card for the first entity are generated from the information extracted from the knowledge panel. - View Dependent Claims (9, 10)
-
-
11. A system comprising:
-
a contacts data store; at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the system to; perform recognition on content displayed on a display of a mobile device, identify a first entity and a second entity in the content, determine a contact in the contacts data store that corresponds to the first entity, generate a first action card for the first entity, the action card having a first action that uses first information from the contacts data store for the contact and a second action that uses second information from the contacts data store for the contact, obtain search results for the second entity from a search engine, the search results including a knowledge panel for the second entity, extract information from the knowledge panel for the second entity, generate a second action card for the second entity from the information extracted from the knowledge panel, the action card including at least three actions, and display the first action card and the second action card on the display. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
Specification