Providing actions for mobile onscreen content
First Claim
Patent Images
1. A system comprising:
- at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising;
receiving an image of a screen captured from a display of a mobile device,determining areas of actionable content in the image,determining that a density of areas of actionable content in a first portion of the image exceeds a threshold, the first portion including a first area of actionable content and a second area of actionable content,selecting the first area in response to determining that content represented by the first area has a higher relevancy ranking than content represented by the second area of actionable content,determining an action for the first area of actionable content,generating annotation data for the image, the annotation data including a visual cue that corresponds to the first area of actionable content, the visual cue being actionable to initiate the respective action when selected and configured to be displayed with a current screen on the mobile device, andproviding the annotation data for display on the mobile device.
2 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods identify actionable content in onscreen content and provide at least a default action for the content. For example, a method may include receiving an image of a screen captured from a display of a mobile device, determining areas of actionable content in the image, and determining a respective action for each area of actionable content. The method may also include generating annotation data for the image, the annotation data including a visual cue that corresponds to a first area of actionable content, the visual cue being actionable to initiate the respective action when selected and providing the annotation data for display on the mobile device.
125 Citations
33 Claims
-
1. A system comprising:
-
at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising; receiving an image of a screen captured from a display of a mobile device, determining areas of actionable content in the image, determining that a density of areas of actionable content in a first portion of the image exceeds a threshold, the first portion including a first area of actionable content and a second area of actionable content, selecting the first area in response to determining that content represented by the first area has a higher relevancy ranking than content represented by the second area of actionable content, determining an action for the first area of actionable content, generating annotation data for the image, the annotation data including a visual cue that corresponds to the first area of actionable content, the visual cue being actionable to initiate the respective action when selected and configured to be displayed with a current screen on the mobile device, and providing the annotation data for display on the mobile device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method comprising:
-
receiving an image of a screen captured from a display of a mobile device; determining areas of actionable content in the image; determining that a density of areas of actionable content in a first portion of the image exceeds a threshold, the first portion including a first area of actionable content, a second area of actionable content, and a third area of actionable content; selecting the first area in response to determining that content represented by the first area has a higher relevancy ranking than content represented by the second area of actionable content; determining a first action for the first area of actionable content; generating annotation data for the image, the annotation data including a first visual cue that corresponds to the first area of actionable content, the first visual cue being actionable to initiate the first action when selected; and providing the annotation data for display on the mobile device. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A mobile device comprising:
-
at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the mobile device to; capture an image of a screen on the mobile device, the screen being displayed on a display of the mobile device, provide the image to a server, receive annotation data, the annotation data including a quantity of visual cues, each visual cue corresponding to an area of actionable content, wherein the quantity of visual cues is restricted by a threshold for a density of areas of actionable content, wherein the quantity of visual cues includes at least a first visual cue that corresponds to a first area of actionable content and a second visual cue that corresponds to a second area of actionable content, the first visual cue being actionable to initiate a first action when selected, the second visual cue being actionable to initiate a second action when selected, and display the annotation data with a screen being displayed on the display of the mobile device. - View Dependent Claims (20, 21, 22, 23)
-
-
24. A mobile system comprising:
-
at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the system to; receive an image of a screen captured on the mobile system, the screen being displayed on a display device of the mobile system, determine areas of actionable content in the image, determine that a density of areas of actionable content in a first portion of the image exceeds a threshold, the first portion including a first area of actionable content and a second area of actionable content, select the first area based on determining that content represented by the first area has a higher relevancy ranking than content represented by the second area of actionable content, for at least the first area of actionable content, determine an action associated with the first area, generate annotation data, the annotation data including a visual cue that corresponds to the first area of actionable content, the visual cue being actionable to initiate the action when selected, and display the annotation data with a screen being displayed on the display device. - View Dependent Claims (25, 26, 27, 28, 29, 30, 31, 32, 33)
-
Specification