Redaction suggestion for sharing screen content

• US 9,762,651 B1
• Filed: 08/21/2014
• Issued: 09/12/2017
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Petrou, David
• Primary Examiner(s): Kells, Asher
• Application Number:
  US14/465,279
• Time in Patent Office: 1,118 Days
• Field of Search: None
• CPC Class Codes: G06F 16/156 Query results presentation
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06Q 10/1097 Task assignment
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

Systems and methods are provided for sharing a screen from a mobile device. For example, a method includes capturing an image of a screen displayed on the mobile device in response to a command to share the screen, receiving user instructions for redacting a portion of the image, and transmitting the image with the selected portion redacted to a recipient device selected by the user. As another example, a method includes receiving, from a first mobile device, an identifier for a recipient and an image representing a captured screen of a first mobile device, copying the image to an image repository associated with the recipient, performing recognition on the image, generating annotation data for the image, based on the recognition, that includes at least one visual cue, and providing the image and the annotation data to a second mobile device, the second mobile device being associated with the recipient.

Claims

1. A mobile device comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, generates a user interface configured to;

capture an image of a screen displayed on the mobile device in response to a command to share the screen,

determine that a portion of the image matches an entity previously selected for redaction by a user of the mobile device,

initiate a prompt to the user, the prompt suggesting the portion of the image be redacted,

receive a response to the prompt from the user, the response indicating the portion should be redacted, and

transmit the image, with the portion redacted pursuant to the instructions, to a recipient device selected by the user.

2. The mobile device of claim 1, wherein transmitting the image, with the portion redacted pursuant to said instructions, includes transmitting the image with the portion removed.

3. The mobile device of claim 1, wherein transmitting the image, with the portion redacted pursuant to said instructions, includes obscuring image data in the portion.

4. The mobile device of claim 1, wherein the recipient device is a server and the portion is viewable by a user of a second device at the server based on permissions established by the user of the mobile device.

5. The mobile device of claim 1, wherein the image is one of a plurality of previously captured screen images stored in a data store.

6. The mobile device of claim 1, wherein the user interface is further configured to:

provide a search result to the user via a display of the mobile device, the search result including a plurality of previously captured screen images from the mobile device;

receive a selection of one of the plurality of previously captured screen images; and

transmit, in response to receiving the selection, the selected image to a recipient device selected by the user.

7. The mobile device of claim 1, wherein the user interface is further configured to:

receive a selection of a plurality of previously captured screen images, the plurality of captured screen images being captured on the mobile device; and

stitch the plurality of previously captured screen images together to generate the image.

8. The mobile device of claim 1, further comprising a data store storing:

a plurality of captured user input actions, each input action having a corresponding timestamp; and

a plurality of captured screen images, the image being one of the plurality of images, each image of the plurality of images having a corresponding timestamp, and

wherein the user interface may further be configured to;

determine a set of the user input actions, the set including user input actions occurring prior to the timestamp corresponding to the image and occurring after a timestamp corresponding to a reference image, the reference image being one of the plurality of images, and

transmit the set with the portion to the recipient device.

9. The mobile device of claim 1, wherein the command occurs automatically when a mobile application identified by the user is active.

10. The mobile device of claim 9, wherein the mobile application is a photo application and the command occurs automatically in response to taking a photo and the recipient device is a second mobile device operated by the user.

11. A method comprising:

capturing an image of a screen displayed on a mobile device in response to a command to share the screen;

determining that a portion of the image matches a second portion of a previously captured image, the second portion having been selected for redaction by a user;

initiating a prompt to the user, the prompt suggesting the portion of the image be redacted;

receiving a response to the prompt from the user, the response indicating the portion should be redacted; and

transmitting the image with the portion redacted to a recipient device selected by the user.

12. The method of claim 11, wherein transmitting the image with the portion redacted includes transmitting the image with the portion removed.

13. The method of claim 11, wherein the recipient device is a server and the portion is viewable by a user of a second device at the server based on permissions established by the user of the mobile device.

14. The method of claim 11 further comprising:

providing a search result to the user via a display of the mobile device, the search result including a plurality of previously captured screen images from the mobile device;

receiving a selection of one of the plurality of previously captured screen image; and

transmitting, in response to receiving the selection, the selected image to a recipient device selected by the user.

15. The method of claim 11 further comprising:

receiving a selection of a plurality of previously captured screen images, the plurality of captured screen images being captured on the mobile device; and

stitching the plurality of previously captured screen images together to generate the image.

16. The method of claim 11, wherein the image is one of a plurality of captured screen images stored in a data store, each image of the plurality of images having a corresponding timestamp, and the method further comprises:

determining a set of user input actions, the set including user input actions stored in a data store that have a respective timestamp occurring between the timestamp corresponding to the image and a timestamp corresponding to a reference image, the reference image being one of the plurality of images, and

transmitting the set with the portion to the recipient device.

17. The method of claim 11, wherein the command occurs automatically when a mobile application identified by the user is active.

18. A method comprising:

capturing an image of a screen displayed on a mobile device in response to a command to share the screen;

determining that a first portion of the image matches a second portion of a previously captured image, the second portion being selected for redaction by a user of the mobile device;

initiating a prompt to the user, the prompt suggesting the first portion be redacted;

receiving a response from the user to the prompt; and

transmitting the image to a recipient device selected by the user, wherein the image is transmitted with the first portion redacted when the response indicates the first portion is to be redacted.

19. The method of claim 18, wherein transmitting the image, with the portion redacted pursuant to said response, includes transmitting the image without the portion.

20. The method of claim 18, wherein transmitting the image with the first portion redacted includes obscuring image data in the first portion.

21. The method of claim 18, further comprising:

determining, from a plurality of captured user input actions, each input action having a corresponding timestamp, a set of the user input actions, the set including user input actions occurring prior to a timestamp corresponding to the image and occurring after a timestamp corresponding to a reference image; and

transmit the set with the first portion to the recipient device.