Privacy aware camera and device status indicator system

• US 9,699,217 B2
• Filed: 04/07/2016
• Issued: 07/04/2017
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kauffmann, Alejandro Jose, Ganapathi, Varun, Murray, Abraham, Dahlkamp, Hendrik, Plagemann, Christian
• Primary Examiner(s): AVERY, JEREMIAH L
• Application Number:
  US15/093,340
• Publication Number:
  US 20160226917A1
• Time in Patent Office: 453 Days
• Field of Search: None
• CPC Class Codes: G06F 21/629 to features or functions of an application
• CPC Class Codes: G06F 21/82 Protecting input, output or interconnection devices
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A privacy indicator is provided that shows whether sensor data are being processed in a private or non-private mode. When sensor data are used only for controlling a device locally, it may be in a private mode, which may be shown by setting the privacy indicator to a first color. When sensor data are being sent to a remote site, it may be in a non-private mode, which may be shown by setting the privacy indicator to a second color. The privacy mode may be determined by processing a command in accordance with a privacy policy of determining if the command is on a privacy whitelist, blacklist, greylist or is not present in a privacy command library. A non-private command may be blocked.

Claims

1. A computer-implemented method comprising:

obtaining, and storing in a destination in memory that is characterized as a trusted memory destination, audio or video data that is generated by a webcam while a visual privacy indicator associated with the webcam visually indicates a private state;

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data;

accessing a privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible;

determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is identified based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination; and

in response to determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is identified based at least on the audio or video data involves providing audio or video data to the destination in memory that is not characterized as a trusted memory destination, causing the visual privacy indicator associated with the webcam to indicate a non-private state.

2. The method of claim 1, wherein determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is determined based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination comprises identifying the command as restricted by a privacy policy that restricts performance of commands that involve providing audio or video data to a destination in memory that is not characterized as a trusted memory destination.

3. The method of claim 1, wherein:

the webcam is included in a mobile computing device, and

causing the visual privacy indicator associated with the webcam to indicate a non-private state comprises displaying, on an electronic display of the mobile computing device, a visual representation of the non-private state.

4. The method of claim 1, wherein:

the visual privacy indicator comprises a light emitting diode (LED),

the visual privacy indicator associated with the webcam visually indicates a private state by activating or de-activating the LED, and

causing the visual privacy indicator associated with the webcam to visually indicate a non-private state comprises activating the LED if the LED was de-activated to set the visual privacy indicator to visually indicate the non-private state, or de-activating the LED if the LED was activated to set the visual privacy indicator to visually indicate the non-private state.

5. The method of claim 1, wherein:

the webcam includes a microphone, and

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data comprises sensing, by the microphone, a voice command to access the audio or video data.

6. The method of claim 1, further comprising:

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data comprising receiving one or more images captured by the webcam and identifying that the one or more images indicate a gesture that is a command to access the audio or video data.

7. The method of claim 1, further comprising:

obtaining audio or video data that is generated by the webcam while the visual privacy indicator associated with the webcam visually indicates the private state comprises continuously or periodically obtaining audio or video data over a period of time.

8. The method of claim 1, wherein determining that the command that is identified based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination comprises determining that the destination that is not characterized as a trusted memory destination is not included in the database of trusted memory destinations.

9. A device, comprising:

one or more processors;

a webcam configured to capture audio or video data;

a user-perceptible visual privacy indicator; and

a memory device storing instructions that, when executed by the one or more processors, cause performance of operations comprising;

obtaining, and storing in a destination in memory that is characterized as a trusted memory destination, audio or video data that is generated by a webcam while a visual privacy indicator associated with the webcam visually indicates a private state;

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data;

accessing a privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible;

determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is identified based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination; and

in response to determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible that the command that is identified based at least on the audio or video data involves providing audio or video data to the destination in memory that is not characterized as a trusted memory destination, causing the visual privacy indicator associated with the webcam to indicate a non-private state.

10. The device of claim 9, wherein determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is determined based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination comprises identifying the command as restricted by a privacy policy that restricts performance of commands that involve providing audio or video data to a destination in memory that is not characterized as a trusted memory destination.

11. The device of claim 9, wherein:

the webcam is included in a mobile computing device, and

causing the visual privacy indicator associated with the webcam to indicate a non-private state comprises displaying, on an electronic display of the mobile computing device, a visual representation of the non-private state.

12. The device of claim 9, wherein:

the visual privacy indicator comprises a light emitting diode (LED),

the visual privacy indicator associated with the webcam visually indicates a private state by activating or de-activating the LED, and

causing the visual privacy indicator associated with the webcam to visually indicate a non-private state comprises activating the LED if the LED was de-activated to set the visual privacy indicator to visually indicate the non-private state, or de-activating the LED if the LED was activated to set the visual privacy indicator to visually indicate the non-private state.

13. The device of claim 9, wherein:

the webcam includes a microphone, and

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data comprises sensing, by the microphone, a voice command to access the audio or video data.

14. The device of claim 9, further comprising:

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data comprises receiving one or more images captured by the webcam and identifying that the one or more images indicate a gesture that is a command to access the audio or video data.

15. The device of claim 9, further comprising:

obtaining audio or video data that is generated by the webcam while the visual privacy indicator associated with the webcam visually indicates the private state comprises continuously or periodically obtaining audio or video data over a period of time.

16. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause performance of operations comprising:

obtaining, and storing in a destination in memory that is characterized as a trusted memory destination, audio or video data that is generated by a webcam while a visual privacy indicator associated with the webcam visually indicates a private state;

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data;

accessing a privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible;

determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is identified based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination; and

in response to determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is identified based at least on the audio or video data involves providing audio or video data to the destination in memory that is not characterized as a trusted memory destination, causing the visual privacy indicator associated with the webcam to indicate a non-private state.

17. The non-transitory computer-readable storage medium of claim 16, wherein determining, while the visual privacy indicator associated with the webcam visually indicates the private state, using the accessed privacy policy that (i) includes a database of trusted memory destinations and (ii) identifies commands that are permissible and commands that are not permissible, that the command that is determined based at least on the audio or video data involves providing audio or video data to a destination in memory that is not characterized as a trusted memory destination comprises identifying the command as restricted by a privacy policy that restricts performance of commands that involve providing audio or video data to a destination in memory that is not characterized as a trusted memory destination.

18. The non-transitory computer-readable storage medium of claim 16, wherein:

the webcam is included in a mobile computing device, and

causing the visual privacy indicator associated with the webcam to indicate a non-private state comprises displaying, on an electronic display of the mobile computing device, a visual representation of the non-private state.

19. The non-transitory computer-readable storage medium of claim 16, wherein:

the visual privacy indicator comprises a light emitting diode (LED),

the visual privacy indicator associated with the webcam visually indicates a private state by activating or de-activating the LED, and

causing the visual privacy indicator associated with the webcam to visually indicate a non-private state comprises activating the LED if the LED was de-activated to set the visual privacy indicator to visually indicate the non-private state, or de-activating the LED if the LED was activated to set the visual privacy indicator to visually indicate the non-private state.

20. The non-transitory computer-readable storage medium of claim 16, wherein:

identifying, while the visual privacy indicator associated with the webcam visually indicates the private state, a command based at least on the audio or video data comprises receiving one or more images captured by the webcam and identifying that the one or more images indicate a gesture that is a command to access the audio or video data.