Privacy aware camera and device status indicator system

• US 9,317,721 B2
• Filed: 10/31/2012
• Issued: 04/19/2016
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Plagemann, Christian, Murray, Abraham, Dahlkamp, Hendrik, Kauffmann, Alejandro, Ganapathi, Varun
• Primary Examiner(s): AVERY, JEREMIAH L
• Application Number:
  US13/664,533
• Publication Number:
  US 20140123208A1
• Time in Patent Office: 1,266 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 21/629 to features or functions of an application
• CPC Class Codes: G06F 21/82 Protecting input, output or interconnection devices
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A privacy indicator is provided that shows whether sensor data are being processed in a private or non-private mode. When sensor data are used only for controlling a device locally, it may be in a private mode, which may be shown by setting the privacy indicator to a first color. When sensor data are being sent to a remote site, it may be in a non-private mode, which may be shown by setting the privacy indicator to a second color. The privacy mode may be determined by processing a command in accordance with a privacy policy of determining if the command is on a privacy whitelist, blacklist, greylist or is not present in a privacy command library. A non-private command may be blocked.

Claims

1. A computer-implemented method, comprising:

changing a privacy state of a privacy indicator to indicate that environmental data that is generated by a physical sensor is not being transmitted over a network or stored outside of a trusted portion of memory;

while the privacy state of the privacy indicator indicates that the environmental data that is generated by the sensor is not being transmitted over the network or stored outside of the trusted portion of memory, receiving a command to access the environmental data that is generated by the sensor;

determining that the command is not permitted without first indicating on the privacy indicator that the environmental data that is generated by the sensor is permitted to be transmitted over the network or stored outside of the trusted portion of memory, by a privacy policy that restricts performance of commands that involve transmission of environmental data over the network or storage of environmental data outside of the trusted portion of memory, wherein determining that the command is not permitted comprises verifying a digital signature associated with the command; and

based on the determining that the command is not permitted by the privacy policy that restricts performance of commands that involve transmission of environmental data over the network or storage of environmental data outside of the trusted portion of memory, changing the privacy state of the privacy indicator to indicate that the environmental data that is generated by the sensor is permitted to be transmitted over the network or stored outside of the trusted portion of memory.

2. The computer-implemented method of claim 1, wherein:

the sensor is included in a mobile computing device, and

changing the privacy state of the privacy indicator to indicate that environmental data that is generated by the sensor is not being transmitted over a network or stored outside of a trusted portion of memory, comprises displaying, on an electronic display of the mobile computing device, a visual representation of the privacy state.

3. The computer-implemented method of claim 1, wherein:

the privacy indicator comprises a light emitting diode (LED),

changing the privacy state of the privacy indicator to indicate that environmental data that is generated by the sensor is not being transmitted over a network or stored outside of a trusted portion of memory, comprises one of activating or de-activating the LED, and

changing the privacy state of the privacy indicator to indicate that the environmental data that is generated by the sensor is permitted to be transmitted over a network or stored outside of a trusted portion of memory, comprises activating the LED if the LED was de-activated to set the privacy indicator to indicate operation in the privacy state, or de-activating the LED if the LED was activated to set the privacy indicator to indicate operation in the privacy state.

4. The computer-implemented method of claim 1, wherein the environmental data that is generated by the sensor includes video data or audio data, and

wherein obtaining the environmental video data or audio data while the privacy indicator is set comprises continuously or periodically obtaining environmental data over a period of time.

5. The computer-implemented method of claim 1, wherein the sensor includes a microphone, and wherein receiving the command to access the environmental data that is generated by the sensor comprises sensing, by the microphone, a voice command that provided by a user to access audio data that is generated by the microphone.

6. The computer-implemented method of claim 1, wherein the sensor includes a camera, and wherein receiving the command to access the environmental data that is generated by the sensor comprises receiving one or more images captured by the camera and determining that the one or more images show a user performing gesture that is a command to access image data that is generated by the camera.

7. The computer-implemented method of claim 6, wherein:

the environmental data that is generated by the sensor comprises video data that is obtained using a camera, and

receiving the command to access the environmental data that is generated by the sensor comprises receiving one or more images captured by the camera and determining that the one or more images show placement of an inanimate object that indicates a user-initiated command to access video data that is generated by the camera.

8. The computer-implemented method of claim 1, wherein determining that the command is not permitted by a privacy policy that restricts performance of commands that involve transmission of environmental data over a network or storage of environmental data outside of a trusted portion of memory comprises:

accessing a security policy that classifies private and non-private commands, and

identifying from the security policy that the command is a non-private command whose performance involves transmission of the environmental, or of subsequently obtained environmental data, over the network, or that the performance of the command involves storage of the environmental data, or of subsequently obtained environmental data, outside of a trusted portion of the memory.

9. The computer-implemented method of claim 1, wherein changing a privacy state of the privacy indicator comprises generating user-perceptible tactile feedback.

10. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause performance of operations comprising:

changing a privacy state of a privacy indicator to indicate that environmental data that is generated by a physical sensor is not being transmitted over a network or stored outside of a trusted portion of memory;

while the privacy state of the privacy indicator indicates that the environmental data that is generated by the sensor is not being transmitted over the network or stored outside of the trusted portion of memory, receiving a command to access the environmental data that is generated by the sensor;

determining that the command is not permitted without first indicating on the privacy indicator that the environmental data that is generated by the sensor is permitted to be transmitted over the network or stored outside of the trusted portion of memory, by a privacy policy that restricts performance of commands that involve transmission of environmental data over the network or storage of environmental data outside of the trusted portion of memory, wherein determining that the command is not permitted comprises verifying a digital signature associated with the command; and

based on the determining that the command is not permitted by the privacy policy that restricts performance of commands that involve transmission of environmental data over the network or storage of environmental data outside of the trusted portion of memory, changing the privacy state of the privacy indicator to indicate that the environmental data that is generated by the sensor is permitted to be transmitted over the network or stored outside of the trusted portion of memory.

11. The non-transitory computer-readable storage medium of claim 10, wherein:

the sensor is included in a mobile computing device, and

changing the privacy state of the privacy indicator to indicate that environmental data that is generated by the sensor is not being transmitted over a network or stored outside of a trusted portion of memory, comprises displaying, on an electronic display of the mobile computing device, a visual representation of the privacy state.

12. The non-transitory computer-readable storage medium of claim 10, wherein:

the privacy indicator comprises a light emitting diode (LED),

changing the privacy state of the privacy indicator to indicate that environmental data that is generated by the sensor is not being transmitted over a network or stored outside of a trusted portion of memory, comprises one of activating or de-activating the LED, and

changing the privacy state of the privacy indicator to indicate that the environmental data that is generated by the sensor is permitted to be transmitted over a network or stored outside of a trusted portion of memory, comprises activating the LED if the LED was de-activated to set the privacy indicator to indicate operation in the privacy state, or de-activating the LED if the LED was activated to set the privacy indicator to indicate operation in the privacy state.

13. The non-transitory computer-readable storage medium of claim 10, wherein the environmental data that is generated by the sensor includes video data or audio data, and

wherein obtaining the environmental video data or audio data while the privacy indicator is set comprises continuously or periodically obtaining environmental data over a period of time.

14. The non-transitory computer-readable storage medium of claim 10, wherein the sensor includes a microphone, and wherein receiving the command to access the environmental data that is generated by the sensor comprises sensing, by the microphone, a voice command that provided by a user to access audio data that is generated by the microphone.

15. The non-transitory computer-readable storage medium of claim 10, wherein the sensor includes a camera, and wherein receiving the command to access the environmental data that is generated by the sensor comprises receiving one or more images captured by the camera and determining that the one or more images show a user performing gesture that is a command to access image data that is generated by the camera.

16. The non-transitory computer-readable storage medium of claim 10, wherein:

the environmental data that is generated by the sensor comprises video data that is obtained using a camera, and

receiving the command to access the environmental data that is generated by the sensor comprises receiving one or more images captured by the camera and determining that the one or more images show placement of an inanimate object that indicates a user-initiated command to access video data that is generated by the camera.

17. The non-transitory computer-readable storage medium of claim 10, wherein determining that the command is not permitted by a privacy policy that restricts performance of commands that involve transmission of environmental data over a network or storage of environmental data outside of a trusted portion of memory comprises:

accessing a security policy that classifies private and non-private commands, and

identifying from the security policy that the command is a non-private command whose performance involves transmission of the environmental, or of subsequently obtained environmental data, over the network, or that the performance of the command involves storage of the environmental data, or of subsequently obtained environmental data, outside of a trusted portion of the memory.

18. A device, comprising:

one or more processors;

a physical environmental sensor that is configured to capture environmental video data or audio data, respectively;

a user-perceptible privacy indicator; and

a computer-readable storage medium having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

changing a privacy state of a privacy indicator to indicate that environmental data that is generated by a physical sensor is not being transmitted over a network or stored outside of a trusted portion of memory;

while the privacy state of the privacy indicator indicates that the environmental data that is generated by the sensor is not being transmitted over the network or stored outside of the trusted portion of memory, receiving a command to access the environmental data that is generated by the sensor;

determining that the command is not permitted without first indicating on the privacy indicator that the environmental data that is generated by the sensor is permitted to be transmitted over the network or stored outside of the trusted portion of memory, by a privacy policy that restricts performance of commands that involve transmission of environmental data over the network or storage of environmental data outside of the trusted portion of memory, wherein determining that the command is not permitted comprises verifying a digital signature associated with the command; and

based on the determining that the command is not permitted by the privacy policy that restricts performance of commands that involve transmission of environmental data over the network or storage of environmental data outside of the trusted portion of memory, changing the privacy state of the privacy indicator to indicate that the environmental data that is generated by the sensor is permitted to be transmitted over

the network or stored outside of the trusted portion of memory.