IMAGE CAPTURE WITH PRIVACY PROTECTION

• US 20170300757A1
• Filed: 10/20/2016
• Published: 10/19/2017
• Est. Priority Date: 11/19/2012
• Status: Active Grant

• Current Assignee: Mace Wolf
• Original Assignee: Mace Wolf
• Inventors: Wolf, Mace
• Granted Patent: US 9,940,525 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04L 9/3247 involving digital signatures
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method of providing obscurant data includes receiving image data including an image of a target and receiving a preference setting corresponding to the target. Obscurant data of at least a portion of the image data corresponding to the target are determined using the received preference setting. A method of providing surveillance image data includes capturing image data including an image of a target, querying a database to receive a preference setting corresponding to the target, determining the obscurant data of the portion of the image data, and selectively modifying the received image data according to the determined obscurant data to provide the surveillance image data.

Claims

1. A surveillance drone system comprising

a remotely controllable drone;

a means for capturing image data in said drone, the image data representing a visible region in view of said drone;

a means within said drone for receiving a preference setting corresponding to a target where said target is chosen from a group of individuals present in the visible region when the image data is captured, wherein to receive the preference setting the means;

analyzes the image data to determine that the target bears a first tag that is not unique to the target; and

determines, based at least on the determination that the target bears the first tag, the preference setting indicating that the target is not to be tracked;

a means for determining, using the received preference setting, a portion of the image data corresponding to the target;

a means for selectively modifying the image data to provide surveillance image data comprising obscurant data in place of the portion of the image data corresponding to the target; and

a means remote from said drone for recording said surveillance image data.

2. The surveillance drone system of claim 1, wherein the means for receiving a preference setting further:

receives a unique identifier of the target;

transmits the received unique identifier to a database; and

receives from the database the preference setting corresponding to the unique identifier.

3. The surveillance drone system of claim 1 wherein the means for receiving the preference setting further includes an increased visibility setting associated with a second tag and further:

identifies from the image data a second target bearing the second tag; and

determines second portion of the image data corresponding to the second target;

wherein the means for selectively modifying the image data further produces the surveillance image to include a visible indicator in the second portion.

5. The surveillance drone system of claim 1, wherein the means for receiving the preference setting further receives a unique identifier of the target and a validity period of the unique identifier and determines that the portion of the image data should be obscured only if a present date or time is within the validity period.

6. The surveillance drone system of claim 5, wherein the means for receiving the preference setting further authenticates the unique identifier with respect to selected authority data.

7. The surveillance drone system of claim 6, wherein the authority data includes a cryptographic key and the means for receiving the preference setting authenticates the unique identifier by validating the unique identifier using the cryptographic key and a digital signature of the unique identifier.

4. (canceled)

8. (canceled)

9. A surveillance data processing system comprising one or more hardware computing devices storing computer program instructions that, upon execution, cause the one or more hardware computing devices to:

receive image data captured at a first time by an imaging device of a drone, the image data representing a surveillance area visible to the drone at the first time;

determine, based at least in part on the image data, that a first tag is present in the surveillance area at the first time, wherein the first tag is not a unique identifier of any specific object present in the surveillance area;

identify a first target appearing in the image data, the first target representing a first object bearing the first tag;

identify a first portion of the image data that is associated with the first target;

transform the first portion of the image data using modification data associated with the first tag to produce surveillance image data; and

send the surveillance image data to a display device in communication with the one or more hardware computing devices.

10. The system of claim 9, wherein the computer program instructions, upon execution, further cause the one or more hardware computing devices to:

identify, based on information stored in memory of the one or more hardware computing devices, the first tag as a do-not-track tag; and

to transform the first portion of the image data;

select obscurant data as the modification data; and

apply the obscurant data to the first portion of the image data to produce the surveillance image data comprising an image of the surveillance area at the first time wherein the first object is obscured.

11. The system of claim 10, wherein the computer program instructions, upon execution, further cause the one or more hardware computing devices to:

determine, based at least in part on the image data, that a second tag different from the first tag is present in the surveillance area at the first time;

determine that the second tag is associated with a first preference setting indicating that targets associated with the second tag are to be highlighted in the surveillance image data;

identify a second target appearing in the image data, the second target representing a second object that is present in the surveillance area at the first time and is associated with the second tag;

identify a second portion of the image data that is associated with the second target; and

together with transforming the first portion of the image data, add a highlighting indicator to the second portion of the image data to produce the surveillance image data, such that the first object is obscured and visibility of the second object is increased.

12. The system of claim 11, wherein to transform the first portion and add a highlighting indicator to the second portion of the image data, the one or more hardware computing devices execute the computer program instructions to:

determine that the first portion of the image data and the second portion of the image data overlap at an overlapping portion;

receive information identifying, in the surveillance area, a first location of the first object and a second location of the second object;

determine which of the first location and the second location is closer to the imaging device at the first time;

responsive to a determination that the first location is closer to the imaging device than the second location, modify the overlapping portion with the obscurant data and not the highlighting indicator; and

responsive to a determination that the second location is closer to the imaging device than the first location, modify the overlapping portion to include the highlighting indicator and not the obscurant data.

13. The system of claim 11, further comprising a database accessible by the one or more hardware computing devices and storing a plurality of records each including:

a corresponding unique identifier of a plurality of unique identifiers; and

one or more preference settings describing a desired manipulation of image data;

wherein the computer program instructions, upon execution, further cause the one or more hardware computing devices to;

before receiving the image data;

generate a user interface enabling a user to enter the corresponding one or more preference settings for one or more of the plurality of records;

send the user interface to a user interface system for display to the user;

receive, from the user interface system, user input comprising a first set of preference settings including the first preference setting; and

store the first set of preference settings with a first unique identifier of the plurality of unique identifiers in a first record of the plurality of records; and

to determine that the second tag is associated with the first preference setting;

determine that the second tag includes the first unique identifier; and

query the database using the first unique identifier to retrieve the first set of preference settings from the first record.

14. The system of claim 9, wherein the computer program instructions, upon execution, further cause the one or more hardware computing devices to:

identify, based on information stored in memory of the one or more hardware computing devices, the first tag as a please-track tag; and

to transform the first portion of the image data;

select a highlighting indicator as the modification data; and

apply the highlighting indicator to the first portion of the image data to produce the surveillance image data comprising an image of the surveillance area at the first time wherein visibility of the first object is increased.

15. The system of claim 9, wherein the computer program instructions, upon execution, further cause the one or more hardware computing devices to:

receive sensor data captured at the first time by a sensor of the drone; and

determine, based on the sensor data, that the first tag is present in the surveillance area at the first time, the first tag emitting a signal that is detectable by the sensor.

16. The system of claim 15, wherein the computer program instructions, upon execution, further cause the one or more hardware computing devices to:

determine that the sensor data includes a first value of a first flag, the first value set by a switch on the first tag; and

before transforming the first portion of the image data, determine that the first value indicates the first object bearing the first tag is to be obscured in the surveillance image data.

17. The system of claim 9, wherein the first tag has a color represented in the image data, and to determine that the first tag is present in the surveillance area at the first time, the computer program instructions, upon execution, cause the one or more hardware computing devices to:

identify the color based on data stored in memory of the one or more hardware computing devices; and

determine that a plurality of adjacent pixels in the image data have the color.

18. The system of claim 9, further comprising a database accessible by the one or more hardware computing devices and storing data representing a grid, the grid comprising grid lines that represent geographic coordinates in the surveillance area and define a plurality of grid squares each representing a corresponding region of the surveillance area, the computer program instructions, upon execution, further causing the one or more hardware computing devices to:

associate the grid with the image data;

determine that a location of the first tag is represented in the image data located within a first grid square of the plurality of grid squares; and

identify, as the first portion of the image data, the image data located within the first grid square.

19. The system of claim 18, wherein the database further stores a plurality of masking layers each associated with the grid and each comprising obscurant data within one or more of the grid squares, wherein to transform the first portion of the image data, the one or more hardware computing devices further execute the computer program instructions to:

query the database to retrieve a first masking layer of the plurality of masking layers, the first masking layer having the obscurant data within the first grid square; and

superimpose the first masking layer over the image data to produce the surveillance image data.

20. A surveillance system comprising a database storing a plurality of records each including an identifier and one or more preference settings describing modifications of a recorded image, and one or more hardware computing devices in communication with the database and storing computer program instructions that, upon execution, cause the one or more hardware computing devices to:

communicate with a user interface system to receive a first set of preference settings input into the user interface system by a user;

store the first set of preference settings with a first identifier in a first record of the plurality of records, the first identifier derived from a first tag;

receive surveillance data including a video stream comprising a plurality frames arranged in a sequence and captured, for a duration from a first time to a second time, by an imaging device of a drone, the video stream representing a surveillance area visible to the drone for the duration;

determine, based on the surveillance data, that one or more objects are present in the surveillance area for the duration and each bear the first tag;

determine the first identifier from the surveillance data pertaining to the first tag;

retrieve the first record from the database using the first identifier;

based on the first set of preference settings, obtain modification data pertaining to display of a first object of the one or more objects, the first object being associated with the user;

transform each frame of the plurality of frames using the modification data to produce a surveillance video stream in which the first object is displayed according to the preference settings; and

send the surveillance video stream to a display device in communication with the one or more hardware computing devices.

21. The system of claim 20, wherein the first object is one of the user'"'"'s person, the user'"'"'s house, and the user'"'"'s vehicle, and the computer program instructions, upon execution, further cause the one or more hardware computing devices to:

generate a user interface enabling the user to, by entering the first set of preference settings into the user interface, control, in the surveillance video stream when the first object is bearing the first tag, an appearance of the first object as obscured, highlighted, or unchanged from the received video stream; and

send the user interface to the user interface system for display to the user.

22. The system of claim 21, wherein the user interface further enables the user to:

change the first tag between a do-not-track tag that causes the one or more hardware computing devices to identify obscurant data as the modification data, and a please-track tag that causes the one or more hardware computing devices to identify a highlighting indicator as the modification data; and

identify a validity period, the one or more hardware computing devices being configured to obtain, at a first time, the modification data for obscuring or highlighting the first object in the surveillance video stream only if the first time is within the validity period.