Methods and apparatus for using video analytics to detect regions for privacy protection within images from moving cameras

• US 10,565,395 B2
• Filed: 01/27/2017
• Issued: 02/18/2020
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: KiwiSecurity Software GmbH
• Original Assignee: KiwiSecurity Software GmbH
• Inventors: Matusek, Florian, Kraus, Klemens, Sutor, Stephan, Erdelyi, Adam, Zankl, Georg
• Primary Examiner(s): Harvey, David E
• Application Number:
  US15/418,176
• Publication Number:
  US 20170220816A1
• Time in Patent Office: 1,117 Days
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G08B 13/19686 Interfaces masking personal details for privacy, e.g. blurring faces, vehicle license plates
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

In some embodiments, an apparatus includes a memory and a processor. The processor is configured to receive a set of images associated with a video recorded by a moving or a non-moving camera. The processor is configured to detect a structure of a region of interest from a set of regions of interest in an image from the set of images. The processor is configured to classify the structure into a geometric class from a set of predefined geometric classes using machine learning techniques. The processor is configured to alter the region of interest to generate an altered image when the geometric class is associated with an identity of a person, such that privacy associated with the identity of the person is protected. The processor is configured to send the altered image to a user interface or store the altered image in a standardized format.

Claims

1. An apparatus, comprising:

a memory; and

a processor operatively coupled to the memory, the processor configured to;

receive a set of images associated with a video recorded by a camera,

detect a structure of a region of interest from a plurality of regions of interest in an image from the set of images,

classify the structure into a geometric class from a set of predefined geometric classes,

alter the region of interest to generate an altered image when the geometric class is associated with an identity of a person, such that privacy associated with the identity of the person is protected, and

at least one of send the altered image to a user interface or store the altered image in a standardized format;

wherein the processor is configured to classify the structure into the geometric class based on detecting the structure of the region of interest as including a set of substantially straight lines, the geometric class being associated with a background and not associated with the identity of the person.

2. The apparatus of claim 1, wherein the processor is configured to:

define a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image,

detect a structure of each region of interest in each grid cell from the plurality of grid cells, and

classify each grid cell into the set of predefined geometric classes using a set of parameters stored in the memory.

3. The apparatus of claim 1, wherein the processor is configured to alter the region of interest by scrambling the region of interest.

4. The apparatus of claim 1, wherein the geometric class is associated with the identity of the person based on the region of interest being substantially non-structured.

5. The apparatus of claim 1, wherein the processor is configured to classify the structure into the geometric class using a set of parameters stored in the memory,

the set of parameters is generated by manually selecting geometric features from a set of sample images and classifying the geometric features into at least one geometric class from the set of predefined geometric classes.

6. The apparatus of claim 1, wherein the set of images are recorded by the camera in a known environment.

7. The apparatus of claim 1, wherein the set of images is associated with audio data.

18. The apparatus of claim 1, wherein the processor is configured to classify the structure into the geometric class based on detecting the structure of the region of interest as being substantially non-structured, the geometric class being associated with the identity of the person.

19. The apparatus of claim 1, wherein the structure includes at least one of an edge or a texture.

20. The apparatus of claim 1, wherein the processor is configured to:

define a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image,

detect a structure within in each grid cell from the plurality of grid cells, the structure being one or more substantially straight lines or one or more substantially non-structured lines, and

classify each grid cell from the plurality of grid cells into a geometric class from the set of predefined geometric classes using a set of parameters stored in the memory and based on the structure detected in that grid cell.

8. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:

receive a set of images associated with a video recorded by a camera;

detect a structure of a region of interest from a plurality of regions of interest in an image from the set of images;

classify the structure into a geometric class from a set of predefined geometric classes;

alter the region of interest to generate an altered image when the geometric class is associated with an identity of a person, such that privacy associated with the identity of the person is protected; and

at least one of send the altered image to a user interface or store the altered image in a standardized format;

wherein the processor is configured to classify the structure into the geometric class based on detecting the structure of the region of interest as including a set of substantially straight lines, the geometric class being associated with a background and not associated with the identity of the person.

9. The non-transitory processor-readable medium of claim 8, wherein the code further comprises code to cause the processor to:

define a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image;

the code to cause the processor to detect includes code to cause the processor to detect a structure of each region of interest in each grid cell from the plurality of grid cells; and

the code to cause the processor to classify includes code to the cause the processor to classify each grid cell into the set of predefined geometric classes using a set of parameters stored in the memory.

10. The non-transitory processor-readable medium of claim 8, wherein the code to cause the processor to alter includes code to cause the processor to:

alter the region of interest by scrambling the region of interest.

11. The non-transitory processor-readable medium of claim 8, wherein the geometric class is at least one of (1) associated with a background and not associated with the identity of the person based on the structure of the region of interest being detected as a set of substantially straight lines, or (2) associated with the identity of the person based on the region of interest being detected as substantially non-structured.

12. The non-transitory processor-readable medium of claim 8, wherein the code to cause the processor to classify includes code to cause the processor to classify the structure into the geometric class using a set of parameters stored in the memory, the set of parameters being generated by manually selecting geometric features from a set of sample images and classifying the geometric features into at least one geometric class from the set of predefined geometric classes.

13. The non-transitory processor-readable medium of claim 8, wherein the set of images are recorded by the camera in a known environment.

14. The non-transitory processor-readable medium of claim 8, wherein the set of images is associated with audio data.

21. The non-transitory processor-readable medium of claim 8, wherein the code further comprises code to cause the processor to:

define a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image,

detect a structure within in each grid cell from the plurality of grid cells, the structure being one or more substantially straight lines or one or more substantially non-structured lines, and

classify each grid cell from the plurality of grid cells into a geometric class from the set of predefined geometric classes using a set of parameters stored in the memory and based on the structure detected in that grid cell.

15. A method, comprising:

receiving a set of images associated with a video recorded by a camera;

detecting a structure of a region of interest from a plurality of regions of interest in an image from the set of images;

classifying the structure into a geometric class from a set of predefined geometric classes;

scrambling the region of interest to generate an altered image when the geometric class is associated with an identity of a person, such that privacy associated with the identity of the person is protected; and

at least one of sending the altered image to a user interface or storing the altered image;

wherein the processor is configured to classify the structure into the geometric class based on detecting the structure of the region of interest as including a set of substantially straight lines, the geometric class being associated with a background and not associated with the identity of the person.

16. The method of claim 15, further comprising:

defining a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image,

the detecting including detecting a structure of each region of interest in each grid cell from the plurality of grid cells; and

the classifying including classifying each grid cell into the set of predefined geometric classes using a set of parameters stored in the memory.

17. The method of claim 15, wherein the geometric class is at least one of (1) associated with a background and not associated with the identity of the person when the structure of the region of interest is a set of substantially straight lines, or (2) associated with the identity of the person when the region of interest is substantially non-structured.

22. The method of claim 15, further comprising:

defining a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image,

detecting a structure within in each grid cell from the plurality of grid cells, the structure being one or more substantially straight lines or one or more substantially non-structured lines, and

classifying each grid cell from the plurality of grid cells into a geometric class from the set of predefined geometric classes using a set of parameters stored in the memory and based on the structure detected in that grid cell.