METHODS AND APPARATUS FOR USING VIDEO ANALYTICS TO DETECT REGIONS FOR PRIVACY PROTECTION WITHIN IMAGES FROM MOVING CAMERAS

• US 20170220816A1
• Filed: 01/27/2017
• Published: 08/03/2017
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: KiwiSecurity Software GmbH
• Original Assignee: KiwiSecurity Software GmbH
• Inventors: MATUSEK, Florian, KRAUS, Klemens, SUTOR, Stephan, ERDLYI, dm, ZANKL, Georg
• Granted Patent: US 10,565,395 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G08B 13/19686 Interfaces masking personal details for privacy, e.g. blurring faces, vehicle license plates
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

In some embodiments, an apparatus includes a memory and a processor. The processor is configured to receive a set of images associated with a video recorded by a moving or a non-moving camera. The processor is configured to detect a structure of a region of interest from a set of regions of interest in an image from the set of images. The processor is configured to classify the structure into a geometric class from a set of predefined geometric classes using machine learning techniques. The processor is configured to alter the region of interest to generate an altered image when the geometric class is associated with an identity of a person, such that privacy associated with the identity of the person is protected. The processor is configured to send the altered image to a user interface or store the altered image in a standardized format.

Claims

1. An apparatus, comprising:

a memory; and

a processor operatively coupled to the memory, the processor configured to;

receive a set of images associated with a video recorded by a moving or a non-moving camera,

detect a structure of a region of interest from a plurality of regions of interest in an image from the set of images,

classify the structure into a geometric class from a set of predefined geometric classes using machine learning techniques,

alter the region of interest to generate an altered image when the geometric class is associated with an identity of a person, such that privacy associated with the identity of the person is protected, and

at least one of send the altered image to a user interface or store the altered image in a standardized format.

2. The apparatus of claim 1, wherein the processor is configured to:

define a grid placed over the set of images, the grid including a plurality of grid cells, each grid cell from the plurality of grid cells associated with a different region of interest from the plurality of regions of interest in the image,

detect a structure of each region of interest in each grid cell from the plurality of grid cells, and

classify each grid cell into the set of predefined geometric classes using a set of parameters stored in the memory.

3. The apparatus of claim 1, wherein the processor is configured to alter the region of interest by scrambling the region of interest.

4. The apparatus of claim 1, wherein the geometric class is associated with a background and not associated with the identity of the person when the structure of the region of interest is a set of substantially straight lines.

5. The apparatus of claim 1, wherein the geometric class is associated with the identity of the person when the region of interest is substantially non-structured.

6. The apparatus of claim 1, wherein the processor is configured to classify the structure into the geometric class using a set of parameters stored in the memory.

7. The apparatus of claim 1, wherein the processor is configured to classify the structure into the geometric class using a set of parameters stored in the memory,

the set of parameters is generated by manually selecting geometric features from a set of sample images and classifying the geometric features into at least one geometric class from the set of predefined geometric classes.

8. The apparatus of claim 1, wherein the processor is configured to:

retrieve a set of parameters stored in the memory,

train a machine learning model using the set of parameters and the machine learning techniques, and

classify the structure into the geometric class using the machine learning model.

9. The apparatus of claim 1, wherein the set of images are recorded by the moving or the non-moving camera in a known environment.

10. The apparatus of claim 1, wherein the set of images includes audio data.

11. A method, comprising:

receiving a plurality of images associated with a video recorded by a moving camera;

selecting a plurality of feature points in an image from the plurality of images, each feature point from the plurality of feature points being associated with a different region of interest from a plurality of regions of interest in the image, the plurality of feature points including a first set of feature points and a second set of feature points;

defining a first movement direction of each feature point from the first set of feature points within the plurality of images;

defining a second movement direction of each feature point from the second set of feature points within the plurality of images;

when the first set of feature points includes a greater number of feature points than the second set of feature points;

altering the region of interest associated with each feature point from the second set of feature points to produce an altered image to protect privacy-related information included in the region of interest associated with each feature point from the second set of feature points; and

at least one of sending the altered image to a user interface or storing the altered image in a standardized format.

12. The method of claim 11, wherein selecting the plurality of feature points is based on a texture associated with each region of interest from the plurality of regions of interest in the image.

13. The method of claim 11, further comprising:

defining a grid associated with each image from the plurality of images, the grid including a plurality of grid cells; and

selecting a center of each grid cell from the plurality of grid cells to define the plurality of feature points.

14. The method of claim 11, further comprising:

defining a grid associated with each image from the plurality of images, the grid including a plurality of grid cells;

selecting a center of each grid cell from the plurality of grid cells to define the plurality of feature points;

the defining the first movement direction of each feature point from the first set of feature points including defining a first trajectory of each feature point from the first set of feature points;

the defining the second movement direction of each feature point from the second set of feature points including defining a second trajectory of each feature point from the second set of feature points;

determining a first set of characteristics of the first trajectory and a second set of characteristics of the second trajectory; and

when the first set of characteristics is less coherent than the second set of characteristics, altering the region of interest associated with the first set of feature points to produce the altered image to protect privacy-related information included in the region of interest associated with the first set of feature points.

15. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:

receive a plurality of images associated with video data recorded by a moving camera, each image from the plurality of images including a set of regions of interest;

detect, based on a characteristic associated with a person, a first region of interest from the set of regions of interest and associated with an identity of the person;

track the first region of interest within the plurality of images;

determine a first movement direction associated with a first set of feature points within a second region of interest from the set of regions of interest, the first movement direction being different from a second movement direction associated with a second set of feature points within a third region of interest from the set of regions of interest, the second region of interest including a fewer number of feature points than the third region of interest;

detect a fourth region of interest within the plurality of images and associated with the person, based on classifying a portion of the image within the fourth region of interest into a geometric class different from a geometric class of a background structure within the plurality of images;

alter at least one of the first region of interest, the second region of interest, or the fourth region of interest to produce an altered plurality of images based on a confidence score of privacy protection meeting a predefined criterion for the at least one of the first region of interest, the second region of interest, or the fourth region of interest; and

at least one of send the altered plurality of images to the user interface or store the altered plurality of images in a standardized format.

16. The non-transitory processor-readable medium of claim 15, wherein the code further comprises code to cause the processor to detect the first region of interest based on a set of parameters and machine learning techniques.

17. The non-transitory processor-readable medium of claim 15, wherein the code further comprises code to cause the processor to detect the fourth region of interest based on a set of parameters and machine learning techniques.

18. The non-transitory processor-readable medium of claim 15, wherein the altered plurality of images is a first altered plurality of images, the code further comprises code to cause the processor to:

determine a fifth region of interest within the second region of interest and associated with the identity of the person;

alter the fifth region of interest to produce a second altered plurality of images; and

at least one of send the second altered plurality of images to the user interface or store the second altered plurality of images in a standardized format.

19. The non-transitory processor-readable medium of claim 15, wherein the code further comprises code to cause the processor to detect the first region of interest based at least in part on a physical motion limit of the person.

20. The non-transitory processor-readable medium of claim 15, wherein the code further comprises code to cause the processor to alter the at least one of the first region of interest, the second region of interest, or the fourth region of interest by at least one of blurring, pixelating, or blanking, the at least one of the first region of interest, the second region of interest, or the fourth region of interest.