METHODS AND SYSTEMS OF PERFORMING ADAPTIVE MORPHOLOGY OPERATIONS IN VIDEO ANALYTICS

• US 20180033152A1
• Filed: 09/12/2016
• Published: 02/01/2018
• Est. Priority Date: 08/01/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Wang, Lei, Gao, Jinglun, Bi, Ning
• Granted Patent: US 10,223,590 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/30 Erosion or dilatation, e.g. thinning
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for performing content-adaptive morphology operations. A first erosion function can be performed on a foreground mask of a video frame, including setting one or more foreground pixels of the frame to one or more background pixels. A temporary foreground mask can be generated based on the first erosion function being performed on the foreground mask. One or more connected components can be generated for the frame by performing connected component analysis to connect one or more neighboring foreground pixels. A complexity of the frame (or of the foreground mask of the frame) can be determined by comparing a number of the one or more connected components to a threshold number. A second erosion function can be performed on the temporary foreground mask when the number of the one or more connected components is higher than the threshold number. The one or more connected components can be output for blob processing when the number of the one or more connected components is lower than the threshold number.

Claims

1. A method of performing content-adaptive morphology operations, the method comprising:

performing a first erosion function on a foreground mask of a frame, the first erosion function setting one or more foreground pixels of the foreground mask to one or more background pixels;

determining a complexity of the foreground mask; and

determining whether to perform one or more additional erosion functions for the frame based on the determined complexity of the foreground mask.

2. The method of claim 1, further comprising:

generating one or more connected components by performing connected component analysis on foreground pixels of the foreground mask to connect one or more neighboring foreground pixels; and

wherein the complexity of the foreground mask is determined by comparing a number of the one or more connected components to a threshold number.

3. The method of claim 2, further comprising:

performing a second erosion function on a temporary foreground mask when the number of the one or more connected components is higher than the threshold number, the temporary foreground mask including the foreground mask after the first erosion function is performed on the foreground mask, and the second erosion function setting one or more foreground pixels of the temporary foreground mask to one or more background pixels.

4. The method of claim 3, further comprising:

generating one or more connected components using the temporary foreground mask after the second erosion function is performed;

determining a complexity of the temporary foreground mask after the second erosion function is performed, the complexity of the temporary foreground mask being determined by comparing a number of the one or more connected components of the temporary foreground mask to the threshold number;

determining the number of the one or more connected components of the temporary foreground mask is below the threshold number; and

outputting the one or more connected components of the temporary foreground mask for blob processing, wherein a blob includes at least one connected component.

5. The method of claim 3, wherein performing the first erosion function includes performing an erosion operation on the foreground mask, and wherein performing the second erosion function includes re-performing the erosion operation on the temporary foreground mask.

6. The method of claim 3, wherein performing the first erosion function includes performing a first erosion operation on the foreground mask, and wherein performing the second erosion function includes performing a second erosion operation on the temporary foreground mask, the first erosion operation being different than the second erosion operation.

7. The method of claim 2, further comprising:

outputting the one or more connected components for blob processing when the number of the one or more connected components is lower than the threshold number, wherein a blob includes at least one connected component.

8. The method of claim 2, wherein a blob includes at least one connected component, and wherein only connected components of blobs smaller than a size threshold are included in the number of the one or more connected components compared to the threshold number.

9. The method of claim 2, further comprising:

performing at least one dilation function on the foreground mask, the at least one dilation function setting one or more background pixels of the foreground mask to one or more foreground pixels, wherein the at least one dilation function sets a background pixel in the foreground mask to a foreground pixel when at least one neighboring pixel of the background pixel includes a foreground pixel.

10. The method of claim 9, wherein the at least one dilation function is performed after the first erosion function is performed on the foreground mask and before the connected component analysis is performed on the foreground pixels of the foreground mask.

11. The method of claim 9, further comprising:

performing multiple dilation functions on the foreground mask, wherein the multiple dilation functions are performed after the first erosion function is performed on the foreground mask and before the connected component analysis is performed on the foreground pixels of the foreground mask.

12. The method of claim 1, wherein performing the first erosion function includes performing a weak erosion function, the weak erosion function setting a foreground pixel of the foreground mask to a background pixel when two or more neighboring pixels of the foreground pixel include background pixels.

13. The method of claim 1, wherein performing the first erosion function includes setting a foreground pixel of the foreground mask to a background pixel when a single neighboring pixel of the foreground pixel includes a background pixel.

14. An apparatus comprising:

a memory configured to store video data; and

a processor configured to;

perform a first erosion function on a foreground mask of a frame, the first erosion function setting one or more foreground pixels of the foreground mask to one or more background pixels;

determine a complexity of the foreground mask; and

determine whether to perform one or more additional erosion functions for the frame based on the determined complexity of the foreground mask.

15. The apparatus of claim 14, wherein the processor is further configured to:

generate one or more connected components by performing connected component analysis on foreground pixels of the foreground mask to connect one or more neighboring foreground pixels; and

wherein the complexity of the foreground mask is determined by comparing a number of the one or more connected components to a threshold number.

16. The apparatus of claim 15, wherein the processor is further configured to:

perform a second erosion function on a temporary foreground mask when the number of the one or more connected components is higher than the threshold number, the temporary foreground mask including the foreground mask after the first erosion function is performed on the foreground mask, and the second erosion function setting one or more foreground pixels of the temporary foreground mask to one or more background pixels.

17. The apparatus of claim 16, wherein the processor is further configured to:

generate one or more connected components using the temporary foreground mask after the second erosion function is performed;

determine a complexity of the temporary foreground mask after the second erosion function is performed, the complexity of the temporary foreground mask being determined by comparing a number of the one or more connected components of the temporary foreground mask to the threshold number;

determine the number of the one or more connected components of the temporary foreground mask is below the threshold number; and

output the one or more connected components of the temporary foreground mask for blob processing, wherein a blob includes at least one connected component.

18. The apparatus of claim 16, wherein performing the first erosion function includes performing an erosion operation on the foreground mask, and wherein performing the second erosion function includes re-performing the erosion operation on the temporary foreground mask.

19. The apparatus of claim 16, wherein performing the first erosion function includes performing a first erosion operation on the foreground mask, and wherein performing the second erosion function includes performing a second erosion operation on the temporary foreground mask, the first erosion operation being different than the second erosion operation.

20. The apparatus of claim 15, wherein the processor is further configured to:

output the one or more connected components for blob processing when the number of the one or more connected components is lower than the threshold number, wherein a blob includes at least one connected component.

21. The apparatus of claim 15, wherein a blob includes at least one connected component, and wherein only connected components of blobs smaller than a size threshold are included in the number of the one or more connected components compared to the threshold number.

22. The apparatus of claim 15, wherein the processor is further configured to:

perform at least one dilation function on the foreground mask, the at least one dilation function setting one or more background pixels of the foreground mask to one or more foreground pixels, wherein the at least one dilation function sets a background pixel in the foreground mask to a foreground pixel when at least one neighboring pixel of the background pixel includes a foreground pixel.

23. The apparatus of claim 22, wherein the at least one dilation function is performed after the first erosion function is performed on the foreground mask and before the connected component analysis is performed on the foreground pixels of the foreground mask.

24. The apparatus of claim 22, wherein the processor is further configured to:

perform multiple dilation functions on the foreground mask, wherein the multiple dilation functions are performed after the first erosion function is performed on the foreground mask and before the connected component analysis is performed on the foreground pixels of the foreground mask.

25. The apparatus of claim 14, wherein performing the first erosion function includes performing a weak erosion function, the weak erosion function setting a foreground pixel of the foreground mask to a background pixel when two or more neighboring pixels of the foreground pixel include background pixels.

26. The apparatus of claim 14, wherein performing the first erosion function includes setting a foreground pixel of the foreground mask to a background pixel when a single neighboring pixel of the foreground pixel includes a background pixel.

27. A computer readable medium having stored thereon instructions that when executed by a processor perform a method, including:

performing a first erosion function on a foreground mask of a frame, the first erosion function setting one or more foreground pixels of the foreground mask to one or more background pixels;

determining a complexity of the foreground mask; and

determining whether to perform one or more additional erosion functions for the frame based on the determined complexity of the foreground mask.

28. The computer readable medium of claim 27, further comprising:

generating one or more connected components by performing connected component analysis on foreground pixels of the foreground mask to connect one or more neighboring foreground pixels; and

wherein the complexity of the foreground mask is determined by comparing a number of the one or more connected components to a threshold number.

29. The computer readable medium of claim 28, further comprising:

performing a second erosion function on a temporary foreground mask when the number of the one or more connected components is higher than the threshold number, the temporary foreground mask including the foreground mask after the first erosion function is performed on the foreground mask, and the second erosion function setting one or more foreground pixels of the temporary foreground mask to one or more background pixels.

30. The computer readable medium of claim 29, further comprising:

generating one or more connected components using the temporary foreground mask after the second erosion function is performed;

determining a complexity of the temporary foreground mask after the second erosion function is performed, the complexity of the temporary foreground mask being determined by comparing a number of the one or more connected components of the temporary foreground mask to the threshold number;

determining the number of the one or more connected components of the temporary foreground mask is below the threshold number; and

outputting the one or more connected components of the temporary foreground mask for blob processing, wherein a blob includes at least one connected component.