Detecting and responding to an out-of-focus camera in a video analytics system

• US 9,113,143 B2
• Filed: 06/28/2013
• Issued: 08/18/2015
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Seow, Ming-Jung, Urech, Dennis G.
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US13/930,958
• Publication Number:
  US 20140015984A1
• Time in Patent Office: 781 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/154, 382/155, 382/162, 382/168, 382/173, 382/181, 382/199, 382/224, 382/232, 382254-260, 382274-276, 382/286, 382/291, 382/305, 382/312, 382/248, 348/222.1
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Techniques are disclosed for detecting an out-of-focus camera in a video analytics system. In one embodiment, a preprocessor component performs a pyramid image decomposition on a video frame captured by a camera. The preprocessor further determines sharp edge areas, candidate blurry edge areas, and actual blurry edge areas, in each level of the pyramid image decomposition. Based on the sharp edge areas, the candidate blurry edge areas, and actual blurry edge areas, the preprocessor determines a sharpness value and a blurriness value which indicate the overall sharpness and blurriness of the video frame, respectively. Based on the sharpness value and the blurriness value, the preprocessor further determines whether the video frame is out-of-focus and whether to send the video frame to components of a computer vision engine and/or a machine learning engine.

Claims

1. A computer-implemented method for detecting a camera being out-of-focus, the method comprising:

receiving a video frame;

performing a pyramid image decomposition on the video frame, the pyramid image decomposition being performed using a multi-level wavelet transform;

determining sharp edge areas, candidate blurry edge areas, and blurry edge areas in each level of the pyramid image decomposition, wherein the sharp edge areas are areas including edges with either a Dirac or an Astep structure;

determining a sharpness value based on at least the sharp edge areas, the sharpness value indicating an overall sharpness of the video frame;

determining a blurriness value based on at least the blurry edge areas and the candidate blurry edge areas, the blurriness value indicating an overall blurriness of the video frame; and

determining, based on the sharpness value and the blurriness value, whether the video frame is out-of-focus.

2. The method of claim 1, wherein the sharpness value is determined as the ratio of a number of sharp edge areas to total edge areas, and wherein the blurriness value is determined as a ratio of a number of blurry edge areas to candidate blurry edge areas.

3. The method of claim 1, further comprising:

upon determining the video frame is not out-of-focus, sending the video frame to a computer vision engine,

wherein the computer vision engine is configured to analyze video frames captured by the camera and send data and provide an information stream to a machine learning engine, and

wherein the machine learning engine is configured to learn patterns of behavior of objects in a scene based on at least data supplied by the computer vision engine.

4. The method of claim 3, wherein the camera is determined to not be in focus and the video frame is not sent to the computer vision engine if:

(blurriness>

BLUR_LEVEL_—

3) AND (sharpness<

SHARP_LEVEL_—

3) OR

(blurriness>

BLUR_LEVEL_—

2) AND (sharpness<

SHARP_LEVEL_—

2) OR

(blurriness>

BLUR_LEVEL_—

1) AND (sharpness<

SHARP_LEVEL_—

1),

where BLUR_LEVEL_—

3>

BLUR_LEVEL_—

2>

BLUR_LEVEL_—

1, and SHARP_LEVEL_—

3>

SHARP_LEVEL_—

2>

SHARP_LEVEL_—

1.

5. The method of claim 1, further comprising, issuing an alert to a user interface if one or more received frames are determined to be out-of-focus.

6. The method of claim 1, wherein the candidate blurry edge areas are areas including edges with either a roof or a Gstep structure.

7. The method of claim 1, wherein the blurry edge areas are areas which do not include sharp edges in a first level of the pyramid image decomposition and which include blurry edges at one or more higher levels of the pyramid image decomposition.

8. The method of claim 1, wherein the sharp edge areas, the candidate blurry edge areas, and the blurry edge areas are each square or rectangular areas in the received video frames.

9. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for detecting a camera being out-of-focus, the operations comprising:

receiving a video frame;

performing a pyramid image decomposition on the video frame, the pyramid image decomposition being performed using a multi-level wavelet transform;

determining sharp edge areas, candidate blurry edge areas, and blurry edge areas in each level of the pyramid image decomposition, wherein the sharp edge areas are areas including edges with either a Dirac or an Astep structure;

determining a sharpness value based on at least the sharp edge areas, the sharpness value indicating an overall sharpness of the video frame;

determining a blurriness value based on at least the blurry edge areas and the candidate blurry edge areas, the blurriness value indicating an overall blurriness of the video frame; and

determining, based on the sharpness value and the blurriness value, whether the video frame is out-of-focus.

10. The computer-readable storage medium of claim 9, wherein the sharpness value is determined as the ratio of a number of sharp edge areas to total edge areas, and wherein the blurriness value is determined as a ratio of a number of blurry edge areas to candidate blurry edge areas.

11. The computer-readable storage medium of claim 9, the operations further comprising:

upon determining the video frame is not out-of-focus, sending the video frame to a computer vision engine,

wherein the computer vision engine is configured to analyze video frames captured by the camera and send data and provide an information stream to a machine learning engine, and

wherein the machine learning engine is configured to learn patterns of behavior of objects in a scene based on at least data supplied by the computer vision engine.

12. The computer-readable storage medium of claim 11, wherein the camera is determined to not be in focus and the video frame is not sent to the computer vision engine if:

(blurriness>

BLUR_LEVEL_—

3) AND (sharpness<

SHARP_LEVEL_—

3) OR

(blurriness>

BLUR_LEVEL_—

2) AND (sharpness<

SHARP_LEVEL_—

2) OR

(blurriness>

BLUR_LEVEL_—

1) AND (sharpness<

SHARP_LEVEL_—

1),

where BLUR_LEVEL_—

3>

BLUR_LEVEL_—

2>

BLUR_LEVEL_—

1, and SHARP_LEVEL_—

3>

SHARP_LEVEL_—

2>

SHARP_LEVEL_—

1.

13. The computer-readable storage medium of claim 9, the operations further comprising, issuing an alert to a user interface if one or more received frames are determined to be out-of-focus.

14. The computer-readable storage medium of claim 9, wherein the candidate blurry edge areas are areas including edges with roof or Gstep structures.

15. The computer-readable storage medium of claim 9, wherein the blurry edge areas are areas which do not include sharp edges in a first level of the pyramid image decomposition and which include blurry edges at one or more high levels of the pyramid image decomposition.

16. The computer-readable storage medium of claim 9, wherein the sharp edge areas, the candidate blurry edge areas, and the blurry edge areas are each square or rectangular areas in the received video frames.

17. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for detecting a camera being out-of-focus, the operations comprising;

receiving a video frame,

performing a pyramid image decomposition on the video frame, the pyramid image decomposition being performed using a multi-level wavelet transform,

determining sharp edge areas, candidate blurry edge areas, and blurry edge areas in each level of the pyramid image decomposition, wherein the sharp edge areas are areas including edges with either a Dirac or an Astep structure,

determining a sharpness value based on at least the sharp edge areas, the sharpness value indicating an overall sharpness of the video frame,

determining a blurriness value based on at least the blurry edge areas and the candidate blurry edge areas, the blurriness value indicating an overall blurriness of the video frame, and

determining, based on the sharpness value and the blurriness value, whether the video frame is out-of-focus.