HIERARCHICAL SUDDEN ILLUMINATION CHANGE DETECTION USING RADIANCE CONSISTENCY WITHIN A SPATIAL NEIGHBORHOOD

• US 20100150471A1
• Filed: 12/16/2008
• Published: 06/17/2010
• Est. Priority Date: 12/16/2008
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Giant Gray, Inc.
• Inventors: SAITWAL, KISHOR ADINATH, COBB, WESLEY KENNETH
• Granted Patent: US 9,373,055 B2
• Time in Patent Office: Days
• US Class Current: 382/274
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model

Abstract

Techniques are disclosed for detecting sudden illumination changes using radiance consistency within a spatial neighborhood. A background/foreground (BG/FG) component of a behavior recognition system may be configured to generate a background image depicting a scene background. Further, the (BG/FG) component may periodically evaluate a current video frame to determine whether a sudden illumination change has occurred. A sudden illumination change occurs when scene lighting changes dramatically from one frame to the next (or over a small number of frames).

Claims

1. A computer-implemented method for detecting a sudden illumination change (SIC) in a sequence of video frames, comprising:

receiving a current video frame, of the sequence of video frames;

identifying one or more sets of pixels in the current video frame, wherein each set of pixels is classified as depicting a foreground object;

for each pixel in at least one of the sets of pixels;

identifying a radiance value for the pixel and for each pixel within a spatial neighborhood of the pixel,

determining, for the pixel, whether the radiance values for the pixel and the pixels in the spatial neighborhood indicate that a sudden illumination change has occurred in a scene depicted by the sequence of video frames; and

upon determining that a majority of pixels in a first one of the sets of pixels indicate that a sudden illumination change has occurred, reclassifying the pixels in the first set of pixels as depicting a background of the scene depicted by the sequence of video frames.

2. The computer-implemented method of claim 1, further comprising:

updating a set of pixels in a background image of the scene corresponding to the pixels in the first set of pixels.

3. The computer-implemented method of claim 1, wherein the spatial neighborhood for a given pixel comprises a 3×

3 grid of pixels with the given pixel in the center of the 3×

3 grid.

4. The computer-implemented method of claim 1, wherein determining, for the pixel, whether the radiance values for the pixel and the pixels in the spatial neighborhood indicate that a sudden illumination change has occurred, comprises:

identifying, in a background image, a pixel corresponding to the pixel in the current video frame;

determining an intensity difference between the corresponding pixel and each pixel in a spatial neighborhood of the corresponding pixel;

determining an intensity difference between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame;

normalizing the intensity differences determined between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame;

determining an absolute difference between each respective pixel in the spatial neighborhood of the pixel in the current video frame and each respective pixel in the spatial neighborhood of the corresponding pixel;

for each pixel in the spatial neighborhood of the pixel in the current video frame, determining whether the absolute difference is below a first specified threshold, and

if so, incrementing a counter representing how many pixels in the spatial neighborhood are below the first specified threshold;

upon determining the counter exceeds a second threshold, setting a status of the pixel as being an SIC candidate pixel, indicating that the pixel was classified as depicting a scene foreground object due to a sudden illumination change in the scene depicted by the sequence of video frames.

5. The computer-implemented method of claim 4, further comprising:

upon determining a count of pixels classified as depicting the foreground object which have a status of SIC candidate pixel is below a third threshold, setting a status of the foreground object as being a true foreground object, the status indicating that a sudden illumination change has not occurred for the pixels classified as depicting the foreground in the scene depicted by the sequence of video frames; and

upon determining a count of pixels classified as depicting the foreground object which have a status of SIC candidate pixel is above a fourth threshold, setting a status of the foreground object blob as being an SIC foreground object, the status indicating that a sudden illumination change has occurred for that foreground object in the scene in the current video frame.

6. The computer-implemented method of claim 4, further comprising:

generating a lower-resolution image of the current video frame; and

repeating the computer-implemented method for detecting a sudden illumination change (SIC) in a sequence of video frames recited by claim 1, using the lower resolution image as the current frame of video.

7. The computer-implemented method of claim 4, wherein normalizing the intensity differences comprises, dividing each of the intensity differences determined between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame using a ratio of the radiance value of the pixel in the current video frame and a radiance value of the corresponding pixel in the background image.

8. The computer-implemented method of claim 4, wherein the first specified threshold is determined using a Gaussian distribution determined using the pixel radiance value as a mean and an expected camera noise at the pixel radiance value as a variance.

9. The computer-implemented method of claim 8, wherein the first specified threshold is three standard deviations from the mean.

10. The computer-implemented method of claim 1, wherein each radiance value provides a grayscale value between 0 and 255.

11. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for detecting a sudden illumination change (SIC) in a sequence of video frames, the operation comprising:

receiving a current video frame, of the sequence of video frames;

identifying one or more sets of pixels in the current video frame, wherein each set of pixels is classified as depicting a foreground object;

for each pixel in at least one of the sets of pixels;

identifying a radiance value for the pixel and for each pixel within a spatial neighborhood of the pixel,

determining, for the pixel, whether the radiance values for the pixel and the pixels in the spatial neighborhood indicate that a sudden illumination change has occurred in a scene depicted by the sequence of video frames; and

upon determining that a majority of pixels in a first one of the sets of pixels indicate that a sudden illumination change has occurred, reclassifying the pixels in the first set of pixels as depicting a background of the scene depicted by the sequence of video frames.

12. The computer-readable storage medium of claim 11, wherein the operation further comprises:

updating a set of pixels in a background image of the scene corresponding to the pixels in the first set of pixels.

13. The computer-readable storage medium of claim 11, wherein the spatial neighborhood for a given pixel comprises a 3×

3 grid of pixels with the given pixel in the center of the 3×

3 grid.

14. The computer-readable storage medium of claim 11, wherein determining, for the pixel, whether the radiance values for the pixel and the pixels in the spatial neighborhood indicate that a sudden illumination change has occurred, comprises:

identifying, in a background image, a pixel corresponding to the pixel in the current video frame;

determining an intensity difference between the corresponding pixel and each pixel in a spatial neighborhood of the corresponding pixel;

determining an intensity difference between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame;

normalizing the intensity differences determined between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame;

determining an absolute difference between each respective pixel in the spatial neighborhood of the pixel in the current video frame and each respective pixel in the spatial neighborhood of the corresponding pixel;

for each pixel in the spatial neighborhood of the pixel in the current video frame, determining whether the absolute difference is below a first specified threshold, and

if so, incrementing a counter representing how many pixels in the spatial neighborhood are below the first specified threshold;

upon determining the counter exceeds a second threshold, setting a status of the pixel as being an SIC candidate pixel, indicating that the pixel was classified as depicting a scene foreground object due to a sudden illumination change in the scene depicted by the sequence of video frames.

15. The computer-readable storage medium of claim 14, wherein the operation further comprises:

upon determining a count of pixels classified as depicting the foreground object which have a status of SIC candidate pixel is below a third threshold, setting a status of the foreground object as being a true foreground object, the status indicating that a sudden illumination change has not occurred for the pixels classified as depicting the foreground in the scene depicted by the sequence of video frames; and

upon determining a count of pixels classified as depicting the foreground object which have a status of SIC candidate pixel is above a fourth threshold, setting a status of the foreground object blob as being an SIC foreground object, the status indicating that a sudden illumination change has occurred for that foreground object in the scene in the current video frame.

16. The computer-readable storage medium of claim 14, wherein normalizing the intensity differences comprises, dividing each of the intensity differences determined between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame using a ratio of the radiance value of the pixel in the current video frame and a radiance value of the corresponding pixel in the background image.

17. The computer-readable storage medium of claim 14, wherein the first specified threshold is determined using a Gaussian distribution determined using the pixel radiance value as a mean and an expected camera noise at the pixel radiance value as a variance.

18. The computer-readable storage medium of claim 17, wherein the first specified threshold is three standard deviations from the mean.

19. A system comprising,

a video input source configured to provide a sequence of video frames, each depicting a scene;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for detecting a sudden illumination change (SIC) in a sequence of video frames by performing the steps of;

receiving a current video frame, of the sequence of video frames,

identifying one or more sets of pixels in the current video frame, wherein each set of pixels is classified as depicting a foreground object,

for each pixel in at least one of the sets of pixels;

identifying a radiance value for the pixel and for each pixel within a spatial neighborhood of the pixel, and

determining, for the pixel, whether the radiance values for the pixel and the pixels in the spatial neighborhood indicate that a sudden illumination change has occurred in a scene depicted by the sequence of video frames, and

upon determining that a majority of pixels in a first one of the sets of pixels indicate that a sudden illumination change has occurred, reclassifying the pixels in the first set of pixels as depicting a background of the scene depicted by the sequence of video frames.

20. The system of claim 19, wherein the operation further comprises:

updating a set of pixels in a background image of the scene corresponding to the pixels in the first set of pixels.

21. The system of claim 19, wherein the spatial neighborhood for a given pixel comprises a 3×

3 grid of pixels with the given pixel in the center of the 3×

3 grid.

22. The system of claim 19, wherein determining, for the pixel, whether the radiance values for the pixel and the pixels in the spatial neighborhood indicate that a sudden illumination change has occurred, comprises:

identifying, in a background image, a pixel corresponding to the pixel in the current video frame;

determining an intensity difference between the corresponding pixel and each pixel in a spatial neighborhood of the corresponding pixel;

determining an intensity difference between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame;

normalizing the intensity differences determined between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame;

determining an absolute difference between each respective pixel in the spatial neighborhood of the pixel in the current video frame and each respective pixel in the spatial neighborhood of the corresponding pixel;

for each pixel in the spatial neighborhood of the pixel in the current video frame, determining whether the absolute difference is below a first specified threshold, and

if so, incrementing a counter representing how many pixels in the spatial neighborhood are below the first specified threshold;

upon determining the counter exceeds a second threshold, setting a status of the pixel as being an SIC candidate pixel, indicating that the pixel was classified as depicting a scene foreground object due to a sudden illumination change in the scene depicted by the sequence of video frames.

23. The system of claim 22, wherein the operation further comprises:

upon determining a count of pixels classified as depicting the foreground object which have a status of SIC candidate pixel is below a third threshold, setting a status of the foreground object as being a true foreground object, the status indicating that a sudden illumination change has not occurred for the pixels classified as depicting the foreground in the scene depicted by the sequence of video frames; and

upon determining a count of pixels classified as depicting the foreground object which have a status of SIC candidate pixel is above a fourth threshold, setting a status of the foreground object blob as being an SIC foreground object, the status indicating that a sudden illumination change has occurred for that foreground object in the scene in the current video frame.

24. The system of claim 22, wherein normalizing the intensity differences comprises, dividing each of the intensity differences determined between the pixel and each pixel in the spatial neighborhood of the pixel in the current video frame using a ratio of the radiance value of the pixel in the current video frame and a radiance value of the corresponding pixel in the background image.

25. The system of claim 22, wherein the first specified threshold is determined using a Gaussian distribution determined using the pixel radiance value as a mean and an expected camera noise at the pixel radiance value as a variance.