Method and System for Video Coding with Noise Filtering

• US 20130170557A1
• Filed: 12/29/2011
• Published: 07/04/2013
• Est. Priority Date: 12/29/2011
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Wang, Lei, Zhu, Hongwei, Aghdasi, Farzin, Millar, Greg
• Granted Patent: US 9,681,125 B2
• Time in Patent Office: Days
• US Class Current: 375/240.24
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic

Abstract

Techniques are discussed herein for providing mechanisms for coding and transmitting high definition video, e.g., over low bandwidth connections. In particular, foreground-objects are identified as distinct from the background of a scene represented by a plurality of video frames. In identifying foreground-objects, semantically significant and semantically insignificant movement (e.g., non-repetitive versus repetitive movement) is differentiated. For example, the swaying motion of a tree'"'"'s leaves being minor and repetitive, can be determined to be semantically insignificant and to belong in a scene'"'"'s background. Processing of the foreground-objects and background proceed at different update rates or frequencies. For example, foreground-objects can be updated 30 or 60 times per second. By contrast, a background is updated less frequently, e.g., once every 10 seconds. In some implementations, if no foreground-objects are identified, no live video is transmitted (e.g., if no motion is detected, static images are not configured to be repeatedly sent). Techniques described herein take advantage of the realization that, in the area of surveillance and wireless communications, updating video of semantically significant movement at a high frame rate is sufficient.

Claims

1. A method of video coding comprising:

receiving a video image having a plurality of pixels;

selecting a plurality of Gaussian models for each pixel in the plurality of pixels in the image;

classifying each pixel as a background pixel or foreground pixel based on the probability of the model that the pixel fits best;

dividing the image into M×

M pixel blocks;

comparing for each block the motion pixels in the current frame with motion pixels in the previous frame to determine if the number of pixels with motion change is greater than a first threshold;

triggering a counter for a block if the number of pixels with motion change is greater than the first threshold;

determining if an accumulated count in the counter in a first predetermined period of time is larger than a second threshold;

ignoring all motion in a block if the accumulated count in the counter in a first predetermined period of time is larger than the second threshold;

tracking objects in the image;

determining if an object is always moving locally by detecting if an object is always moving within a predetermined distance and changing the direction of moving frequently for a second predetermined period of time;

removing an object from classification as an object if the determining step determines that an object is always moving locally;

ignoring the motion of an object if the determining step determines that an object is always moving locally and marking the area that the object moves within as a noisy area on a noise mask buffer;

updating background models based on motion detection and noise filtering;

updating a current background image;

updating an evolving background image; and

coding the current background image, background update blocks and objects for transmission.

2. A method as recited in claim 1 further comprising the steps of:

creating a queue to buffer a plurality of entries in the queue including objects, background blocks and images that need to be updated;

providing frame and object identifiers to entries in the queue;

determining if an object is a true object;

determining if an object is in a queue for a period of time greater than a third predetermined period of time; and

removing an object from a buffer if the object has not been identified as a true object and it has been determined that the object has been in a queue for a period of time greater than the third predetermined period of time.

3. A method as recited in claim 1 wherein the step of receiving a video image having a plurality of pixels comprises receiving a video image that was captured by a camera viewing a scene and further comprising the steps of determining if an object has moved into the scene and stopped in the scene, and if the step for determining if an object has moved into the scene and stopped in the scene determines that a first object has moved into the scene and stopped, updating a background model of the scene and coding the first object as background.

4. A method as recited in claim 1 further comprising the steps of establishing a current background image as a reference image for coding and decoding;

establishing an evolving background image;

updating the current background image with the still objects detected;

updating the evolving background image with received images based on motion detection; and

replacing the current background image with the evolving background image.

5. A method as recited in claim 1 further comprising the steps of:

using a long-term counter with each Gaussian model to determine if a current pixel is a foreground pixel;

using a short-term counter with each Gaussian model to determine which Gaussian model to use for the pixel of the current background image;

if a pixel is detected as a background pixel, updating the corresponding evolving background model with a moving average method;

if an object is detected as a still object, updating the pixels in the area in the current frame into the current background image and evolving background image;

if an object is detected as a still object, maximizing the short-term counters of the Gaussian models corresponding to the pixels in the area of the current frame and updating the long-term counters to ensure that the pixels will be classified as background pixels; and

if a pixel is detected as a background pixel and the background model for the pixel is different to the pixel in the previous frame, updating the pixel of both of the current and evolving background image and coding and transmitting the updated current background pixels.

6. A method as recited in claim 1 wherein the step of coding the current background image, background update blocks and objects for transmission comprises the step of using a current background image and a previous decoded frame as potential reference images.

7. A system for video coding comprising:

a processor adapted to receive a video image having a plurality of pixels, select a plurality of Gaussian models for each pixel in the plurality of pixels in the image, classify each pixel as a background pixel or foreground pixel based on the probability of the model that the pixel fits best divide the image into M×

M pixel blocks, compare for each block the motion pixels in the current frame with motion pixels in the previous frame to determine if the number of pixels with motion change is greater than a first threshold, trigger a counter for a block if the number of pixels with motion change is greater than the first threshold, determine if an accumulated count in the counter in a first predetermined period of time is larger than a second threshold, ignore all motion in a block if the accumulated count in the counter in a first predetermined period of time is larger than the second threshold, track objects in the image, determine if an object is always moving locally by detecting if an object is always moving within a predetermined distance and changing the direction of moving frequently for a second predetermined period of time, remove an object from classification as an object if the processor determines that an object is always moving locally, ignore the motion of an object if the determining step determines that an object is always moving locally and marking the area that the object moves within as a noisy area on a noise mask buffer, update background models based on motion detection and noise filtering, update a current background image, update an evolving background image and code the current background image, background update blocks and objects for transmission.

8. A system as recited in claim 7 wherein the processor is further adapted to create a queue to buffer a plurality of entries in the queue including objects, background blocks and images that need to be updated;

providing frame and object identifiers to entries in the queue, determine if an object is a true object, determine if an object is in a queue for a period of time greater than a third predetermined period of time, and remove an object from a buffer if the object has not been identified as a true object and it has been determined that the object has been in a queue for a period of time greater than the third predetermined period of time.

9. A system as recited in claim 7 wherein the processor is further adapted to receive a video image that was captured by a camera viewing a scene and to determine if an object has moved into the scene and stopped in the scene, and if the processor determines that a first object has moved into the scene and stopped, to update a background model of the scene and to code the first object as background.

10. A system as recited in claim 7 wherein the processor is further adapted to establish a current background image as a reference image for coding and decoding, establish an evolving background image, update the current background image with the still objects detected, update the evolving background image with received images based on motion detection, and replace the current background image with the evolving background image.

11. A system as recited in claim 7 wherein the processor is further adapted to use a long-term counter with each Gaussian model to determine if a current pixel is a foreground pixel, use a short-term counter with each Gaussian model to determine which Gaussian model to use for the pixel of the current background image, if a pixel is detected as a background pixel, to update the corresponding evolving background model with a moving average method, if an object is detected as a still object, to update the pixels in the area in the current frame into the current background image and evolving background image, if an object is detected as a still object, to maximize the short-term counters of the Gaussian models corresponding to the pixels in the area of the current frame and to update the long-term counters to ensure that the pixels will be classified as background pixels, and if a pixel is detected as a background pixel and the background model for the pixel is different to the pixel in the previous frame, to update the pixel of both of the current and evolving background image and coding and transmitting the updated current background pixels.

12. A system as recited in claim 7 wherein the processor is further adapted to use a current background image and a previous decoded frame as potential reference images.

13. A non-transitory computer readable medium comprising instructions configured to cause a processor to:

receive a video image having a plurality of pixels;

selecting a plurality of Gaussian models for each pixel in the plurality of pixels in the image;

classify each pixel as a background pixel or foreground pixel based on the probability of the model that the pixel fits best;

divide the image into M×

M pixel blocks;

compare for each block the motion pixels in the current frame with motion pixels in the previous frame to determine if the number of pixels with motion change is greater than a first threshold;

trigger a counter for a block if the number of pixels with motion change is greater than the first threshold;

determine if an accumulated count in the counter in a first predetermined period of time is larger than a second threshold;

ignore all motion in a block if the accumulated count in the counter in a first predetermined period of time is larger than the second threshold;

track objects in the image;

determine if an object is always moving locally by detecting if an object is always moving within a predetermined distance and changing the direction of moving frequently for a second predetermined period of time;

remove an object from classification as an object if the processor determines that an object is always moving locally;

ignore the motion of an object if the processor determines that an object is always moving locally and marking the area that the object moves within as a noisy area on a noise mask buffer;

update background models based on motion detection and noise filtering;

update a current background image;

update an evolving background image; and

code the current background image, background update blocks and objects for transmission.

14. A non-transitory computer readable medium as recited in claim 13 further comprising instructions configured to cause the processor to:

create a queue to buffer a plurality of entries in the queue including objects, background blocks and images that need to be updated;

provide frame and object identifiers to entries in the queue;

determine if an object is a true object;

determine if an object is in a queue for a period of time greater than a third predetermined period of time; and

remove an object from a buffer if the object has not been identified as a true object and it has been determined that the object has been in a queue for a period of time greater than the third predetermined period of time.

15. A non-transitory computer readable medium as recited in claim 13 further comprising instructions configured to cause the processor to:

receive a video image that was captured by a camera viewing a scene;

determine if an object has moved into the scene and stopped in the scene, and if the processor determines that a first object has moved into the scene and stopped, updating a background model of the scene and coding the first object as background.

16. A non-transitory computer readable medium as recited in claim 13 further comprising instructions configured to cause the processor to:

establish a current background image as a reference image for coding and decoding;

establish an evolving background image;

update the current background image with the still objects detected;

update the evolving background image with received images based on motion detection; and

replace the current background image with the evolving background image.

17. A non-transitory computer readable medium as recited in claim 13 further comprising instructions configured to cause the processor to:

use a long-term counter with each Gaussian model to determine if a current pixel is a foreground pixel;

use a short-term counter with each Gaussian model to determine which Gaussian model to use for the pixel of the current background image;

if a pixel is detected as a background pixel, to update the corresponding evolving background model with a moving average method;

if an object is detected as a still object, to update the pixels in the area in the current frame into the current background image and evolving background image;

if an object is detected as a still object, to maximize the short-term counters of the Gaussian models corresponding to the pixels in the area of the current frame and to update the long-term counters to ensure that the pixels will be classified as background pixels; and

if a pixel is detected as a background pixel and the background model for the pixel is different to the pixel in the previous frame, to update the pixel of both of the current and evolving background image and coding and to transmit the updated current background pixels.

18. A non-transitory computer readable medium as recited in claim 13 wherein the instructions configured to cause the processor to of code the current background image, background update blocks and objects for transmission comprise instructions configured to cause the processor to use a current background image and a previous decoded frame as potential reference images.