Identification of objects in a video

• US 8,873,801 B2
• Filed: 08/03/2010
• Issued: 10/28/2014
• Est. Priority Date: 08/03/2010
• Status: Expired due to Fees

• Current Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Wang, Zhanjie
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Kholdebarin, Iman K
• Application Number:
  US13/380,819
• Publication Number:
  US 20120106797A1
• Time in Patent Office: 1,547 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Techniques related to identifying objects in a video are generally described. One example method for identifying a moving object in a video may include generating a background frame and a foreground frame based on the video, comparing the foreground and the background frames at each corresponding location, acquiring an object area based on the comparison, determining if object area contains a moving object based on size and shape of the object area, identifying the moving object against templates of target objects, and updating the background frame according to the comparison.

Claims

1. A method to identify a moving object in a video, comprising:

generating a background frame based on a first frame in the video;

generating a foreground frame based on a second frame in the video, wherein the second frame follows the first frame in the video;

comparing the foreground frame with the background frame at each corresponding location;

acquiring an object area including corresponding locations each having a difference greater than a threshold;

determining, using a processor, whether the object area contains a moving object based on one or more of a size and a shape of the object area;

when the object area contains a moving object, identifying, using the processor, the moving object;

updating, using the processor, the background frame according to the difference between the background and the foreground frames at each corresponding location, wherein updating the background frame comprises, at each corresponding location, responsive to determination that the difference between the background and the foreground frames at the corresponding location exceeds the threshold, adding a predetermined factor to a pixel value of the background image;

generating an other foreground frame based on a third frame in the video, wherein the third frame follows the second frame in the video; and

repeating comparing, acquiring, and determining from the updated background frame and the other foreground frame.

2. The method of claim 1, further comprising:

converting the background and the foreground frames into grayscale images prior to comparing the foreground frame with the background frame so as to acquire a difference in grayscale values between the background and the foreground frames at each corresponding location.

3. The method of claim 2, wherein updating the background frame comprises, at each corresponding location:

when the difference between the background and the foreground frames at the corresponding location does not exceed the threshold, summing (1) a first product of a first pixel value in the background image and a first factor and (2) a second product of a second pixel value in the foreground image and a second factor.

4. The method of claim 3, wherein updating the background frame comprises, at each corresponding location:

B_n+1(x,y)=α

*B_n(x,y)+(1−

α

)*F_n(x,y)

wherein B_n+1(x,y) is a pixel value in the updated background frame at a pixel location (x,y), B_n(x,y) is a pixel value in the background frame at the pixel location (x,y), F_n(x,y) is a pixel value in the foreground frame at the pixel location (x,y), α

is the first factor and (1−

α

) is the second factor.

5. The method of claim 1, wherein updating the background frame comprises, at each corresponding location:

B_n+1(x,y)=B_n(x,y)−

β

wherein B_n+1(x,y) is a pixel value in the updated background frame at a pixel location (x,y), B_n(x,y) is a pixel value in the background frame at the pixel location (x,y), and β

is a predetermined factor.

6. A method to identify a moving object in a video, comprising:

generating a background frame based on a first frame in the video;

generating a foreground frame based on a second frame in the video, wherein the second frame follows the first frame in the video;

comparing the foreground frame with the background frame at each corresponding location;

acquiring an object area including corresponding locations each having a difference greater than a threshold;

removing shadow from the object area, comprising;

removing a shadow pixel at a bit location (x,y) from the object area when

7. The method of claim 6, wherein:

acquiring the object area comprises generating a binary map where a bit is set to a first value when the difference between the background and the foreground frames at a corresponding location exceeds the threshold, and the bit is set to a second value when the difference between the background and the foreground frames at the corresponding location is less than the threshold, wherein the object area comprises the corresponding locations in the foreground frame having the first value in the binary map; and

the shadow pixel is removed from the object area by setting the shadow pixel to the second value.

8. A method to identify a moving object in a video, comprising:

generating a background frame based on a first frame in the video;

generating a foreground frame based on a second frame in the video, wherein the second frame follows the first frame in the video;

comparing the foreground frame with the background frame at each corresponding location;

acquiring an object area including corresponding locations each having a difference greater than a threshold;

determining, using a processor, whether the object area contains a moving object based on one or more of a size and a shape of the object area;

responsive to determination that the object area contains a moving object, identifying, using the processor, the moving object, wherein identifying the moving object comprises;

acquiring a first edge map of the object area by applying edge detection to the object area;

determining a ratio of a length over a width of the edge map; and

selecting a template of a target object having a ratio similar to the ratio of the object area, the template comprising a second edge map;

updating, using the processor, the background frame according to the difference between the background and the foreground frames at each corresponding location;

generating an other foreground frame based on a third frame in the video, wherein the third frame follows the second frame in the video; and

repeating comparing, acquiring, and determining from the updated background frame and the other foreground frame.

9. The method of claim 8, further comprising:

acquiring templates of the target object by taking a plurality of photos of the target object at a plurality of angles and applying edge detection to the plurality of photos of the target object.

10. The method of claim 8, wherein identifying the moving object further comprises:

normalizing the first and the second edge maps.

11. The method of claim 8, wherein identifying the moving object further comprises:

overlapping the first and the second edge maps at a first position;

calculating a Hausdorff distance between the first and the second edge maps; and

determining that the moving object contained in the object area matches the target object when the Hausdorff distance is less than a predetermined distance.

12. The method of claim 11, wherein calculating the Hausdorff distance comprises:

for M reference points on edges in the second edge map, determining distances to closest target point on edges in the first edge map;

select n number of the M reference points having smallest distances, wherein n is an integer smaller than M; and

setting the Hausdorff distance as a quotient of a sum of the smallest distances and n.

13. The method of claim 12, further comprising:

overlapping the first and the second edge maps in a second position;

determining an other Hausdorff distance between the first and the second edge maps overlapped at the second position;

replacing the Hausdorff distance with the other Hausdorff distance when the other Hausorff distance is less than the Hausdorff distance.

14. The method of claim 13, wherein:

overlapping the first and the second edge maps in the first position comprises overlapping the first and the second edge maps border by border, and

overlapping the first and the second images in the second position comprises moving one of the first and the second edge maps in a predetermined direction for a predetermined distance after overlapping the first and the second edge maps border by border.

15. A method to identify a moving object in a video, comprising:

generating a background frame based on a first frame in the video;

generating a foreground frame based on a second frame in the video, wherein the second frame follows the first frame in the video;

comparing the foreground frame with the background frame at each corresponding location;

acquiring an object area including corresponding locations each having a difference greater than a threshold;

determining, using a processor, whether the object area contains a moving object based on one or more of a size and a shape of the object area;

responsive to determination that the object area contains a moving object, identifying, using the processor, the moving object, wherein identifying the moving object comprises;

acquiring an object edge map of the object area by applying edge detection to the object area;

determining a ratio of a length over a width of the edge map;

selecting templates of target objects having ratios similar to the ratio of the object area, the templates comprising template edge maps;

calculating Hausdorff distances between the object edge map and the template edge maps; and

determining that the moving object contained in the object area matches a target object in a template when its Hausdorff distance is the smallest among the Hausdorff distances of the templates;

updating, using the processor, the background frame according to the difference between the background and the foreground frames at each corresponding location;

generating an other foreground frame based on a third frame in the video, wherein the third frame follows the second frame in the video; and

repeating comparing, acquiring, and determining from the updated background frame and the other foreground frame.