Motion detection of object

• US 10,460,456 B2
• Filed: 11/02/2016
• Issued: 10/29/2019
• Est. Priority Date: 12/10/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Xie, Wenxuan, Lan, Cuiling, Zeng, Wenjun
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/342,070
• Publication Number:
  US 20170169574A1
• Time in Patent Office: 1,091 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

In implementations of the subject matter described herein, a current captured frame of a video is compared with a respective reference frame to determine a correlation therebetween. The correlation is used to indicate a change degree of the current frame. If the correlation for the current frame is below a predetermined threshold, the current frame may not be directly determined as including a motion of an object. Instead, correlations between one or more frames before or after the current frame and their respective reference frames are took into account. If the correlations of the frames under consideration are below the predetermined threshold, it may be detected that the current frame includes a motion of an object. In this way, incorrect detection of the object motion is reduced in the cases when larger changes in frames of a video are caused by factors such as noise and error, and the accuracy of the object motion detection is improved.

Claims

1. A method comprising:

receiving, by a video surveillance device, a first frame of a video and at least one second video frame of the video;

identifying, by the video surveillance device, a plurality of foreground pixels from pixels in the first frame;

determining, by the video surveillance device, a first correlation between the first frame of the video and a first reference frame based on intensity values of the identified foreground pixels;

in response to the first correlation being below a first threshold, by the video surveillance device, determining whether a second correlation between at least one second frame of the video and a second reference frame is below the first threshold; and

detecting, by the video surveillance device, whether the first frame of the video includes a motion of an object at least in part based on the determination that the second correlation is below the first threshold.

2. The method of claim 1, further comprising:

in response to determining that a third correlation between a third frame after the first frame and a third reference frame is below a second threshold, detecting that the third frame includes a motion of an object, the second threshold being larger than the first threshold.

3. The method of claim 1, further comprising:

in response to detecting that the first frame includes a motion of an object, generating an alarm for a user.

4. The method of claim 1, wherein determining the first correlation based on intensity values of the identified foreground pixels comprises:

determining intensity differences between the intensity values of the foreground pixels in the first frame and intensity values of corresponding pixels in the first reference frame; and

determining the first correlation based on the intensity differences.

5. The method of claim 1, wherein determining the first correlation comprises:

obtaining color properties of the foreground pixels, the color properties including at least one of the following;

hue values and saturation values; and

determining the first correlation further based on the intensity values of the foreground pixels and the color properties of the foreground pixels.

6. The method of claim 2, further comprising:

in response to a first number of the foreground pixels in the first frame exceeding a third threshold, determining whether a second number of foreground pixels in the at least one second frame exceeds the third threshold, and

wherein the detecting comprises;

in response to determining that the second correlation is below the first threshold and the second number exceeds the third threshold, detecting that the first frame includes a motion of an object.

7. The method of claim 6, further comprising:

in response to determining that a third number of foreground pixels in a fourth frame after the first frame exceeds a fourth threshold, detecting that the fourth frame includes a motion of an object, the fourth threshold being smaller than the third threshold.

8. A video surveillance device for use with at least one video capturing device, the video surveillance device comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code are configured to, with the at least one processor, cause the device to;

receive a first frame of a video and at least one second video frame of the video,

identify a plurality of foreground pixels from pixels in the first frame,

determine a first correlation between the first frame of the video and a first reference frame based on intensity values of the identified foreground pixels,

in response to the first correlation being below a first threshold, determine whether a second correlation between at least one second frame of the video and a second reference frame is below the first threshold, and

detect whether the first frame of the video includes a motion of an object at least in part based on the determination that the second correlation is below the first threshold.

9. The video surveillance device of claim 8, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the device to:

in response to determining that a third correlation between a third frame after the first frame and a third reference frame is below a second threshold, detect that the third frame includes a motion of an object, the second threshold being larger than the first threshold.

10. The video surveillance device of claim 8, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the device to:

in response to detecting that the first frame includes a motion of an object, generate an alarm for a user.

11. The video surveillance device of claim 8, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the device to:

determine intensity differences between the intensity values of the foreground pixels in the first frame and intensity values of corresponding pixels in the first reference frame; and

determine the first correlation based on the intensity differences.

12. The video surveillance device of claim 8, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the device to:

obtain color properties of the foreground pixels, the color properties including at least one of the following;

hue values and saturation values; and

determine the first correlation further based on the intensity values of the foreground pixels and the color properties of the foreground pixels.

13. The video surveillance device of claim 8, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the device to:

in response to a first number of the foreground pixels in the first frame exceeding a third threshold, determine whether a second number of foreground pixels in the at least one second frame exceeds the third threshold; and

in response to determining that the second correlation is below the first threshold and the second number exceeds the third threshold, detect that the first frame includes a motion of an object.

14. The video surveillance device of claim 13, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the device to:

in response to determining that a third number of foreground pixels in a fourth frame after the first frame exceeds a fourth threshold, detect that the fourth frame includes a motion of an object, the fourth threshold being smaller than the third threshold.

15. The video surveillance device of claim 8, further comprising:

a receiver configured to receive the video from a remote video capture device.

16. A method comprising:

receiving, by a video surveillance device, a first frame of a video and at least one second video frame of the video;

identifying, by the video surveillance device, a plurality of foreground pixels from pixels in the first frame;

determining, by the video surveillance device, a first correlation between the first frame of the video and a first reference frame based on intensity values of the identified foreground pixels;

in response to the first correlation being below a first threshold, by the video surveillance device, determining whether a second correlation between at least one second frame of the video and a second reference frame is below the first threshold;

detecting, by the video surveillance device, whether the first frame of the video includes a motion of an object at least in part based on the determination that the second correlation is below the first threshold; and

in response to determining that a third correlation between a third frame after the first frame and a third reference frame is below a second threshold, detecting that the third frame includes a motion of an object, the second threshold being larger than the first threshold.

17. The method of claim 16, further comprising:

in response to a first number of the foreground pixels in the first frame exceeding a third threshold, determining whether a second number of foreground pixels in the at least one second frame exceeds the third threshold, and

wherein the detecting comprises;

in response to determining that the second correlation is below the first threshold and the second number exceeds the third threshold, detecting that the first frame includes a motion of an object.

18. The method of claim 16, further comprising:

in response to detecting that the first frame includes a motion of an object, generating an alarm for a user.

19. The method of claim 16, further comprising:

determining intensity differences between the intensity values of the foreground pixels in the first frame and intensity values of corresponding pixels in the first reference frame; and

determining the first correlation based on the intensity differences.

20. The method of claim 16, further comprising:

obtaining color properties of the foreground pixels, the color properties including at least one of the following;

hue values and saturation values; and

determining the first correlation further based on the intensity values of the foreground pixels and the color properties of the foreground pixels.