Method and apparatus for detecting moving object

• US 6,931,146 B2
• Filed: 12/07/2000
• Issued: 08/16/2005
• Est. Priority Date: 12/20/1999
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Shiohara, Morito, Aoki, Takahiro, Nakayama, Osafumi, Murakami, Yoshishige
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US09/731,010
• Publication Number:
  US 20010004400A1
• Time in Patent Office: 1,713 Days
• Field of Search: 382/100, 382/103, 382/107, 348/154, 348/155, 348/352, 348/407.1, 356/27, 734/88
• US Class Current: 382/107
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

For automatically detecting a moving object in an input image taken using a monitor camera or the like, an apparatus includes an input image extracting section, an input image retaining section, a background image retaining section, a background difference calculating section, a moving direction recognizing section, and a united judging section. A movement of an object is detected with precision even in an environment where illumination varies, irrespective of the relative position of a ridge line of the object and a border of unit blocks. And only the moving object can be detected stably even when the camera encounters swing, vibration, etc.

Claims

1. A method of detecting a moving object, comprising:

(a) a first dividing step of dividing an input image into a plurality of first unit blocks;

(b) a second dividing step of dividing a given background image into a plurality of second unit blocks;

(c) a moving block extracting step of comparing said first unit blocks with said second unit blocks for every unit block and extracting a number of said first unit blocks, which are different in brightness distribution pattern from the corresponding second unit blocks, as moving blocks;

(d) a moving area setting step of setting a moving area that surrounds said moving blocks extracted in said moving block extracting step; and

(e) a brightness comparing step of comparing a distribution of brightness values of the input image with a distribution of brightness values of the background image in the moving area, which is set by said moving area setting step, to discriminate whether or not a moving object appears in the input image based on a size of the moving object.

2. A moving object detecting method according to claim 1, wherein said brightness comparing step includes:

an appearance frequency measuring step of measuring a frequency of occurrence of pixels having a predetermined brightness value; and

a trimming step of trimming the pixels, whose appearance frequency is lower than the predetermined value, from the pixels of the input image in the whole range of various brightness values.

3. A moving object detecting method according to claim 1, wherein in said moving area setting step, said moving area surrounding the moving blocks is rectangular in shape.

4. A method of detecting a moving object, comprising:

(a) a first dividing step of dividing an input image into a plurality of first unit blocks;

(b) a second dividing step of dividing a given background image into a plurality of second unit blocks;

(c) a moving block extracting step of comparing said first unit blocks with said second unit blocks for every unit block and extracting a number of said first unit blocks, which are different in brightness distribution pattern from the corresponding second unit blocks, as moving blocks;

(d) a moving area setting step of setting a moving area that surrounds said moving blocks extracted in said moving block extracting step; and

(e) a brightness comparing step of comparing a distribution of brightness values of the input image with a distribution of brightness values of the background image in the moving area, which is set by said moving area setting step, to discriminate whether or not a moving object appears in the input image,

wherein in each of said first dividing step and said second dividing step, the individual first unit block overlaps neighboring unit blocks disposed adjacent to the last-named first unit block.

5. A method of detecting a moving object, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from an input image;

(b) a region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said region dividing step; and

(d) an appearance detecting step of discriminating whether or not a moving object appears in the input image based on the variance calculated in said variance calculating step,

wherein said appearance detecting step includes;

a history information creating step of creating history information about the variance calculated for every one of the plural zones obtained in said region and dividing step; and

a variance comparing step of comparing the past variance before a current time based on the history information created in said history information creating step with the current variance in the current time based on the input image to discriminate whether an moving object appears in the input image.

6. A moving object detecting method according to claim 5, wherein in said appearance detecting step,

if the number of the zones whose variance is equal to or higher than a first threshold is equal to or larger than a predetermined value, the appearance of the moving object is notified, and

if the number of the zones whose variance is equal to or lower than a second threshold is equal to or larger than a predetermined value, the absence of the moving object is notified.

7. A moving object detecting method according to claim 5, wherein in said appearance detecting step,

if the number of the zones whose increase value of variance is equal to or higher than a third threshold is equal to or larger than a predetermined value, the appearance of the moving object is notified, and

if the number of the zones whose decrease value of variance is equal to or higher than a fourth threshold is equal to or larger than a predetermined value, the absence of the moving object is notified.

8. A moving object detecting method according to claim 7, wherein in said appearance detecting step, a reference value, based on which the increase value of variance and the decrease value of variance are to be calculated, is created from average values of the past variances before the current time.

9. A moving object detecting method according to claim 5, wherein in said appearance detecting step,

if the variance from which entering of the moving object is to be detected is equal to or higher than a first threshold, the appearance of the moving object is notified, and

if the decrease value of variance from which leaving of the moving object is to be detected is equal to or larger than a fourth threshold, the absence of the moving object is notified.

10. A moving object detecting method according to claim 5, wherein in said appearance detecting step,

if an increase value of variance from which entering of the moving object is to be detected is equal to or higher than a third threshold, the appearance of the moving object is notified, and

if the variance from which leaving of the moving object is to be detected is equal to or lower than a second threshold, the absence of the moving object is notified.

11. A moving object detecting method according to claim 5, wherein in said appearance detecting step, the monitoring region is set by expanding an assumed area of the moving object.

12. A method of detecting a moving object, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from an input image;

(b) a region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said region dividing step; and

(d) an appearance detecting step of discriminating whether or not a moving object appears in the input image based on the variance calculated in said variance calculating step, wherein in said appearance detecting step,

if the number of the zones whose variance is equal to or higher than a first threshold is equal to or larger than a predetermined value, the appearance of the moving object is notified, and

if the number of the zones whose variance is equal to or lower than a second threshold is equal to or larger than a predetermined value, the absence of the moving object is notified.

13. A method of detecting a moving object, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from an input image;

(b) a region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said region dividing step; and

(d) an appearance detecting step of discriminating whether or not a moving object appears in the input image based on the variance calculated in said variance calculating step, wherein in said appearance detecting step,

if the number of the zones whose increase value of variance is equal to or higher than a third threshold is equal to or larger than a predetermined value, the appearance of the moving object is notified, and

if the number of the zones whose decrease value of variance is equal to or higher than a fourth threshold is equal to or larger than a predetermined value, the absence of the moving object is notified.

14. A moving object detecting method according to claim 13, wherein in said appearance detecting step, a reference value, based on which the increase value of variance and the decrease value of variance are to be calculated, is created from average values of the past variances before the current time.

15. A method of detecting a moving object, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from an input image;

(b) a region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said region dividing step; and

(d) an appearance detecting step of discriminating whether or not a moving object appears in the input image based on the variance calculated in said variance calculating step,

wherein in said appearance detecting step, if the variance from which entering of the moving object is to be detected is equal to or higher than a first threshold, the appearance of the moving object is notified, and

if the decrease value of variance from which leaving of the moving object is to be detected is equal to or larger than a fourth threshold, the absence of the moving object is notified.

16. A method of detecting a moving object, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from an input image;

(b) a region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said region dividing step; and

(d) an appearance detecting step of discriminating whether or not a moving object appears in the input image based on the variance calculated in said variance calculating step, wherein in said appearance detecting step,

if an increase value of variance from which entering of the moving object is to be detected is equal to or higher than a third threshold, the appearance of the moving object is notified, and

if the variance from which leaving of the moving object is to be detected is equal to or lower than a second threshold, the absence of the moving object is notified.

17. A method of detecting a moving object, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from an input image;

(b) a region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said region dividing step; and

(d) an appearance detecting step of discriminating whether or not a moving object appears in the input image based on the variance calculated in said variance calculating step, wherein in said appearance detecting step, the monitoring region is set by expanding an assumed area of the moving object.

18. A method of detecting a moving objection an input image, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from the input image;

(b) a first region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones in a direction perpendicular to a direction in which the moving object enters;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said first region dividing step;

(d) a zone discriminating step of discriminating whether an individual one of the plural zones is an object-appearing zone in which the moving object appears or an object-free zone in which the moving object is absent, by comparing the variance obtained in said variance calculating step with a predetermined value; and

(e) a moving-direction recognizing step of recognizing a direction in which the moving object moves, based on a direction of occurrence of the object-appearing zone found as the result of discrimination in said zone discriminating step,

wherein said zone discriminating step uses a first discriminating way such that

if the variance is equal to or larger than said predetermined value, the plural zones are judged as the object-appearing zones, and

if the variance is equal to or smaller than said predetermined value, the plural zones are judged as the object-free zones.

19. A moving object detecting method according to claim 18, wherein in said moving-direction recognizing step, the direction of moving of the moving object is recognized using one of said first and second recognizing ways.

20. A method of detecting a moving object in an input image, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from the input image;

(b) a first region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones in a direction perpendicular to a direction in which the moving object enters;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said first region dividing step;

(d) a zone discriminating step of discriminating whether an individual one of the plural zones is an object-appearing zone in which the moving object appears or an object-free zone in which the moving object is absent, by comparing the variance obtained in said variance calculating step with a predetermined value; and

(e) a moving-direction recognizing step of recognizing a direction in which the moving object moves, based on a direction of occurrence of the object-appearing zone found as the result of discrimination in said zone discriminating step,

wherein said zone discriminating step uses a second discriminating way such that

if the amount of change in the variance is equal to or larger than a predetermined value, the plural zones are judged as the object-appearing zones, and

if the amount of change in the variance is equal to or smaller than said predetermined value, the plural zones are judged as the object-free zones.

21. A moving object detecting method according to claim 20, wherein in said zone discriminating step, said predetermined value is created from an average value of the past variance of brightness values before the current time.

22. A moving object detecting method according to claim 21,

wherein in said moving-direction recognizing step, the direction of moving of the moving object is recognized using one of said first and second recognizing ways.

23. A moving object detecting method according to claim 20, wherein in said moving-direction recognizing step, the direction of moving of the moving object is recognized using one of said first and second recognizing ways.

24. An apparatus for detecting a moving object in an input image, comprising:

(a) an input image retaining section for retaining the input image;

(b) a background image retaining section, connected to said input image retaining section, for retaining a given background image;

(c) a background difference calculating section, connected to said input image retaining section and said background image retaining section, for (c1) calculating a difference between the input image and the background image, and for (c2) comparing an input-image-related unit block and a background-image-related unit block to extract a moving block which changes a shape in brightness distribution;

(d) a moving direction recognizing section, connected to said input image retaining section, for dividing data of the input image into a plurality of pieces of data one piece for each of a plurality of zones and evaluating the data; and

(e) a united judging section, connected to said background difference calculating section and said moving direction recognizing section, for judging the appearance of the moving object and the direction of moving of the moving object.

25. An apparatus for detecting a moving object, comprising:

(a) an input image retaining section for retaining the input image;

(b) a background image retaining section, connected to said input image retaining section, far retaining a given background image;

(c) a background difference calculating section, connected to said input image retaining section and said background image retaining section, for calculating a difference between the input image and the background image;

(d) a moving direction recognizing section, connected to said input image retaining section, for dividing data of the input image into a plurality of pieces of data one piece for each of a plurality of zones and evaluating the data; and

(e) a united judging section, connected to said background difference calculating section and said moving direction recognizing section, for judging the appearance of the moving object and the direction of moving of the moving object,

wherein said background difference calculating section includes;

a block-background difference calculating unit, connected to said input image retaining section and the background image retaining section, for comparing first unit blocks related to the input image with second unit blocks related to the background image to extract a moving block in which a change occurs in brightness distribution pattern; and

an in-moving-area background difference calculating unit, connected to said block-background difference calculating unit, for setting a moving area surrounding the extracted moving block and comparing the distribution pattern of brightness values of the input image and the distribution pattern of brightness values of the background image to discriminate whether the moving object appears in the input image.

26. An apparatus for detecting a moving object, comprising:

(a) an input image retaining section for retaining the input image;

(b) a background image retaining section, connected to said input image retaining section, for retaining a given background image;

(c) a background difference calculating section, connected to said input image retaining section and said background image retaining section, for calculating a difference between the input image and the background image;

(d) a moving direction recognizing section, connected to said input image retaining section, for dividing data of the input image into a plurality of pieces of data one piece for each of a plurality of zones and evaluating the data; and

(e) a united judging section, connected to said background difference calculating section and said moving direction recognizing section, for judging the appearance of the moving object and the direction of moving of the moving object,

wherein said moving direction recognizing section includes;

a zone information retaining unit for retaining and outputting zone division information related to division of the input image into the plural zones;

a variance calculating unit, connected to said zone information retaining unit and said input image retaining section, for dividing data of the input image, which is retained in said input image retaining section, into the plural pieces of data one piece for each the plural zones in accordance with the zone division information output from said zone information retaining unit, and for calculating variance of brightness values for every one of the plural zones;

a variance history managing unit for retaining and outputting history information related to the past variance before the current time and calculated for each and every one of the plural zones; and

a variance evaluating unit, connected to said variance calculating unit and said variance history managing unit, for evaluating the variance calculated by said variance calculating unit and the history information output from said variance history managing unit.

27. A method of detecting a moving object in an input image, comprising:

(a) a monitoring region clipping step of clipping a monitoring region, which is to be monitored, from the input image;

(b) a first region dividing step of dividing the monitoring region, which is clipped in said monitoring region clipping step, into a plurality of zones in a direction perpendicular to a direction in which the moving object enters;

(c) a variance calculating step of calculating a variance of brightness values for each and every one of the plural zones obtained in said first region dividing step;

(d) a zone discriminating step of discriminating whether an individual one of the plural zones is an object-appearing zone in which the moving object appears or an object-free zone in which the moving object is absent, by comparing the variance obtained in said variance calculating step with a predetermined value; and

(e) a moving-direction recognizing step of recognizing a direction in which the moving object moves, based on a direction of occurrence of the object-appearing zone found as the result of discrimination in said zone discriminating step.