Method and device of object detectable and background removal, and storage media for storing program thereof

• US 6,333,993 B1
• Filed: 10/05/1998
• Issued: 12/25/2001
• Est. Priority Date: 10/03/1997
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Sakamoto, Shizuo
• Primary Examiner(s): Chang, Jon
• Assistant Examiner(s): Wu, Jingge
• Application Number:
  US09/166,213
• Time in Patent Office: 1,177 Days
• Field of Search: 382/171, 382/172, 382/173, 382/174, 382/176, 382/180, 382/270, 385/455, 385/456, 385/462, 385/464, 385/465, 385/466
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method and device for an object detectable and background removal and storage media for storing program thereof enable automatic detection of an object to be executed minutely and high precisely for outline. A sectional image statistic calculation measure calculates a mean value and standard deviation of characteristic value of brightness and so forth of the sectional image with input image being subjected to division processing into sectional image. A background sectional image selection measure causes a sectional image whose standard deviation is the smallest value in the sectional images to be taken as the sectional image with high probability of including only the background. A background statistic estimation measure investigates the sectional image including only the background and another sectional image under the relationship between the mean value and the standard deviation. This investigation is implemented in terms of whole sectional images, a threshold generation object detectable and background removal measure discriminates the background and the detected target object based on predetermined calculation procedure. For instance, a second threshold is in use, which is obtained in such a way that the standard deviation multiplied by constant number given beforehand from the mean value is added thereto.

Claims

1. A method of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours comprising the steps of:

using an image consisting of a virtually even background and an object of detection target as an input image, by way of an input process of said image;

calculating a statistic in every respective sectional images, while dividing said input image into sectional images, by way of a statistic calculation process;

selecting a sectional image including only said background based on said statistic calculated previously in said statistic calculation process, by way of a background only sectional image selection process;

estimating a statistic of a whole picture from said statistic of said sectional image including only said background, by way of a statistic estimation process;

determining a threshold in the whole picture from said estimated statistic, by way of a threshold determination process; and

comparing said threshold determined in the whole picture with said input image, by way of a comparison process, wherein said method causes said object of detection target to be isolated from said input image,

wherein when said statistic estimation process estimates a mean value and a standard deviation of the prescribed characteristic value by way of the background in the sectional image except the background, if there exists the mean value and the standard deviation of the prescribed characteristic value of the sectional image including only the background located in the neighborhood and the mean value and the standard deviation of the prescribed characteristic value of the sectional image except the background estimated previously in said same neighborhood, thus estimating the mean value and the standard deviation by averaging above respective average values and the standard deviations, while if there does not exist them, estimating the mean value and the standard deviation of the prescribed characteristic value in another sectional images except the background, thus repeating estimation processing both of a mean value and a standard deviation of the prescribed characteristic value of said background until when it is capable of estimating a mean value and a standard deviation of the prescribed characteristic value of the background in the whole sectional images except the background.

2. A method of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours comprising the steps of:

using an image consisting of a virtually even background and an object of detection target as an input image, by way of an input process of said image;

calculating a statistic in every respective sectional images, while dividing said input image into sectional images, by way of a statistic calculation process;

selecting a sectional image including only said background based on said statistic calculated previously in said statistic calculation process, by way of a background only sectional image selection process;

estimating a statistic of a whole picture from said statistic of said sectional image including only said background, by way of a statistic estimation process;

determining a threshold in the whole picture from said estimated statistic, by way of a threshold determination process; and

comparing said threshold determined in the whole picture with said input image, by way of a comparison process, wherein said method causes said object of detection target to be isolated from said input image,

wherein in regard to isolation of only said object of detection target at said comparison process, when said comparison process implements threshold processing in the respective location of the pixels, firstly a first threshold is calculated to be defined in such a way that a constant set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied value is subtracted from a mean value of the prescribed characteristic value of the background estimated previously, secondly a second threshold is calculated to be defined in such a way that also a constant set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied value is added to a mean value of the prescribed characteristic value of the background estimated previously, thus in case where the prescribed characteristic value of said location of the pixel is larger than the first threshold and is smaller than the second threshold, judging said pixel as the background, so that said comparison process removes the background to isolate the object due to the fact that said comparison process causes the same processing to be executed over the whole pixels.

3. A method of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours as claimed in claim 2, wherein in regard to isolation of only said object of detection target at said comparison process, said comparison process implements a threshold processing over the whole pixels, to detect areas linked one another by way of areas to be a candidate of a background, thus judging the areas concerned as areas including the most large number of sectional images whose probability of including only the background among candidate areas of said background.

4. A method of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours as claimed in claim 2, wherein in regard to isolation of only said object of detection target at said comparison process, said comparison process implements a threshold processing over the whole pixels, to detect areas linked one another by way of areas to be a candidate of a background, thus judging the areas including the most largest number of the sectional images including only the background among candidate areas of said background as a background area, so that said comparison process removes the background to isolate the object.

5. A method of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours as claimed in claim 2, wherein in regard to isolation of only said object of detection target at said comparison process, said comparison process implements a threshold processing over the whole pixels, to detect areas linked one another by way of areas to be a candidate of a background, thus judging the area concerned including the most smallest number of sectional images which include image with the exception of the background among the candidate areas of said background as a background area, so that said comparison process removes the background to isolate the object.

6. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, with an image constituted by virtually even background and an object of detection target, said device roughly consisting of a sectional image statistic calculation means, a background sectional image selection means, a background statistic estimation means, and a threshold generation object detectable and background removal means,

said sectional image statistic calculation means comprising:

a sectional image division means for dividing input images into sectional images;

a mean value and a standard deviation calculation means which calculates to be outputted a mean value and a standard deviation of the prescribed characteristic value in every respective sectional images with said sectional image signals as inputs; and

a sectional image statistic storage means storing to be outputted the mean value and the standard deviation of the prescribed characteristic value of respective sectional images with the mean value and the standard deviation of the prescribed characteristic value of said sectional images as inputs,

said background sectional image selection means comprising;

a minimum standard deviation reference background only sectional image selection means for outputting a sectional image whose standard deviation of the prescribed characteristic value is of the most smallest value among sectional images as a sectional image whose probability of including only a background is high with the mean value and the standard deviation of the prescribed characteristic value of the sectional image;

a background only sectional image selection means comparing a standard deviation of the prescribed characteristic value of a sectional image whose probability of including only the background is high with a standard deviation of the prescribed characteristic value in another sectional images, thus judging to be outputted a partial image having standard deviation whose difference between the standard deviation concerned and a standard deviation of the prescribed characteristic value of a sectional image whose probability of including only said background is less than a threshold as a sectional image including only a background; and

a background only sectional image statistic storage means storing a location of the sectional image including only said background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned to output them whenever necessary,

said background statistic estimation means comprising;

a background-exception sectional image selection means, when a command for investigating a sectional image except a background enters thereto, investigating both of a mean value and a standard deviation of the prescribed characteristic value in a sectional image including only a background and a mean value and a standard deviation of the prescribed characteristic value by way of an estimated background in another sectional images, if there exists a sectional image whose no estimated value of a mean value and a standard deviation of the prescribed characteristic value by way of a background exists, outputting the partial image concerned, in case where a mean value and a standard deviation of the prescribed characteristic value are estimated with regard to whole sectional images, so that said background statistic estimation means issues a command of generating a threshold for the sake of object detectable and background removal;

a neighborhood background only sectional image existence judgement means investigating mean values and standard deviations both of sectional images including images with the exception of a background and sectional images including only a background located in the neighborhood of said sectional images, and investigating mean values and standard deviations of the prescribed characteristic value by way of a background estimated previously in another sectional images, when there exists a sectional image whose only one set of a mean value and a standard deviation of the prescribed characteristic value are estimated in the neighborhood thereof, thus issuing a command so as to estimate a mean value and a standard deviation of the prescribed characteristic value of a sectional image except said background;

a mean value and standard deviation interpolation/extrapolation means, when receiving a command to estimate a mean value and a standard deviation of the prescribed characteristic value in the sectional image except said background, estimating to be outputted by averaging both of mean values and standard deviations of the prescribed characteristic value of a sectional image including only the background in the neighborhood thereof, simultaneously, outputting an estimated sectional image selection command signal so as to select next sectional image; and

an estimated statistic storage means storing the mean value and the standard deviation of the prescribed characteristic value estimated previously to be outputted whenever necessary, and

said threshold generation object detectable and background removal means comprising;

a threshold generation means, when a command for calculating a threshold is entered therein after completing whole mean values and standard deviations of the prescribed characteristic value in the whole sectional images, by using the mean value and the standard deviation in the whole sectional images, a second threshold is calculated to be defined in such a way that also a constant set beforehand is multiplied by the standard deviation of the prescribed characteristic value of the background estimated previously, then the above multiplied number is added to a mean value of the prescribed characteristic value of the background estimated previously, subsequently, calculating it all over the pictures to be outputted; and

a threshold processing means judging pixels within the threshold as a background while using said two kinds of thresholds and judging pixels without the threshold as an object of detection target.

7. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means comprises a minimum standard deviation criterion background only sectional image selection means outputting sectional images selects the sectional images in answer to specified number in order of the smaller value of a standard deviation of the prescribed characteristic value while taking such sectional images to be the sectional image whose probability of including only the background is high with the mean value and the standard deviation of the prescribed characteristic value of the sectional image as inputs, a background only sectional image selection means compares a standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is high with a standard deviation of the prescribed characteristic value of another sectional images, thus judging to be outputted the sectional image having a standard deviation whose difference between the standard deviation concerned and a standard deviation of the prescribed value of the sectional image whose probability of including only the background is high as a sectional image including only a background, and a background only sectional image statistic storage means storing the location of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, thus outputting them whenever necessary.

8. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means further comprises a standard deviation difference background only sectional image selection means selecting to be outputted a sectional image whose standard deviation is of the most nearest value of the standard deviation of the prescribed characteristic value instructed beforehand, a background only sectional image selection means comparing a standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background with a standard deviation of the prescribed characteristic value of another sectional images, thus judging to be outputted the sectional image having a standard deviation whose difference between the standard deviation concerned and a standard deviation of the prescribed value of the sectional image whose probability of including only the background is high as a sectional image including only a background, and a background only sectional image statistic storage means storing the location of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, thus outputting them whenever necessary.

9. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means comprises a standard deviation reference background only sectional image selection means which judges to be outputted sectional images in answer to specified number in order of the most nearest value to the standard deviation of the prescribed characteristic value instructed beforehand, while regarding as the sectional image whose probability of including only the background is high, a background only sectional image selection means comparing a standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is high with a standard deviation of the prescribed characteristic value of another sectional images, thus judging to be outputted the sectional image having a standard deviation whose difference between the standard deviation concerned and a standard deviation of the prescribed value of the sectional image whose probability of including only the background is high as a sectional image including only a background, and a background only sectional image statistic storage means storing the location of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, thus outputting them whenever necessary.

10. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means comprises a mean value and standard deviation criterion background only sectional image selection means for outputting a sectional image whose values are of the most nearest values to the mean value and the standard deviation of the prescribed characteristic value instructed beforehand as a sectional image whose probability of including only a background is high, a background only sectional image selection means for comparing a standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is high with a standard deviation of the prescribed characteristic value of another sectional images, thus judging to be outputted the sectional image having a standard deviation whose difference between the standard deviation concerned and a standard deviation of the prescribed value of the sectional image whose probability of including only the background is high as a sectional image including only a background, and a background only sectional image statistic storage means storing the location of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, thus outputting them whenever necessary.

11. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means comprises a mean value and standard deviation criterion background only sectional image selection means which outputs sectional images in answer to specified number in order of nearer value to the mean value and the standard deviation of the prescribed characteristic value instructed beforehand as a sectional image whose probability of including only a background is high, a background only sectional image selection means for comparing a standard deviation of the prescribed characteristic value of the sectional image whose probability of including only the background is high with a standard deviation of the prescribed characteristic value of another sectional images, thus judging to be outputted the sectional image having a standard deviation whose difference between the standard deviation concerned and a standard deviation of the prescribed value of the sectional image whose probability of including only the background is high as a sectional image including only a background, and a background only sectional image statistic storage means storing the location of the sectional image including only the background and the mean value and the standard deviation of the prescribed characteristic value of the sectional image concerned, thus outputting them whenever necessary.

12. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means selects a sectional image whose probability of including only a background is high from sectional image involved in an area instructed beforehand.

13. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said background sectional image selection means selects a sectional image whose probability of including only a background is high from sectional images involved in a plurality of areas instructed beforehand.

14. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said threshold generation object detectable and background removal means comprises a threshold generation means, when receiving a command for calculating threshold after the mean value and the standard deviation of the prescribed characteristic value by way of the background in the whole sectional image had been estimated, a first threshold is obtained in such a way that the method causes a standard deviation multiplied by a constant given beforehand to be subtracted from the mean value by using the mean value and the standard deviation of the prescribed characteristic value by way of the estimated background in the whole sectional images for the sake of detecting object and removing background, and a second threshold is obtained in such a way that it causes a standard deviation multiplied by a constant given beforehand to be added to the mean value, thus the first and the second thresholds are calculated over whole picture to be outputted, a threshold processing means taking pixels involved between two thresholds to be a background, a background candidate area detection means detecting area connecting together to the pixel judged as the background, by way of an area to be candidate of a background, and a background judgement means taking a candidate area including the greatest number of sectional images whose probability of including only the background is high to be background areas and taking another candidate areas to be target objects.

15. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said threshold generation object detectable and background removal means comprises a threshold generation means, when receiving a command for calculating threshold after the mean value and the standard deviation of the prescribed characteristic value by way of the background in the whole partial image had been estimated, a first threshold is obtained in such a way that the method causes a standard deviation multiplied by a constant given beforehand to be subtracted from the mean value by using the mean value and the standard deviation of the prescribed characteristic by way of the estimated background in the whole sectional images for the sake of detecting object and removing background, and a second threshold is obtained in such a way that the method causes a standard deviation multiplied by a constant given beforehand to be added to the mean value, thus the first and the second thresholds are calculated over whole picture to be outputted, a threshold processing means taking pixels involved between two thresholds to be a background, a background candidate area detection means detecting area connecting together to the pixel judged as the background, by way of an area to be candidate of a background, and a background judgement means taking a candidate area including the greatest number of sectional images including only the background to be background areas and taking another candidate areas to be target objects.

16. A device of object detectable and background removal for enabling an object to be automatically detected minutely and accurately as far as contours, as claimed in claim 6, wherein said threshold generation object detectable and background removal means comprises a threshold generation means, when receiving a command for calculating threshold after the mean value and the standard deviation of the prescribed characteristic value by way of the background in the whole partial image had been estimated, a first threshold is obtained in such a way that the method causes a standard deviation multiplied by a constant given beforehand to be subtracted from the mean value by using the mean value and the standard deviation of the prescribed characteristic by way of the estimated background in the whole sectional images for the sake of detecting object and removing background, and a second threshold is obtained in such a way that the method causes a standard deviation multiplied by a constant given beforehand to be added to the mean value, thus the first and the second threshold are calculated over whole picture to be outputted, a threshold processing means taking pixels involved between two thresholds to be a background, a background candidate area detection means detecting area connecting together to the pixel judged as the background, by way of an area to be candidate of a background, and a background judgement means taking a candidate area including the smallest number of sectional images except backgrounds to be background areas and taking another candidate areas to be target objects.