Object detecting apparatus, method for detecting an object, and object detection program

• US 20070280540A1
• Filed: 06/04/2007
• Published: 12/06/2007
• Est. Priority Date: 06/05/2006
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Ikeda, Hiroo
• Granted Patent: US 8,311,273 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

Stationary or moving target object from a changing background in moving picture with changing background is detected. Image capturing unit 1 captures image from moving picture, and characteristic extracting unit 21 extracts characteristics for storing in characteristic storing unit 31. Pixel state determining unit 22 derives variance value for characteristics for a short period of time in the past stored in characteristic storing unit 31, stores variance value in characteristic variance value storing unit 32, obtains a ratio between the numbers of the variance values not more than a predetermined threshold value in the stored data and all the stored data, and determines the pixel state using these variance values and the ratio. First characteristic distance calculating unit 23 calculates a characteristic distance between the extracted characteristic and a preceding background model. Background model generating unit 24 adaptively generates a background model from characteristics in the characteristic storing unit 31 and characteristic storing unit 33 for background model generation using the characteristic distance and the pixel state determined as conditions. Second characteristic distance calculating unit 25 calculates a characteristic distance between the extracted characteristic and the latest background model, and object judging unit 26 detects an object using this characteristic distance.

Claims

1. An object detecting apparatus comprising:

pixel state determining unit that determines a pixel state indicative of temporal properties of pixel characteristics of an input image;

background model generating unit that adaptively generates a background model using said pixel state and a characteristic distance indicative of the degree of similarity between pixel characteristics of an input image and a preceding background model as conditions; and

object judging unit that judges an object based on a characteristic distance indicative of the degree of similarity between a generated background model and pixel characteristics of an input image.

2. The object detecting apparatus as defined in claim 1 wherein a pixel state has the temporal properties of pixel characteristics of an input image, and is in one of the following states:

a stationary state that is a stable state over a short period of time, a moving state that is an unstable state over a short period of time, a continuously stationary state that is a stable state over a long period of time, and a continuously moving state that is an unstable state over a long period of time.

3. The object detecting apparatus as defined in claim 1 wherein said pixel state determining unit determines a pixel state from variance values based on characteristics (from a latest value to a value a short while ago) stored in characteristic storing unit that time-sequentially stores pixel characteristics of an input image and a ratio R_S between the numbers of variance values not more than a threshold value for a stable state in characteristic variance value storing unit that time-sequentially stores said variance values and all stored data.

4. The object detecting apparatus as defined in claim 1 wherein pixel state determining unit determines that a pixel state is continuously stationary when a ratio R_S is not less than a threshold for a continuously stationary state, and a continuously moving state when said ratio R_S is not more than a threshold value for a continuously moving state;

and when a pixel state is neither continuously stationary nor continuously moving, said pixel state determining unit determines that a pixel state is stationary when a variance value is not more than a threshold value for a stable state, and moving when said variance value is more than said threshold value for a stable state.

5. The object detecting apparatus as defined in claim 1 wherein said background model generating unit judges that a pixel state is a stationary background when the pixel state is continuously stationary, generates a background model from characteristics stored in characteristic storing unit that time-sequentially stores pixel characteristics of an input image, and copies all characteristics stored in said characteristic storing unit into characteristic storing unit for background model generation;

said background model generating unit judges that a pixel state is a moving background when the pixel state is continuously moving, generates a background model from characteristics stored in said characteristic storing unit, and copies all characteristics stored in said characteristic storing unit into said characteristic storing unit for background model generation;

said background model generating unit judges that pixel characteristics of an input image are a background when a pixel state is stationary and a characteristic distance that indicates the degree of similarity between pixel characteristics of an input image and a preceding background model is not more than a threshold value, indicating a high degree of similarity, stores latest characteristics stored in said characteristic storing unit into said characteristic storing unit for background model generation, and generates a background model from characteristics stored in said characteristic storing unit for background model generation; and

no background model is generated and a preceding background model is utilized when a pixel state does not meet any of the above conditions.

6. The object detecting apparatus as defined in claim 1 wherein a background model is constituted by a mean of characteristics and a variance of characteristics.

7. The object detecting apparatus as defined in claim 1 wherein a background model is constituted by a weighted mean of characteristics and a weighted variance of characteristics;

and the weights in said weighted mean and said weighted variance are set so that the latest value is largest, and the older a value is, the smaller it becomes.

8. The object detecting apparatus as defined in claim 1 wherein a background model is constituted by a plurality of units of characteristics, i.e., a plurality of means, a plurality of variances and a plurality of weighting coefficients under assumption of mixture of normal distributions;

background models having weighting coefficients not less than a threshold value are selected from background models; and

a smallest characteristic distance among a plurality of characteristic distances derived from characteristics extracted from an input image and a plurality of means and a plurality of variances in the characteristics of said selected background model is recognized as the characteristic distance.

9. The object detecting apparatus as defined in claim 1 wherein characteristics are constituted by multi-dimensional vector;

a background model is constituted by multi-dimension of a mean vector and a covariance matrix; and

a variance value is a sum of diagonal components or a maximum value of the diagonal components of covariance matrices of characteristics (from a latest value to a value a short while ago) stored in characteristic storing unit.

10. An object detecting apparatus that judges an object by dividing an image comprising:

an image dividing unit that divides an input image;

a temporal characteristic determining unit that determines temporal characteristics of an image per divided image unit; and

an image unit specifying unit that analyzes temporal characteristics of a plurality of image units and specifies an image unit in which an object exists.

11. The object detecting apparatus as defined in claim 10 wherein temporal characteristic determining unit derives temporal characteristics from temporal transition information in a state determined by the relationship between the degree of temporal stability of characteristic information obtained per image unit and a continuity degree how long said degree of temporal stability continues.

12. The object detecting apparatus as defined in claim 10 wherein, when analyzing temporal characteristics of a plurality of image units, said image unit specifying unit extracts distinctive features of parts with similarity in temporal properties to each other from a time sequence of characteristic information obtained per image unit, and recognizes characteristic information different from said distinctive features as an image unit in which an object exists.

13. The object detecting apparatus as defined in claim 11 wherein characteristic information includes at least one of the following information of an image unit selected from the group consisting of the intensity, hue, each value of RGB, edge value, optical flow value, and the distance value in a distance image.

14. The object detecting apparatus as defined in claim 10 wherein said image unit is a pixel.

15. A method for detecting an object comprising:

a pixel state determining step that determines a pixel state having temporal properties of pixel characteristics of an input image;

a background model generating step that adaptively generates a background model using said pixel state and a characteristic distance that indicates the degree of similarity between pixel characteristics of an input image and a preceding background model as conditions; and

an object judging step that judges an object based on a characteristic distance that indicates the degree of similarity between a generated background model and pixel characteristics of an input image.

16. The method for detecting an object as defined in claim 15 wherein a pixel state has the temporal properties of pixel characteristics of an input image, and is in one of the following states:

a stationary state that is a stable state over a short period of time, a moving state that is an unstable state over a short period of time, a continuously stationary state that is a stable state over a long period of time, and a continuously moving state that is an unstable state over a long period of time.

17. The method for detecting an object as defined in claim 15 wherein a pixel state is determined from variance values based on characteristics (from a latest value to a value a short while ago) stored in characteristic storing unit that time-sequentially stores pixel characteristics of an input image and a ratio R_S between the numbers of variance values not more than a threshold value for a stable state in characteristic variance value storing unit that time-sequentially stores said variance values and all stored data in a pixel state determining step.

18. The method for detecting an object as defined in claim 15 wherein, in a pixel state determining step, a pixel state is determined to be continuously stationary when a ratio R_S is not less than a threshold for a continuously stationary state, and continuously moving when said ratio R_S is not more than a threshold value for a continuously moving state;

and when a pixel state is determined to be neither continuously stationary nor continuously moving, a pixel state is determined to be stationary when a variance value is not more than a threshold value for a stable state, and moving when said variance value is more than said threshold value for a stable state.

19. The method for detecting an object as defined in claim 15 wherein, in the background model generating step, a pixel state is judged to be a stationary background when the pixel state is continuously stationary, a background model is generated from characteristics stored in characteristic storing unit that time-sequentially stores pixel characteristics of an input image, and all characteristics stored in said characteristic storing unit are copied into characteristic storing unit for background model generation;

a pixel state is judged to be a moving background when the pixel state is continuously moving, a background model is generated from characteristics stored in said characteristic storing unit, and all characteristics stored in said characteristic storing unit are copied into said characteristic storing unit for background model generation;

pixel characteristics of an input image are judged to be a background when a pixel state is stationary and a characteristic distance that indicates the degree of similarity between pixel characteristics of an input image and a preceding background model is not more than a threshold value, indicating a high degree of similarity, latest characteristics stored in said characteristic storing unit are stored into said characteristic storing unit for background model generation, and a background model is generated from characteristics stored in said characteristic storing unit for background model generation; and

no background model is generated and a preceding background model is utilized when a pixel state does not meet any of the above condition.

20. The method for detecting an object as defined in claim 15 wherein the background model is constituted by a mean of characteristics and a variance of characteristics.

21. The method for detecting an object as defined in claim 15 wherein the background model is constituted by a weighted mean of characteristics and a weighted variance of characteristics;

and the weights in said weighted mean and said weighted variance are set so that the latest value is largest, and the older a value is, the smaller it becomes.

22. The method for detecting an object as defined in claim 15 wherein a background model is constituted by a plurality of units of characteristics i.e., a plurality of means, a plurality of variances, and a plurality of weighting coefficients under assumption of mixture of normal distributions;

a background model having weighting coefficients not less than a threshold value is selected from the background models; and

a smallest characteristic distance among a plurality of characteristic distances derived from characteristics extracted from an input image and a plurality of characteristic distance obtained from a plurality of means and variances of said selected background model is recognized as the characteristic distance.

23. The method for detecting an object as defined in claim 15 wherein characteristics are constituted by multi-dimensional vector;

a background model is constituted as multi-dimension by a mean vector and a covariance matrix; and

a variance value is a sum of diagonal components of covariance matrices or the maximum value of the diagonal components of characteristics (from a latest value to a value a short while ago) stored in characteristic storing unit.

24. A method for detecting an object, in which an object is judged by dividing an image, comprising:

an image dividing step that divides an input image;

a temporal characteristic determining step that determines temporal characteristics of an image per divided image unit; and

an image unit specifying step that analyzes temporal characteristics of a plurality of image units and that specifies image units in which an object exists.

25. The method for detecting an object as defined in claim 24 wherein, in the temporal characteristic determining step, temporal characteristics are derived from temporal transition information in a state determined by the relationship between the degree of temporal stability of characteristic information obtained per image unit and a continuity degree how long said degree of temporal stability continues.

26. The method for detecting an object as defined in claim 24 wherein, when temporal characteristics of a plurality of image units are analyzed in the image unit specifying step, distinctive features of parts with similarity in temporal properties to each other are extracted from the time sequence of characteristic information obtained per image unit;

and characteristic information different from said distinctive features is recognized as an image unit in which an object exists.

27. The method for detecting an object as defined in claim 24 wherein the image unit is a pixel.

28. An object detection program having a computer execute:

a pixel state determining processing that determines a pixel state having temporal properties of pixel characteristics of an input image;

a background model generating processing that adaptively generates a background model using said pixel state and a characteristic distance that indicates the degree of similarity between pixel characteristics of an input image and a preceding background model as conditions; and

an object judging processing that judges an object based on a characteristic distance that indicates the degree of similarity between a generated background model and pixel characteristics of an input image.

29. The object detection program as defined in claim 28 having a computer execute a pixel state determining processing wherein a pixel state is determined from variance values based on characteristics (from a latest value to a value a short while ago) stored in characteristic storing unit that time-sequentially stores pixel characteristics of an input image and a ratio R_S between the numbers of variance values not more than a threshold value for a stable state in characteristic variance value storing unit that time-sequentially stores said variance values and all stored data.

30. The object detection program as defined in claim 28 having a computer execute a pixel state determining processing wherein a pixel state is determined to be continuously stationary when a ratio R_S is not less than a threshold for a continuously stationary state, and continuously moving when said ratio R_S is not more than a threshold value for a continuously moving state;

and when a pixel state is determined to be neither continuously stationary nor continuously moving, a pixel state is determined to be stationary when a variance value is not more than a threshold value for a stable state, and moving when said variance value is more than said threshold value for a stable state.

31. The object detection program as defined in claim 28 having a computer execute a background model generating processing wherein,

when a pixel state is continuously stationary, a background model is generated from characteristics stored in characteristic storing unit that time-sequentially stores pixel characteristics of an input image, and all characteristics stored in said characteristic storing unit are copied into characteristic storing unit for background model generation;

when a pixel state is continuously moving, a background model is generated from characteristics stored in said characteristic storing unit, and all characteristics stored in said characteristic storing unit are copied into said characteristic storing unit for background model generation;

when a pixel state is stationary and a characteristic distance that indicates the degree of similarity between pixel characteristics of an input image and a preceding background model is not more than a threshold value, indicating a high degree of similarity, latest characteristics stored in said characteristic storing unit are stored into said characteristic storing unit for background model generation, and a background model is generated from characteristics stored in said characteristic storing unit for background model generation; and

no background model is generated and a preceding background model is utilized when a pixel state does not meet any of the above condition.

32. An object detection program having a computer that judges an object by dividing an image execute:

an image dividing processing that divides an input image;

a temporal characteristic determining processing that determines temporal characteristics of an image per divided image unit; and

an image unit specifying processing that analyzes temporal characteristics of a plurality of image units and that specifies an image unit in which an object exists.

33. The object detection program as defined in claim 32 having a computer execute a temporal characteristic determining processing wherein temporal characteristics are derived from temporal transition information in a state determined by the relationship between the degree of temporal stability of characteristic information obtained per image unit and a continuity degree how long said degree of temporal stability continues.

34. The object detection program as defined in claim 32 having a computer execute an image unit specifying processing wherein, when temporal characteristics of a plurality of image units are analyzed, distinctive features of parts with similarity in temporal properties to each other are extracted from the time sequence of characteristic information obtained per image unit;

and characteristic information different from said extracted distinctive features is recognized as an image unit in which an object exists.