Method and apparatus for detection of target object with improved robustness

• US 5,291,563 A
• Filed: 12/12/1991
• Issued: 03/01/1994
• Est. Priority Date: 12/17/1990
• Status: Expired due to Fees

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Maeda, Eisaku
• Primary Examiner(s): Couso, Jose L.
• Application Number:
  US07/805,612
• Time in Patent Office: 810 Days
• Field of Search: 382/1, 382/30, 382/34, 382/39, 382/48, 358/103, 358/105, 358/107, 358/125
• US Class Current: 382/103
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation

Abstract

An object detection system and a state transition detection system capable of realizing the improved robustness, general applicability, and customizability. In the system, features for the target image are calculated, where the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the target image and the reference image, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1, 1) direction in the distribution of brightness of corresponding points in each target area on the target image and the reference image; and the calculated features for the target image is classified into a first class corresponding to a presence of the target object and a second class corresponding to an absence of the target by using a classifier, according to which a detection result indicating the presence or absence of the target object is outputted.

Claims

1. An object detection apparatus for detecting a presence of a target object in prescribed target areas on a target image taken from a fixed camera position, comprising:

camera means, located at said fixed camera position, for taking the target image, a reference image in which the target object is known to be absent, and training images taken in which the target object is known to be present;

feature calculation means for calculating features for the target image taken by the camera means, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the target image and the reference image, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1,

     1) direction in the distribution of brightness of corresponding points in each target area on the target image and the reference image;

training means for training a classifier to classify the features calculated by the feature calculation means into those corresponding to a presence of the target object and those corresponding to an absence of the target object, by applying a training process using the training images taken by the camera means;

classifier means for classifying the features calculated by the feature calculation means into a first class corresponding to a presence of the target object and a second class corresponding to an absence of the target object by using the classifier trained by the training means; and

output means for outputting a detection result indicating the presence of the target object in the prescribed target areas on the target image taken by the camera means when the features for the target image calculated by the feature calculation means are classified into the first class by the classifier means, and the absence of the target object in the prescribed target areas on the target image taken by the camera means when the features for the target image calculated by the feature calculation means are classified into the second class by the classifier means.

2. The apparatus of claim 1, further comprising normalization means for normalizing brightness in each of the target image and the reference image, and wherein the feature calculation means calculates the features by using the target image and the reference image normalized by the normalization means.

3. The apparatus of claim 2, wherein the normalization means normalizes the brightness in each of the target image and the reference image such that a mean and a variance of the brightness in every target area of each of the target image and the reference image become uniform.

4. The apparatus of claim 1, further comprising partitioning means for dividing each target area on the target image and the reference image into a prescribed form of partitions, and wherein the feature calculation means calculates the features for each of the partitions.

5. The apparatus of claim 4, wherein the classifier means classifies the features in terms of a feature vector in which the features calculated by the feature calculation means for each of the partitions are arranged in a prescribed order assigned to the partitions.

6. The apparatus of claim 1, wherein the training means applies one of a Fisher'"'"'s linear discrimination method, a k-nearest neighbor method, a neural network method, and a stratified linear discrimination method as the training process.

7. The apparatus of claim 1, wherein in the training process, the feature calculation means calculates the features for the training images, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on each of the training images and the reference image, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1, 1) direction in the distribution of brightness of corresponding points in each target area on each of the training images and the reference image, and the training means utilizes the features for the training images in training the classifier.

8. The apparatus of claim 1, wherein the feature calculation means also calculates features for sequential target images, the features includes at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the sequential target images, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1, 1) direction in the distribution of brightness of corresponding points in each target area on the sequential target images;

and which further comprises state transition classifier means for classifying the features for the sequential target images into a first class corresponding to an occurrence of a state transition between the sequential target images and and a second class corresponding to an absence of the state transition by using a state transition classifier.

9. The apparatus of claim 8, wherein the output means outputs the detection result indicating the presence of the target object and the absence of the target object according to a classification by the classifier means and a classification by the state transition classifier means.

10. The apparatus of claim 8, further comprising training means for training the state transition classifier used in the state transition classifier means by applying a training process using sequential training images taken from said fixed camera position in which the target object is known to be present.

11. The apparatus of claim 10, wherein the training means applies one of a Fisher'"'"'s linear discrimination method, a k-nearest neighbor method, a neural network method, and a stratified linear discrimination method as the training process.

12. The apparatus of claim 10, wherein in the training process, the feature calculation means calculates the features for the sequential training images, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the sequential training images, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1, 1) direction in the distribution of brightness of corresponding points in each target area on the sequential training images, and the training means utilizes the features for the sequential training images in training the state transition classifier.

13. A method for detecting a presence of a target object in prescribed target areas on a target image taken from a fixed camera position, comprising the steps of:

taking a reference image in which the target object is known to be absent from said fixed camera position;

taking the target image from said fixed camera position;

calculating features for the target image, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the target image and the reference image, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1,

     1) direction in the distribution of brightness of corresponding points in each target area on the target image and the reference image;

taking training images in which the target object is known to be present from said fixed camera position;

training a classifier to be used in classifying the features into those corresponding to a presence of the target object and those corresponding to an absence of the target object, by applying a training process using the training images to the classifier;

classifying the features into a first class corresponding to a presence of the target object and a second class corresponding to an absence of the target object by using the classifier trained at the training step; and

outputting a detection result indicating the presence of the target object in the prescribed target areas on the target image when the features for the target image calculated at the calculating step are classified into the first class at the classifying step, and the absence of the target object in the prescribed target areas on the target image when the features for the target image calculated at the calculating step are classified into the second class at the classifying step.

14. The method of claim 13, further comprising the step of normalizing brightness in each of the target image and the reference image, and wherein at the calculating step the features are calculated by using the target image and the reference image normalized at the normalization step.

15. The method of claim 14, wherein at the normalizing step the brightness in each of the target image and the reference image is normalized such that a mean and a variance of the brightness in every target area of each of the target image and the reference image become uniform.

16. The method of claim 13, further comprising the step of dividing each target area of the target image and the reference image into a prescribed form of partitions, and wherein at the calculating step the features are calculated for each of the partitions.

17. The method of claim 16, wherein at the classifying step, the features are classified in terms of a feature vector in which the features calculated at the calculating step for each of the partitions are arranged in a prescribed order assigned to the partitions.

18. The method of claim 13, wherein at the training step, one of a Fisher'"'"'s linear discrimination method, a k-nearest neighbor method, a neural network method, and a stratified linear discrimination method is applied to the classifier as the training process.

19. The method of claim 13, wherein at the training step, the features for the training images are calculated in the training process, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on each of the training images and the reference image, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1, 1) direction in the distribution of brightness of corresponding points in each target area on each of the training images and the reference image, and the features for the training images are utilized in the training process.

20. The method of claim 13, further comprising the steps of:

calculating features for sequential target images, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the sequential target images, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1,

     1) direction in the distribution of brightness of corresponding points in each target area on the sequential target images; and

classifying the features for the sequential target images into a first class corresponding to an occurrence of a state transition between the sequential target images and a second class corresponding to an absence of the state transition by using a state transition classifier.

21. The method of claim 20, wherein at the outputting step the detection result indicating the presence of the target object is outputted according to a classification using the classifier means and a classification using the state transition classifier.

22. The method of claim 20, further comprising the steps of:

taking sequential training images in which the target object is known to be present, from said fixed camera position; and

training the state transition classifier used at the step for classifying the features for the sequential target images by applying a training process using the sequential training images to the state transition classifier.

23. The method of claim 22, wherein at the training step, one of a Fisher'"'"'s linear discrimination method, a k-nearest neighbor method, a neural network method, and a stratified linear discrimination method is applied to the state transition classifier as the training process.

24. The method of claim 22, wherein at the training step, the features for the sequential training images are calculated in the training process, the features including at least one of a quantity expressing a difference of contributions by principal axis components and secondary axis components in a distribution of brightness of corresponding points in each target area on the sequential training images, and a quantity expressing a difference between a direction of a principal axis with respect to a vector (1, 1) direction in the distribution of brightness of corresponding points in each target area on the sequential training images, and the features for the sequential training images are utilized in the training process.