Moving object measurement device employing a three-dimensional analysis to obtain characteristics of the moving object

• US 5,764,786 A
• Filed: 06/10/1994
• Issued: 06/09/1998
• Est. Priority Date: 06/10/1993
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Oh-Yoh Keisoku Kenkyusho
• Original Assignee: Kabushiki Kaisha Oh-Yoh Keisoku Kenkyusho
• Inventors: Nakamura, Toru, Shimizu, Masao, Sekiguchi, Dairoku, Kuwashima, Shigesumi
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Davis, Monica S.
• Application Number:
  US08/381,864
• Time in Patent Office: 1,460 Days
• Field of Search: 382/107, 382/285, 382/291, 382/302, 354/402
• US Class Current: 382/107
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01B 11/022 by means of tv-camera scanning
• CPC Class Codes: G01B 11/04 specially adapted for measuring length or width of objects while moving
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

The present invention defines the area where the targeted moving object can move in the three-dimensional real spatial coordinate system, then maps it to the two-dimensional area corresponding to the capturing method and the capturing range of the capturing device, and by limiting the image processing for obtaining the coordinate to that area, obtains the coordinate of the moving object in the three-dimensional real space rapidly by using the signal which is capturing the moving object. Furthermore, the present invention is able to control the capturing direction and the capturing range which correspond to the movement of the capturing target, to overlap the image signal with the data which is always changing by the capturing condition and the movement of the measurement data, to improve the detection precision, and to track automatically and smoothly.

Claims

1. A moving object measurement device comprising:

first capturing means which captures an image of the moving object;

driving means to define a capturing direction and a capturing range of the capturing means;

image processing means to calculate a location of the moving object in an image included in the image signal obtained by the capturing means;

location calculation means which calculates coordinates of the moving object in real space based on location information on the location obtained by the image processing means and the capturing direction and the capturing range of the capturing means responsive to obtaining the image signal;

three-dimensional area setting means to set an area where the moving object can move in a three-dimensional real spatial coordinate system;

area mapping means to map the area in the three-dimensional real spatial coordinate system to a two-dimensional area corresponding to the capturing direction and the capturing range of the capturing means; and

two-dimensional area setting means to provide the two-dimensional area to the image processing means, wherein the image processing means includes area limiting means to limit a calculation of the location of the moving object to the two-dimensional area set by the two-dimensional area setting means.

2. A moving object measuring device according to claim 1, further comprising three-dimensional vector setting means to define a moving velocity vector of the moving object in the three-dimensional real special coordinate system;

vector mapping means to map the moving velocity vector in the three-dimensional real spatial coordinate system to a two-dimensional moving velocity vector corresponding to the capturing direction and the capturing range of the capturing means;

vector setting means to set the two-dimensional moving velocity vector for each corresponding capturing means, wherein the image processing device further includes means to detect a change of the location of the moving object between a current screen and a screen that is pre-defined steps before in the two-dimensional area limited by the above area limiting means; and

means to start the location calculation means if a detected change of location is within the capturing range defined in advance for the two-dimensional moving velocity vector set by the vector setting means.

3. A moving object measurement device according to claims 1 and 2, wherein the above capturing means includes a universal head adapted to rotate about at least one axis and a capturing device fixed on the universal head, and the driving means includes angle setting means to set the rotation angle around each axis of the above universal head and a zoom lens means attached to the capturing device.

4. A moving object measurement device according to claim 3, wherein the angle setting means includes, for at least one axis, a motor to rotate the above universal head, an angle encoder to detect the actual rotation angle, angle control means to control the motor to reduce the difference of a detected angle and an objective azimuth angle set by the angle setting means, and the zoom lens means includes a zoom driving motor to set the zooming position, a zoom encoder to detect the actual zooming position, and zooming control means to control the zooming drive motor to reduce a difference of a detected zooming position and an objective position set by the driving means, and the output of the angle encoder and the output of the zooming encoder are supplied to the location calculation means as the location information on the capturing direction and the capturing range of the capturing means.

5. A moving object measurement device according to claim 1 further comprising motion data setting means to determine motion data predicted for the moving object in the three-dimensional real spatial coordinate system, and drive control means to control the driving means based on the motion data.

6. A moving object measurement device according to claim 5, wherein the motion data setting means includes vector setting means to define a moving velocity vector that the moving object can have, and the drive control means includes vector mapping means to map the moving velocity vector to the two-dimensional moving velocity vector corresponding to the capturing range of the above capturing means, and drive limiting means to control the driving means to prevent the direction of the capturing means from changing velocity more than the mapped two-dimensional moving velocity vector.

7. A moving object measurement device according to claim 1 further comprising a drive control means to control at least the capturing direction or the capturing range of the above capturing means by the above driving means based on the location in the image obtained by the above image processing means.

8. A moving object measurement device according to claim 7, wherein the above image processing means includes means to calculate weighted averages of locations in the image of multiple moving objects.

9. A moving object measurement device according to claim 1 further comprising drive control means to control setting of at least the capturing direction and the capturing range of the above capturing means with the above driving means based on a coordinate in real space obtained by the location calculation device.

10. A moving object measurement device according to claim 9 further comprising vector setting means to set a moving velocity vector that the moving object can have, wherein the driving control means includes vector mapping means to map the moving velocity vector to the two-dimensional moving velocity vector corresponding to the capturing range of the above capturing means, drive limiting means to control the driving means to prevent the direction of the capturing means from changing in the velocity more than the mapped two-dimensional moving velocity vector.

11. A moving object measurement device according to claim 1 further comprising recording means to record motion data measured with the image measuring means, and means to detect that the moving object moves out of the capturing range of the capturing means, wherein the above drive control means includes means to control the capturing means based on past motion data recorded in the recording means.

12. A moving object measurement device according to claim 11, wherein the means to control the capturing means includes means to predict a moving direction of the moving object based on the past motion data.

13. A moving object measurement device according to claim 11, wherein the means to control the capturing means includes means to control the capturing direction of the capturing means so that the capturing directions becomes a direction defined in advance.

14. A moving object measurement device according to claims 1 and 13, further comprising means to set shape and a motion data of the moving object in the three-dimensional real spatial coordinate system, means to map the shape and the motion data in the three-dimensional real spatial coordinate system to the two-dimensional area data corresponding to the capturing range of the capturing means, and means to overlap the mapped shape and motion data with the image signal output by the capturing means.

15. A moving object measurement device according to claim 1 further comprising a plurality of capturing means, the image processing means, and the motion data calculation devices, and also includes precision estimation means to estimate a measurement precision of the motion data obtained by the motion data calculation means in the three-dimensional real space coordinate system-based on a location of the above capturing means, a precision mapping means to map the measurement precision in the three-dimensional real spatial coordinate system to the two-dimensional area data corresponding to each capturing area of the capturing means, and means to overlap the precision information based on the mapped measurement precision with a image signal output by the capturing means.

16. A moving object measurement device according to claims 1 and 15 further compromising a plurality of capturing means, each of which captures the moving object, and a plurality of driving means to set a capturing area of the multiple capturing means individually, and also includes means to overlap information on the capturing area of other capturing means with at least one output image signal of the first capturing means.

17. A moving object measurement device according to claim 16, wherein the overlapping means includes means to obtain a mutual relation of the capturing area of the plurality of capturing means by capturing setting value by the driving means, means to distribute the mutual relation corresponding to each capturing means, and processing means to process an image output by a corresponding capturing means based on the mutual relation distributed by the distribution means.

18. A moving object measurement device according to claim 1 further comprising multiple image processing means, wherein each of the image processing means has a different threshold value to separate the moving object from a background and provide means to select credible location data from data output by the multiple image processing means.

19. A moving object measurement device according to claim 1 further comprising multiple image processing means, wherein each of the multiple image processing means has a different valid range which can detect the moving object, and means to select credible location data from data output by the multiple image processing means.

20. A moving object measurement device according to claim 1 further comprising a plurality of the capturing means and a plurality of the image processing means, wherein each of the image processing means has a different valid range which can detect the moving object, and means to select credible location data from the data outputted by the multiple image processing means.

21. A moving object measurement device according to claim 1, wherein the image processing means includes means to calculate an area of the moving object in a screen included in the image signal obtained by the capturing means, and means to judge the credibility of the location obtained by the image processing means by the change of the area obtained by the image processing means.

22. A moving object measurement device according to claims 1 and 21, wherein the location calculation means sets a projection center of a camera lens of the capturing means, and includes means to transform a subject of the moving object captured with a symmetry point on this projection center, to a location on a projection plane which is on the symmetry point of a surface of the capturing device, and setting a vector from the projection center to the subject of the image transformed by the transformation means, means to calculate a distance between the vector and constraint plane on which the moving object exists, and the three-dimensional spatial coordinate of the targeted object based on a rotation angle of the above camera.

23. A moving object measurement device according to claims 1 and 21, wherein the location calculation means includes a means to define transformation variables required for calculating a location of a projection center in the image, which is a center location of a lens included in the capturing means and the three-dimensional spatial coordinate of the moving object, and calculates a spatial location coordinate by capturing a reference point, which is fixed on a constraint plane and has a known spatial location coordinate, and means to calculate the projection center and transformation variables by capturing a reference point again while changing the projection center of the camera and the transformation variables gradually based on the calculation result.

24. A moving object measurement device according to claims 1 and 21, further comprising means to operate the driving means of the capturing means for multiple operation points and to obtain a value to drive the driving means as an electric signal, wherein the location calculation means includes means to obtain a relation of the electric signal for the multiple operation points and the location of a projection center of a lens including in a camera in the capturing means, and to calculate a three-dimensional spatial coordinate of the moving object by obtaining relation of a location of the projection center and a value to drive the driving means based on the relation of the multiple operation points.

25. A moving object measurement device according to claim 1, further comprising movement prediction means to predict a moving direction of the moving object based on the real spatial coordinate of the moving object obtained by the location calculation means, and drive control means to control the driving means using a prediction result of the movement prediction means, wherein the movement prediction means includes means to predict the location of the moving object for multiple times at an interval shorter than the location calculation of the location calculation means, and the drive control means includes means to send a control objective value to the driving means based on the predicted location of the moving objects at the interval shorter than the location calculation of the location calculation means.

26. A moving object measurement device according to claims 1 and 25, wherein the capturing means includes color camera and the image processing means includes a color extraction means to separate the moving object from a background using colors from the captured images.

27. A moving image measurement device according to claim 1, further comprising movement prediction means to predict a moving direction of the moving object based on the real spatial coordinate of the moving object obtained by the location calculation means, and drive control means to control the driving means based on a result of a prediction of movement by the prediction means, wherein the drive control means includes means to obtain one of illumination distribution information and white balance distribution information in the capturing range of the capturing means in advance and to adjust one of a diaphragm and a white balance of a camera included in the capturing means by controlling the driving means based on one of the illumination distribution and white balance information.

28. A moving object measurement device according to claim 4 further comprising motion data setting means to determine motion data predicted for the moving object in the three-dimensional real spatial coordinate system, and drive control means to control the driving means based on the motion data.

29. A moving object measurement device according to claim 4 further comprising a drive control means to control at least the capturing direction or the capturing range of the above capturing means by the above driving means based on the location in the image obtained by the above image processing means.

30. A moving object measurement device according to claim 4 further comprising drive control means to control setting of at least the capturing direction and the capturing range of the above capturing means with the above driving means based on a coordinate in real space obtained by the location calculation device.

31. A moving object measurement according to claim 4 further comprising recording means to record motion data measured with the image measuring means, and means to detect that the moving object moves out of the capturing range of the capturing means, wherein the above drive control means includes means to control the capturing means based on past motion data recorded in the recording means.

32. A moving object measurement device according to claim 14 further comprising a plurality of capturing means, the image processing means, and the motion data calculation devices, and also includes precision estimation means to estimate a measurement precision of the motion data obtained by the motion data calculation means in the three-dimensional real space coordinate system-based on a location of the above capturing means, a precision mapping means to map the measurement precision in the three-dimensional real spatial coordinate system to the two-dimensional area data corresponding to each capturing area of the capturing means, and means to overlap the precision information based on the mapped measurement precision with a image signal output by the capturing means.

33. A moving object measurement device according to claim 15 further compromising a plurality of capturing means, each of which captures the moving object, and a plurality of driving means to set a capturing area of the multiple capturing means individually, and also includes means to overlap information on the capturing area of other capturing means with at least one output image signal of the first capturing means.

34. A moving object measurement device according to claim 17, wherein the overlapping means includes means to obtain a mutual relation of the capturing area of the plurality of capturing means by capturing setting value by the driving means, means to distribute the mutual relation corresponding to each capturing means, and processing means to process an image output by a corresponding capturing means based on the mutual relation distributed by the distribution means.

35. A moving object measurement device according to claim 17, further comprising a plurality of the capturing means and the image processing means, wherein each of the image processing means has a different valid range which can detect the moving object, and means to select credible location data from the data outputted by the above multiple image processing means.

36. A moving object measurement device according to claim 17, wherein the image processing means includes means to calculate an area of the moving object in a screen included in the image signal obtained by the capturing means, and means to judge the credibility of the location obtained by the image processing means by the change of the area obtained by the image processing means.

37. A moving object measurement device according to claim 24, further comprising movement prediction means to predict a moving direction of the moving object based on the real spatial coordinate of the moving object obtained by the location calculation means, and drive control means to control the driving means using a prediction result of the movement prediction means, wherein the movement prediction means includes means to predict the location of the moving object for multiple times at the interval shorter than the location calculation of the location calculation means, and the drive control means includes means to send a control objective value to the driving means based on the predicted location of the moving objects at the interval shorter than the location calculation of the location calculation means.

38. A moving image measurement device according to claim 1 further comprising movement prediction means to predict a moving direction of the moving object based on the real spatial coordinate of the moving object obtained by the location calculation means, and drive control means to control the driving means based on a result of a prediction of movement by the prediction means, wherein the drive control means includes means to obtain one of illumination distribution information and white balance distribution information in the capturing range of the capturing means in advance and to adjust one of a diaphragm and a white balance of a camera included in the capturing means by controlling the driving means based on one of the illumination distribution and white balance information.