Method and apparatus for extracting features of moving objects

• US 5,687,249 A
• Filed: 09/01/1994
• Issued: 11/11/1997
• Est. Priority Date: 09/06/1993
• Status: Expired due to Fees

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Kato, Koichi
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Davis, Monica S.
• Application Number:
  US08/299,072
• Time in Patent Office: 1,167 Days
• Field of Search: 382/104, 382/190, 382/100, 340/937, 340/936, 340/935, 340/934, 358/105, 348/135, 348/577, 348/578, 348/136, 348/700, 348/25
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G08G 1/017 identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098

Abstract

The present invention is intended to reconstruct or acquire the shape of an object without rotating it. A TV camera with a fish-eye lens is used to conduct the image-sensing of a vehicle as a moving object over a certain section of its path of travel and a background image is subtracted from each captured image to obtain a silhouette of the vehicle. The loci of movement of the vehicle'"'"'s tires in each image are calculated and used to calculate the positions of the viewpoint of the TV camera and the vehicle relative to each other in each image. The shape of the vehicle is reconstructed by projecting the silhouette for each captured image into a projection space while holding the viewpoint of the TV camera and the vehicle at the calculated relative positions.

Claims

1. A method for acquiring a feature of a moving object, said method comprising:

an image capturing step using a fixed TV camera for performing image-sensing of said moving object at least two times along a predetermined section of the path of travel of said moving object from a position on a line crossing the path of travel of said moving object;

a silhouette extraction step for calculating the difference between each image of said moving object captured by said TV camera and its background image to generate a silhouette for said each image of said moving object;

a relative position calculation step for calculating the relative positions of the viewpoint of said TV camera and said moving object on the basis of information about a surface feature point of said moving object in said each image; and

a shape extraction step for extracting the shape of said moving object as a reconstructed shape by projecting said silhouette of said each image into a three-dimensional projection space assumed to be large enough to accommodate an assumed version of the shape of said moving object on the assumption that said viewpoint of said TV camera and said moving object are held at said relative positions.

2. The method of claim 1, which further comprising a step of generating said background image from said images captured by said TV camera.

5. The method of any one of claims 1 through 4, wherein said relative position calculation step comprises the steps of:

projecting each of said images captured by said TV camera onto a spherical surface to form a spherical projection image;

projecting each said spherical projection image onto a planar surface to form a planar projection image;

arraying said planar projection images onto a time axis to generate a spatio-temporal image;

detecting a ground-contact-line image of said moving object from said spatio-temporal image; and

calculating a perpendicular angle at which to view a ground-contact-line corresponding to said detected ground-contact-line image to thereby calculate said relative positions of said moving object and said TV camera.

6. The method of claim 5, wherein said relative position calculation step for said moving object comprises the steps of:

calculating the distance between said TV camera and said moving object at a predetermined position from said perpendicular angle at said predetermined position;

calculating the speed of said moving object from the inclination of said detected ground-contact-line image; and

calculating the distance of travel of said moving object from said predetermined position on the basis of said speed of said moving object and the time elapsed from the capturing of the image of said moving object at said predetermined position to the capturing of another image of said moving object.

7. The method of any one of claims 1 through 4, which further comprises a step of splitting into minute patches a part of said reconstructed shape of said moving object corresponding to a surface thereof desired to observe, acquiring luminance, color or similar information on the patches of each captured image corresponding to said patches, and statistically processing said information for said patches to obtain their value.

8. The method of claim 1 or 2, wherein said TV camera has a wide-angle lens.

9. The method of claim 1 or 2, which further comprises the steps of:

projecting each of said images captured by said TV camera onto a spherical surface to obtain a spherical projection image;

projecting each of said spherical projection images onto a planar surface to obtain a planar projection image;

arraying said planar projection images on a time axis to generate a spatio-temporal image;

obtaining, from said spatio-temporal image, images of loci of parts of said moving object at the side facing said TV camera and the side opposite therefrom; and

calculating speeds of said moving object from the inclinations of said images of said loci to calculate the width of said moving object.

3. A method for acquiring a feature of a moving object, said method comprising:

an image capturing step using a TV camera for performing image-sensing of said moving object at least two times along a predetermined section of the path of travel of said moving object while turning the line of view of said TV camera to keep track of said moving object in its field of view at all times;

a silhouette extraction step for calculating the difference between each image of said moving object captured by said TV camera and its background image to generate a silhouette for said each image of said moving object;

a relative position calculation step for calculating the relative position of the center of turning of said line of view of said TV camera and said moving object on the basis of information of a surface feature point of said moving object in said each image; and

a shape extraction step for acquiring the shape of said moving object by projecting said silhouette of said each image into a three-dimensional projection space assumed to be large enough to accommodate an assumed version of the shape of said moving object on the assumption that the center of turning of said line of view of said TV camera and said moving object are held at said relative positions.

4. The method of claim 3, which further comprises a background image generating step for generating a background image form said images captured by said TV camera.

10. The method of claim 3 or 4, which further comprises the steps of:

detecting the entering of said moving object into said predetermined section; and

, upon detection of the entering of said moving object in said predetermined section, starting said image capturing step, detecting the speed of said moving object, and controlling the turning movement of said TV camera as a function of said detected speed.

11. The method of claim 10, wherein said step of detecting the entering of said moving object into said predetermined section is a step of always image-sensing the scene in the vicinity of one end of said predetermined section by said TV camera, removing a background image from each captured image, and deciding that said moving object has entered said predetermined section when an image larger than a predetermined size is detected in the remaining image.

12. The method of claim 11, wherein said speed of said moving object is detected from the size of the inter-frame difference of said remaining image in the direction of travel of said moving object.

13. An apparatus for acquiring a feature of a moving object, said apparatus comprising:

a stationary TV camera disposed apart from a path of travel of said moving object, for performing image-sensing of said moving object at least two times in a predetermined section of said path of travel, said TV camera having a wide-angle objective lens; and

an image processor which generates a silhouette of said moving object by subtracting a background image from each image captured by said TV camera, calculates the relative positions of the viewpoint of said TV camera and said moving object on the basis of information about a surface feature point of said moving object in each image captured by said TV camera, and reconstructs the shape of said moving object by projecting said silhouette of said moving object into a three-dimensional projection space while holding said viewpoint of said TV camera and said moving object at said relative positions.

14. An apparatus for acquiring a feature of a moving object, said apparatus comprising:

a TV camera disposed apart from a path of travel of said moving object, for image-sensing said moving object at least two times in a predetermined section of said path of travel;

means for turning said TV camera so that said moving object having entered said predetermined section stays in the field of view of said TV camera; and

an image processor which generates a silhouette of said moving object by subtracting a background image from each image captured by said TV camera, calculates the relative positions of the turning center of said TV camera and said moving object on the basis of information about a surface feature point of said moving object in each image captured by said TV camera, and reconstructs the shape of said moving object by projecting said silhouette of said moving object into a three-dimensional projection space while holding said turning center of said TV camera and said moving object at said relative positions.