Apparatus for modeling three dimensional information

• US 5,819,016 A
• Filed: 06/18/1996
• Issued: 10/06/1998
• Est. Priority Date: 10/05/1993
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Nakamura, Tatsuro, Suzuki, Kaoru, Hattori, Hiroshi, Wada, Takashi, Yagi, Minoru, Onoguchi, Kazunori, Watanabe, Mutsumi
• Primary Examiner(s): Fetting, Anton W.
• Application Number:
  US08/665,546
• Time in Patent Office: 840 Days
• Field of Search: 395/119, 395/152, 382/106, 382/154, 382/144, 382/145, 382/209, 382/218
• US Class Current: 345/419
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

A modelling apparatus including a three-dimensional information input unit for measuring and inputting three-dimensional information concerning objects in a space. An environmental model processing unit constructs an environmental model for each object in the space by comparing prestored object data to the three-dimensional information. A picture composing unit composes an artificial picture from a specific viewpoint within the space according to the environmental model constructed by the environmental model processing unit. An analyzing unit validates the environmental model by comparing the artificial picture composed by the picture composing unit with the three-dimensional information from the specific viewpoint. An observation control unit controls the measurement and input operation of the three-dimensional information input unit according to the result of validation by the analyzing unit.

Claims

1. A modeling apparatus comprising:

three-dimensional information input means for measuring and inputting pieces of three-dimensional information concerning objects in a space, said pieces of three-dimensional information comprising (I) three-dimensional geometric information composed of distance and shape data, and (ii) surface attribute information including at least one data item selected from a group consisting of color, pattern and reflectance,

environmental model processing means for constructing an environmental model for each object in said space by comparing prestored object data to the pieces of three-dimensional information input by said three-dimensional information input means,

picture composing means for composing an artificial picture viewed from a specific viewpoint within said space according to the environmental model constructed by said environmental model processing means,

analyzing means for comparing a predetermined plane region in the artificial picture composed by said picture composing means with a predetermined plane region viewed from a specific viewpoint and extracted from the pieces of three-dimensional information comprising three dimensional geometric information and surface attribute information input by said three-dimensional information input means and for providing a revise command for change of the environmental model when a difference is found between the plane region in said artificial picture and the plane region viewed from said specific viewpoint, and

observation control means responsive to the revise command, for changing a measurement position of said three-dimensional information input means according to the revise command,

wherein said observation control means includes means for changing the environmental model by varying the measurement position of said three-dimensional information input means to allow a difference between the plane region in the artificial picture and the plane region viewed from the specific viewpoint to be minimized in said analyzing means while causing said environmental model processing means to provide an environmental model within said space.

2. The modeling apparatus according to claim 1 wherein said three-dimensional information input means comprises

stereo measuring means for obtaining three-dimensional information on an object from a plurality of camera images,

three-dimensional measuring means for obtaining three-dimensional information on an object from a series of pictures taken by varying the focal distance of a single camera, and

light pattern projection control means for projecting a light pattern to the object and inputting its camera images into said stereo measuring means.

3. The modeling apparatus according to claim 2 wherein said environmental model processing means comprises

intra-environmental position detecting means for detecting a position of said three-dimensional information input means in a world coordinate system according to an amount of movement of the modeling apparatus,

three-dimensional environmental data updating means for transforming the three-dimensional information input by said three-dimensional information input means into data in the world coordinate system according to the position detected by said intra-environmental position detecting means,

three-dimensional environmental data memory updating means for tessellating said space into a three-dimensional grid of cells and for storing and updating a probability that a cell corresponds to an object in said space, and

object data managing means for constructing an environmental model description for an object in the space by collating cells having stored probabilities which are higher than a predetermined threshold value.

4. The modeling apparatus according to claim 3 wherein said picture composing means comprises

camera coordinate system transforming means for transforming the environmental model to data in a camera coordinate system based on a measuring position of said three-dimensional information input means,

model picture composing means which composes an artificial picture of said three-dimensional information input means according to the environmental model transformed by said camera coordinate system transforming means, and

display means for displaying the artificial picture.

5. The modeling apparatus according to claim 4 wherein said analyzing means comprises

comparative region means for extracting and comparing characteristic regions from said artificial picture and said three-dimensional information input by said three-dimensional information input means,

environmental model description revise command preparing means which, when a disagreement occurs between respective ones of said characteristic regions extracted and compared by said comparative region means, outputs a command to revise the environmental model description in accordance with the three-dimensional information, and

a mobile observation command preparing means, responsive to said command to revise the environmental model description, outputs a mobile observation command to said three-dimensional information input means.

6. The modeling apparatus according to claim 5 wherein said observation control means comprises

mobile mechanism control means which, in response to said mobile observation command, prepares a range, speed and steering angle command and outputs it as mobile control information to said three-dimensional information input means and

movement amount computing means for outputting the amount of movement for the detection of position according to said range, speed and steering angle to said intra-environmental position detecting means.

7. The modeling apparatus according to claim 6 wherein said object data managing means is responsive to said command to revise the environmental model description by using three-dimensional information newly measured and fed by said three-dimensional information input means.

8. The modeling apparatus according to claim 1 wherein said environmental model processing means comprises

object data memory means for previous storage of data relevant to an intra-environmental object,

recognition means for recognizing an intra-environmental object from the data stored in said object data memory means and environmental configuration and surface attribute data input from said three-dimensional information input means and determining the intra-environmental position of the recognized object,

object data disposition means for substituting the data relevant to the intra-environmental object recognized by said recognition means for the environmental configuration and surface attribute data obtained by said three-dimensional information input means in the intra-environmental position of the object as determined by said recognition means, and

environmental model memory means for storing the object data so substituted by said object data disposition means.

9. A modeling apparatus comprising:

three-dimensional information input means for measuring pieces of three-dimensional information using visual information input from a TV camera or ultrasonic sensors, said pieces of three-dimensional information comprising (I) three-dimensional geometric information composed of distance and shape data, and (ii) surface attribute information including at least one data item selected from a group consisting of color, pattern and reflectance,

environmental model processing means for managing the pieces of three-dimensional information input from said three-dimensional information input means and for constructing an environmental model,

picture composing means for composing an artificial picture viewed from a specific viewpoint within a space according to the constructed environmental model,

analyzing means for comparing a predetermined plane region in the composed artificial picture with a predetermined plane region viewed from said specific viewpoint and extracted from the pieces of three-dimensional information comprising three dimensional geometric information and surface attribute information, and for providing a revise command for change of the environmental model when a difference is found between the plane region in said artificial picture and the plane region viewed from said specific viewpoint,

observation control means responsive to the revise command from said analyzing means for changing a measurement position of said three-dimensional information input means,

said observation control means including means for distinguishing between outline and non-outline data of a target space in the pieces of three-dimensional information and for changing a measurement position of said three-dimensional information input means so as to sequentially construct a detailed environmental model starting with said outline data, and

said observation control means further including means for changing the environmental model by varying the measurement position of said three-dimensional information input means to allow a difference between the plane region in the artificial picture and the plane region viewed from the specific viewpoint to be minimized in said analyzing means while causing said environmental model processing means to provide an environmental model approximating the object within said space.

10. The modeling apparatus according to claim 9 wherein said three-dimensional information input means comprises

picture input means for input of pictures,

depth computing means which acquires depth using the pictures input from said picture input means, and

coordinate transforming means which transforms depth data from said depth computing means to coordinate data in a pre-established coordinate system to acquire three-dimensional data.

11. The modeling apparatus according to claim 9 wherein said observation control means comprises

outline data generating means which generates outline data, non-outline data and picture data for a target space from the data from said three-dimensional information input means,

accumulating means for accumulating the outline data, non-outline data and picture data generated by said outline data generating means,

surface information analyzing means which performs a surface information analysis of the picture data accumulated in said accumulating means, and

observation position/direction computing means for computing an observation position/direction which clears a background of the observation object using the surface information analyzed by said surface information analyzing means and the outline and non-outline data accumulated in said accumulating means.

12. A modeling apparatus comprising:

three-dimensional information input means for measuring, using a TV camera or ultrasonic sensor means, pieces of three-dimensional information comprising (I) three-dimensional geometric information composed of distance and shape data, and (ii) surface attribute information including at least one data item selected from a group consisting of color, pattern and reflectance,

environmental model processing means for managing the pieces of three-dimensional information and for constructing and storing an environmental model description,

picture composing means for composing an artificial picture viewed from a specific viewpoint within a space according to the environmental model constructed by said environmental model processing means,

analyzing means for comparing a predetermined plane region in the artificial picture composed by said picture composing means with a predetermined plane region viewed from said viewpoint and extracted from the pieces of three-dimensional information comprising three-dimensional geographic information and surface attribute information, and for providing a revise command for change of the environmental model when a difference is found between the plane region in said artificial picture and the plane region viewed from said specific viewpoint, and

observation control means responsive to the revise command of said analyzing means for changing a measurement position of said three-dimensional information input means,

said observation control means comprising

error region estimating means for estimating an error region for a position of an observation point measured by said three-dimensional information input means,

observing site determining means for predicting an error region for said position of the observation point from a new observing site and for determining said new observing site so that an overlap region between the error region estimated by said error region estimating means and the predicted error region will not be greater than the estimated error region, and

position re-estimating means for re-estimating the position of the observation point based on a result of measurement by said three-dimensional information input means from the observing side determined by said observing site determining means,

wherein said observation control means changes the environmental model by varying the measurement position of said three-dimensional information input means to allow a difference between the plane region in the specific viewpoint to be minimized in said analyzing means to provide an environmental model approximating the object within said space.

13. The modeling apparatus according to claim 12 wherein said observing site determining means is adapted to select, when a plurality of candidates are available for said new observing site, a site providing for a smallest distance of movement as the new observing site.

14. The modeling apparatus according to claim 12 wherein said observing site determining means is adapted to insure that a distance between said new observing site and said position of observation point is equal to a distance between said position and the modeling apparatus.

15. The modeling apparatus according to claim 14 wherein said observing site determining means is adapted to select, when said estimated error region is a polyhedron, a diagonal line from among diagonal lines of said polyhedron, which has a smallest angle with a straight line interconnecting said position of observation point and the modeling apparatus and to determine said new observing site on a line of intersection of a plane perpendicular to said selected diagonal line with a horizontal plane.

16. The modeling apparatus according to claim 14 wherein said observing site determining means is adapted to perform a principal component analysis of coordinate values of points within said estimated error region, select a principal axis, from among a first, second and third principal axes, which has a smallest angle with a straight line passing through said position of observation point and the modeling apparatus, and to determine said new observing site on a line of intersection of a plane perpendicular to said principal axis with a parallel plane including said position of observation point.

17. A modeling apparatus comprising:

three-dimensional information input means for inputting, using an observation device such as a camera, ultrasound or other visual sensor means, pieces of three-dimensional information comprising (I) three-dimensional geometric information composed of distance and shape data, and (ii) surface attribute information including at least one data item selected from a group consisting of color, pattern and reflectance,

environmental model processing means for managing the pieces of three-dimensional information and for constructing and storing an environmental model description,

picture composing means for composing an artificial picture viewed from a specific viewpoint within a space according to the environmental model description constructed by said environmental model processing means,

analyzing means comparing a predetermined plane region in the composed artificial picture with a predetermined plane region viewed from said specific viewpoint and extracted from said pieces of three-dimensional information comprising three-dimensional geometric information and surface attribute information, and for providing a revise command for change of the environmental model when a difference is found between the plane region in said specific viewpoint, and

observation control means responsive to the revise command from said analyzing means for changing a measurement point of said three-dimensional information input means,

said observation control means being adapted to control a rotary motion of said observation device so that the three-dimensional information relevant to an outline of an observation target space is input by a change of a direction of view from one visual point and,

computing a position and direction of said observation device from said three-dimensional information relevant to the outline of said observation target space, and controlling the rotary motion of said observation device to input three-dimensional information relevant to an interior of said target spaced, and

wherein said observation control means changes the environmental model by varying the measurement position of said three-dimensional information input means to allow a difference between the plane region in the artificial picture and the plane region viewed from the specific viewpoint to be minimized in said analyzing means while causing said environmental model processing means to provide an environmental model approximating the object within said space.