Three-dimensional model data forming apparatus

• US 6,031,941 A
• Filed: 12/20/1996
• Issued: 02/29/2000
• Est. Priority Date: 12/27/1995
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Iijima, Katsumi, Yano, Kotaro
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US08/770,317
• Time in Patent Office: 1,166 Days
• Field of Search: 382/276, 382/285, 382/295, 382/154, 382/312, 382/313, 382/318
• US Class Current: 382/276
• CPC Class Codes: G06T 7/571 from focus

Abstract

In a three-dimensional shape extraction apparatus, a distance distribution extraction unit (54) obtains the distance distribution of an object on the basis of a plurality of images which are sensed by a camera at different image sensing positions, so that the images of the object partially overlap each other, and the position information of the respective image sensing positions of the camera. A three-dimensional model forming unit (55) forms a three-dimensional model of the object by sequentially combining the sequentially calculated distance distributions and the sensed images using the position information. A 3D display image generation unit (56) generates and displays a two-dimensional image from the three-dimensional model.

Claims

1. A three-dimensional model data forming apparatus for forming three-dimensional model data of an object sensed by image sensing means, comprising:

first detection means for detecting an image sensing position of said image sensing means;

second detection means for detecting a lens position of said image sensing means;

calculation means for calculating a parameter on the basis of outputs from said first and second detection means;

distance distribution extraction means for extracting a distance distribution of the object on the basis of a plurality of images which are sensed by said image sensing means at a plurality of image sensing positions so that the respective images of the object partially overlap each other, and the parameter calculated by said calculation means;

three-dimensional model data forming means for forming the three-dimensional model data of the object on the basis of the distance distribution extracted by said distance distribution extraction means, the plurality of images, and the parameter;

two-dimensional image generation means for generating a two-dimensional image on the basis of the three-dimensional model data output from said three-dimensional model data forming means; and

monitoring means for displaying the two-dimensional image and for designating the displayed two-dimensional image corresponding to a part shape of the object, so that it is possible to visually monitor a forming process for forming three-dimensional model data corresponding to a desired portion of the three-dimensional shape of the object.

2. The apparatus according to claim 1, wherein said three-dimensional model data forming means describes the three-dimensional model data using parameters of a pixel value and a radial component of a three-dimensional polar coordinate system with respect to an address designated by two angle parameters of the three-dimensional polar coordinate system.

3. The apparatus according to claim 2, wherein said two-dimensional image generation means describes the three-dimensional model data at an address designated by a two-dimensional model at an address designated by a two-dimensional orthogonal coordinate system by performing a viewing transformation on the basis of directions of coordinate axes of a coordinate system, a view point position, and a visual axis direction of a display image, which are set in advance, and the parameters of the pixel value and the radial component of the three-dimensional polar coordinate system.

4. The apparatus according to claim 1, wherein said calculation means calculates view point position data, view point moving amount data, and optical axis rotation amount data in accordance with the outputs from said first and second detection means.

5. The apparatus according to claim 4, wherein said calculation means calculates the view point moving amount data and the optical axis rotation amount data in accordance with the output from said first detection means and predetermined view point conversion data.

6. The apparatus according to claim 4, wherein said calculation means calculates the view point position data on the basis of the output from said second detection means and predetermined view point position correction data.

7. The apparatus according to claim 1, wherein said distance distribution extraction means extracts a predetermined object from each of the plurality of images, and extracts the distance distribution with respect to the extracted object.

8. The apparatus according to claim 1, wherein said monitoring means is also for displaying the image sensed by said image sensing means together with the two-dimensional image.

9. The apparatus according to claim 1, wherein said distance distribution extraction means performs matching processing of corresponding points between two image data.

10. The apparatus according to claim 9, wherein said distance distribution extraction means calculates a distance to the object on the basis of a pair of corresponding points obtained by the matching processing and the parameter.

11. The apparatus according to claim 1, wherein said distance distribution extraction means extracts a predetermined object from each of the plurality of images, and performs matching processing of corresponding points between the two extracted objects.

12. The apparatus according to claim 11, wherein said distance distribution extraction means calculates a distance to the object on the basis of a pair of corresponding points obtained by the matching processing and the parameter.

13. The apparatus according to claim 1, wherein said three-dimensional model data forming means converts the distance distribution into polar coordinates on the basis of the parameter.

14. The apparatus according to claim 13, further comprising storage means for storing the distance distribution using a coordinate value of the polar coordinates as an address.

15. A method of forming three-dimensional model data of an object sensed by image sensing means, comprising the steps of:

first detection step of detecting an image sensing position of said image sensing means;

second detection step of detecting a lens position of said image sensing means;

calculation step of calculating a parameter on the basis of outputs from said first and second detection steps;

extraction step of extracting a distance distribution of the object on the basis of a plurality of images which are sensed by said images sensing means at a plurality of image sensing positions so that the respective images of the object partially overlap each other, and the parameter calculated by said calculation step;

forming step of forming three-dimensional model data of the object on the basis of the distance distribution extracted by said extraction step, the plurality of images, and the parameter;

generation step of generating a two-dimensional image of the basis of the three-dimensional model data formed in said forming step; and

monitoring step of displaying the two-dimensional image and of designating the displayed two-dimensional image corresponding to a part shape of the object, so that it is possible to visually monitor a forming process for forming three-dimensional model data corresponding to a desired portion of the three-dimensional shape of the object.

16. The method according to claim 15, wherein said forming step describes the three-dimensional model data using parameters of a pixel value and a radial component of a three-dimensional polar coordinate system with respect to an address designated by two angle parameters of the three-dimensional polar coordinate system.

17. The method according to claim 16, wherein said generation step describes the three-dimensional model data at an address designated by a two-dimensional orthogonal coordinate system by performing a viewing transformation on the basis of directions of coordinate axes of a coordinate system, a view point position, and a visual axis direction of a display image, which are set in advance, and the parameters of the pixel value and the radial component of the three-dimensional polar coordinate system.

18. The method according to claim 15, wherein said calculation step calculates view point position data, view point moving amount data, and optical axis rotation amount data in accordance with the detected results in said first and second detection steps.

19. The method according to claim 18, wherein said calculation step calculates the view point moving amount data and the optical axis rotation amount data in accordance with the detected results in said first detection steps and predetermined view point conversion data.

20. The method according to claim 17, wherein said calculation step calculates the view point position data on the basis of the detected result in said second detection step and predetermined view point position correction data.

21. The method according to claim 15, wherein said extraction step extracts a predetermined object from each of the plurality of images, and extracts the distance distribution with respect to the extracted object.

22. The method according to claim 15, wherein said monitoring step also displays the image sensed by said image sensing means together with the two-dimensional image.

23. The method according to claim 15, wherein said extraction step performs matching processing of corresponding points between two image data.

24. The method according to claim 23, wherein said extraction step calculates a distance to the object on the basis of a pair of corresponding points obtained by the matching processing and the parameter.

25. The method according to claim 15, wherein said extraction means extracts a predetermined object from each of the plurality of images, and performs matching processing of corresponding points between the two extracted objects.

26. The method according to claim 25, wherein said extraction step calculates a distance to the object on the basis of a pair of corresponding points obtained by the matching processing and the parameter.

27. The method according to claim 15, wherein said forming step converts the distance distribution into polar coordinates on the basis of the parameter.

28. The method according to claim 27, further comprising storage step of storing the distance distribution using a coordinate value of the polar coordinates as an address.

29. A three-dimensional model data processing apparatus for processing three-dimensional model data of an object sensed by image sensing means, comprising;

first input means for inputting an image sensing position of said image sensing means;

second input means for inputting operating state information of said image sensing means;

calculation means for calculating a parameter on the basis of outputs from said first and second detection means;

distance distribution detecting means for detecting a distance distribution of the object on the basis of a plurality of images which are sensed by said image sensing means at a plurality of image sensing positions so that the respective images of the object partially overlap each other, the parameter calculated by said calculation means;

three-dimensional model data forming means for forming a three-dimensional model data of the object on the basis of the distance distribution detected by said distance distribution detecting means, the plurality of images, and the parameter;

two-dimensional image generation means for generating a two-dimensional image on the basis of the three-dimensional model data output from said three-dimensional model data forming means; and

monitoring means for displaying the two-dimensional image and for designating the displayed two-dimensional image corresponding to a part shape of the object, so that it is possible to visually monitor a processing for forming three-dimensional model data corresponding to a desired portion of the three-dimensional shape of the object.

30. The apparatus according to claim 29, wherein said three-dimensional model data forming means describes the three-dimensional model data using parameters of a pixel value and a radial component of a three-dimensional polar coordinate system with respect to an address designated by two angle parameters of the three-dimensional polar coordinate system.

31. The apparatus according to claim 30, wherein said two-dimensional image generation means describes the three-dimensional model data at an address designated by a two-dimensional orthogonal coordinate system by performing a viewing transformation on the basis of directions of coordinate axes of a coordinate system, a view point position, and a visual axis direction of a display image, which are set in advance, and the parameters of the pixel value and the radial component of the three-dimensional polar coordinate system.

32. The apparatus according to claim 29, wherein said calculation means calculates view point position data, view point moving amount data, and optical axis rotation amount data in accordance with the outputs from said first and second input means.

33. The apparatus according to claim 32, wherein said calculation means calculates the view point moving amount data and the optical axis rotation amount data in accordance with the output from said first input means and predetermined view point conversion data.

34. The apparatus according to claim 32, wherein said calculation means calculates the view point position data on the basis of the output from said second input means and predetermined view point position correction data.

35. The apparatus according to claim 29, wherein said monitoring means is also for displaying the image sensed by said image sensing means together with the two-dimensional image.

36. The apparatus according to claim 29, wherein said distance distribution detecting means performs matching processing of corresponding points or images between two image data and calculates a distance to the object on the basis of a pair of corresponding points obtained by the matching processing and the parameter.

37. The apparatus according to claim 29, wherein said three-dimensional model data processing means converts the distance distribution into polar coordinates on the basis of the parameter.