Three-dimensional information processing apparatus and method

• US 6,445,814 B2
• Filed: 06/30/1997
• Issued: 09/03/2002
• Est. Priority Date: 07/01/1996
• Status: Expired due to Term

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Matsugu, Masakazu, Mori, Katsuhiko, Iijima, Katsumi, Yano, Kotaro, Katayama, Tatsushi, Okauchi, Shigeki, Sekine, Masayoshi, Ishikawa, Motohiro, Kurahashi, Sunao
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US08/885,823
• Publication Number:
  US 20010014171A1
• Time in Patent Office: 1,891 Days
• Field of Search: 382/153, 382/154, 382/190, 382/195, 382/203, 382/201, 348/42, 348/46, 348/47, 348/50, 348/51, 359/462, 352/86, 352/2, 356/2, 356/12, 356/375, 356/376
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A three-dimensional information processing apparatus for obtaining three-dimensional information from an object having a three-dimensional shape, and performing predetermined information processing, comprises: a camera for sensing images of the object from a plurality of coordinate positions using an image sensing system having one or a plurality of optical systems. A plurality of depth information are extracted from image sensing related information sensed by the camera at the plurality of coordinate positions, and the plurality of extracted depth information are converted and unified into depth information expressed by a unified coordinate system.

Claims

1. A three-dimensional information processing apparatus for obtaining three-dimensional information from an object having a three-dimensional shape, and performing predetermined information processing, comprising:

image sensing means for sensing images of the object from a plurality of coordinate positions using an image sensing system having one or a plurality of optical systems;

information extraction means for extracting a plurality of depth information from the images sensed by said image sensing means at the plurality of coordinate positions;

conversion/unification means for converting and unifying the plurality of depth information extracted by said depth information extraction means into depth information expressed by a unified coordinate system, and generating three-dimensional information on the basis of the depth information expressed by the unified coordinate system;

reliability determination means for determining reliability of the converted/unified three-dimensional information based on the images sensed by said image sensing means; and

correcting means for correcting the three-dimensional information generated by said conversion and unification means by using a reliability determined by said reliability determination means.

2. The apparatus according to claim 1, wherein said conversion/unification means obtains image information of the object from said image sensing means, detects a displacement between coordinate systems of the plurality of depth information on the basis of the obtained image information, and converts and unifies the plurality of depth information onto the unified coordinate system.

3. The apparatus according to claim 1, wherein the unified coordinate system has five different projection planes.

4. The apparatus according to claim 2, wherein the image information includes luminance information of the object, and said conversion/unification means detects the displacement between the coordinate systems on the basis of the luminance information.

5. A three-dimensional information processing method for obtaining three-dimensional information from an object having a three-dimensional shape, and performing pre-determined information processing, comprising:

the first step of sensing images of the object from a plurality of coordinate positions using an image sensing system having one or a plurality of optical systems;

the second step of extracting a plurality of depth information from the images sensed at the plurality of coordinate positions in the first step;

the third step of converting and unifying the plurality of depth information extracted by said depth information extraction means into depth information expressed by a unified coordinate system, and generating three-dimensional information on the basis of the depth information expressed by the unified coordinate system;

the fourth step of determining reliability of the converted and unified three-dimensional information based on the images sensed by said image sensing means; and

the fifth step of correcting the three-dimensional information generated in said conversion and unification steps by using a reliability determined in said fourth step.

6. The method according to claim 5, wherein the third step has the step of obtaining image information of the object obtained in the first step, detecting a displacement between coordinate systems of the plurality of depth information on the basis of the obtained image information, and converting and unifying the plurality of depth information onto the unified coordinate system.

7. The method according to claim 5, wherein the unified coordinate system has five different protection planes.

8. The method according to claim 6, wherein the image information includes luminance information of the object, and the displacement between the coordinate systems is detected on the basis of the luminance information.

9. A three-dimensional information processing apparatus for obtaining three-dimensional information from an object having a three-dimensional shape, and performing predetermined information processing, comprising:

image sensing means for sensing images of the object using an image sensing system having one or a plurality of optical systems;

three-dimensional shape extraction means for extracting three-dimensional shape information of the object from the images sensed by said image sensing means;

reliability determination means for determining reliability of the three-dimensional shape information extracted by said three-dimensional shape extraction means based on the images sensed by said image sensing means; and

correcting means for correcting the three-dimensional shape information of the object by using the reliability determined by said reliability determination means.

10. The apparatus according to claim 9, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an angle of the object with respect to an image sensing plane into account.

11. The apparatus according to claim 9, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a distance between said image sensing means and the object into account.

12. The apparatus according to claim 9, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an angle a pad that places the object thereon makes with an image sensing plane of said image sensing means into account.

13. The apparatus according to claim 9, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an area ratio of a pad that places the object thereon to an image sensing region into account.

14. The apparatus according to claim 9, wherein said reliability information means determines the reliability of the three-dimensional shape information by taking a position of a pad that places the object thereon into account.

15. The apparatus according to claim 9, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking reflected light information reflected by the object into account.

16. The apparatus according to claim 9, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a degree of correspondence of pixels between a plurality of image sensing related data sensed by said image sensing means into account.

17. A three-dimensional information processing apparatus for obtaining three-dimensional information from an object having a three-dimensional information from an object having a three-dimensional shape, and performing predetermined information processing comprising:

image sensing means for sensing images of the object using an image sensing system having one or a plurality of optical systems;

three-dimensional shape extraction means for extracting three-dimensional shape information of the object from the images sensed by said image sensing means;

reliability determination means for determining reliability of the three-dimensional shape information extracted by said three-dimensional shape extraction means based on the images sensed by said image sensing means;

informing means for informing a reliability determination result of said reliability determination means; and

correcting means for correcting the three-dimensional shape information of the object by using the reliability determination result.

18. The apparatus according to claim 17, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an angle of the object with respect to an image sensing plane into account.

19. The apparatus according to claim 17, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a distance between said image sensing means and the object into account.

20. The apparatus according to claim 17, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an angle a pad that places the object thereon makes with an image sensing plane of said image sensing means into account.

21. The apparatus according to claim 17, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an area ratio of a pad that places the object thereon to an image sensing region into account.

22. The apparatus according to claim 17, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a position of a pad that places the object thereon into account.

23. The apparatus according to claim 17, wherein said reliability determination means determines shape information by taking reflected light information reflected by the object into account.

24. The apparatus according to claim 17, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a degree of correspondence of pixels between a plurality of image sensing related data sensed by said image sensing means into account.

25. A three-dimensional information processing apparatus for obtaining three-dimensional information from an object having a three-dimensional shape, and performing predetermined information processing, comprising:

image sensing means for sensing images of the object using an image sensing system having one or a plurality of optical systems;

three-dimensional shape extraction means for extracting three-dimensional shape information of the object from the images sensed by said image sensing means;

reliability determination means for determining reliability of the three-dimensional shape information extracted by said three-dimensional shape extraction means based on the images sensed by said image sensing means; and

display means for correcting the three-dimensional shape information in accordance with a reliability determination result of said reliability determination means, and displaying the corrected three-dimensional shape information.

26. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an angle of the object with respect to an image sensing plane into account.

27. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a distance between said image sensing means and the object into account.

28. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an angle a pad that places the object thereon makes with an image sensing plane of said image sensing means into account.

29. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking an area ratio of a pad that places the object thereon to an image sensing region into account.

30. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a position of a pad that places the object thereon into account.

31. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking reflected light information reflected by the object into account.

32. The apparatus according to claim 25, wherein said reliability determination means determines the reliability of the three-dimensional shape information by taking a degree of correspondence of pixels between a plurality of image sensing related data sensed by said image sensing means into account.