INFORMATION PROCESSING METHOD AND INFORMATION PROCESSING APPARATUS

• US 20120321173A1
• Filed: 02/22/2011
• Published: 12/20/2012
• Est. Priority Date: 02/25/2010
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Mitarai, Yusuke, Matsugu, Masakazu, Yoshii, Hiroto
• Granted Patent: US 9,429,418 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/03 by measuring coordinates of points
• CPC Class Codes: G01B 11/26 for measuring angles or tapers; for testing the alignment of axes
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A multi view-point image composed of a great number of images according to a shape of an object is generated or an information processing method used for generating a three-dimensional model or performing image processing of arbitrary view-point object recognition is provided, and based on a plurality of captured images obtained by imaging of the object from a plurality of view points by an imaging means, a relative position and orientation with respect to the object relative to the imaging means for each of the plurality of view points is calculated, and based on the calculated plurality of relative positions and orientations, a missing position and orientation of the imaging means in a direction in which imaging by the imaging means is missing is calculated, and an image used for displaying the calculated missing position and orientation on a display means is generated.

Claims

1. An information processing apparatus comprises:

a first calculation unit configured to calculate a relative position and orientation with respect to the object relative to the imaging unit for each of the plurality of view points based on a plurality of images captured by imaging an object from a plurality of view points by the imaging unit;

a lacking position and orientation calculation unit configured to calculate a lacking position and orientation of the imaging unit from which imaging by the imaging unit is acking based on the calculated plurality of relative positions and orientations; and

a lacking view-point image generation unit configured to generate an image used for display of the calculated lacking position and orientation on a display unit.

2. The information processing apparatus according to claim 1, the lacking position and orientation calculation unit calculates the relative position and orientation, with respect to the object whose imaging by the imaging unit is lacking relative to the imaging unit, as the missing position and orientation.

3. The information processing apparatus according to claim 1, further comprises a learning unit configured to allow a classifier that estimates orientations of the plurality of objects to learn the plurality of captured images, wherein the lacking position and orientation calculation unit calculates the lacking position and orientation based on the orientations of the plurality of the calculated relative positions and the orientation of the object estimated by the classifier.

4. The information processing apparatus according to claim 3, further comprises an image normalizing unit configured to normalize the plurality of captured images in a unified format, and the learning unit for classifier allows learning of the plurality of captured images which have been normalized.

5. The information processing apparatus according to claim 3, the lacking position and orientation calculation unit calculates the lacking position and orientation by comparing the orientations of the plurality of the calculated relative positions and orientations and a set position and orientation which is set based on the plurality of captured images learned by the classifier.

6. The information processing apparatus according to claim 3, the apparatus further comprises an estimated image generation unit configured to generate an estimated image in a case where imaging is performed from a set position and orientation which is set based on the plurality of captured images learned by the classifier, and

a projection image generation unit configured to generate a rendering image of a three-dimensional model of the object stored in advance in a case where observation is performed from the set position and orientation, and

wherein the lacking position and orientation is calculated by comparing the estimated image and the rendering image.

7. The information processing apparatus according to claim 5, with respect to the estimated image generation unit, the set position and orientation is an intermediate position and orientation of positions and orientations used for imaging the plurality of captured images learned by the classifier.

8. The information processing apparatus according to claim 6 the plurality of captured images leaned by the classifier are two captured images.

9. The information processing apparatus according to claim 1, the captured image includes a plurality of markers arranged on the object and in a periphery of the object, and three-dimensional positions of the plurality of markers in a real space are known.

10. The information processing apparatus according to claim 4, each of the plurality of markers has a color different from a color of the object and the plurality of markers which are different from the marker.

11. The information processing apparatus according to claim 4, the relative position and orientation calculation unit calculates the relative position and orientation with respect to the object relative to the imaging unit for each of the plurality of view points by detecting positions of the plurality of markers in the captured image.

12. The information processing apparatus according to claim 11, positions of the detected plurality of markers are four or more.

13. The information processing apparatus according to claim 1 the relative position and orientation calculation unit calculates the relative position and orientation with respect to the object relative to the imaging unit for each of the plurality of view points by comparing characteristics of the three-dimensional model of the object stored in advance and characteristics of the object in the captured image.

14. The information processing apparatus according to claim 1, the lacking position and orientation calculation unit comprises:

an approximately uniform direction calculation unit configured to calculate a plurality of approximately uniform directions in which imaging of the object can be approximately uniformly performed having the object at a center;

a similar direction determination unit configured to determine the approximately uniform direction which is similar to a direction of imaging based on the plurality of the calculated relative positions and orientations; and

a dissimilar direction calculation unit configured to calculate a direction whose imaging by the imaging unit is lacking based on the approximately uniform direction which is determined as not similar.

15. The information processing apparatus according to claim 1, the missing position and orientation calculation unit calculates, if the calculated relative position and orientation from a position in a vicinity of a view point which can be used in imaging a complex structure of a three-dimensional model of the object stored in advance is small in number, a relative position and orientation of the view point as a lacking position and orientation.

16. The information processing apparatus according to claim 1, the apparatus further comprises a three-dimensional model generation unit configured to generate the three-dimensional model based on the plurality of captured images.

17. The information processing apparatus according to claim 1, the apparatus further comprises an imaging moving unit used for moving the imaging unit.

18. An information processing method comprises:

calculating a relative position and orientation with respect to the object relative to the imaging unit for each of the plurality of view points based on a plurality of images captured by imaging an object from a plurality of view points by an imaging unit;

calculating a missing position and orientation of the imaging unit from which imaging by the imaging unit is missing based on the calculated plurality of relative positions and orientations; and

generating an image used for display of the calculated missing position and orientation on a display unit.

20. A storage medium storing a computer-executable program configured to allow a computer to execute the information processing method according to claim 18.

19. (canceled)