INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD AND PROGRAM

• US 20110044504A1
• Filed: 08/12/2010
• Published: 02/24/2011
• Est. Priority Date: 08/21/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Fukuchi, Masaki, OI, Kenichiro
• Granted Patent: US 8,861,785 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An information processing device, including: a three-dimensional information generating section for obtaining position and attitude of a moving camera or three-dimensional positions of feature points by successively receiving captured images from different viewpoints, and updating status data using observation information which includes tracking information of the feature points, the status data including three-dimensional positions of the feature points within the images and position and attitude information of the camera; and a submap generating section for generating submaps by dividing an area for which the three-dimensional position is to be calculated. The three-dimensional information generating section obtains position and attitude of the camera or three-dimensional positions of the feature points by generating status data corresponding to the submaps not including information about feature points outside of a submap area for each of the generated submaps and updating the generated status data corresponding to the submaps.

Claims

1. An information processing device, comprising:

a three-dimensional information generating section for obtaining position and attitude of a camera or three-dimensional positions of feature points by successively receiving images from two or more different viewpoints captured with the camera which moves in a space, and updating status data using observation information which includes tracking information of the feature points, the status data including three-dimensional positions of the feature points within the images and position and attitude information of the camera; and

a submap generating section for generating submaps by dividing an area for which the three-dimensional position is to be calculated,

wherein the three-dimensional information generating section obtains the position and attitude of the camera or the three-dimensional positions of the feature points by generating status data corresponding to the submaps which do not include information about feature points outside of a submap area for each of the submaps generated by the submap generating section and updating the generated status data corresponding to the submaps.

2. The information processing device according to claim 1, wherein the three-dimensional information generating section generates, as the status data corresponding to the submaps, multidimensional normal distribution data which includes a mean vector representing the position and attitude of the camera and positions of each of the feature points in the submaps, and a variance-covariance matrix in correspondence with each of the submaps, and updates the multidimensional normal distribution data corresponding to the submaps using the observation information.

3. The information processing device according to claim 2, wherein the variance-covariance matrix includes variance of inherent status values, such as position and attitude of a camera, and position information about each of the feature points, and covariance which corresponds to combinations of correlation information of different status values from among the status values.

4. The information processing device according to any one of claims 1 to 3, wherein the three-dimensional information generating section updates the status data corresponding to the submaps using a Kalman filter or an Extended Kalman filter (EKF).

5. The information processing device according to claim 1, wherein, if the submaps are to be processed sequentially from a preceding submap to a current submap, the three-dimensional information generating section generates status data corresponding to the current submap directly using camera information included in status data corresponding to the preceding submap.

6. The information processing device according to any one of claims 1 to 5, wherein, when the number of feature points included in a submap reaches a predetermined upper limit number N or greater, the submap generating section divides the submap into two with an overlapping area formed therebetween, the two submaps having at least one feature point in common.

7. The information processing device according to claim 6, wherein the three-dimensional information generating section generates status data corresponding to each of the submaps generated by dividing the submap, and generates status data corresponding to each of the two divided submaps by deleting information about feature points which are not included in each of the divided submaps from among the status data corresponding to the submaps which are not yet divided.

8. The information processing device according to any one of claims 1 to 5, wherein the submap generating section divides a two-dimensional plane generated through projective transformation of an area for which a three-dimensional position is to be calculated into two or more submaps of a predetermined size which have overlapping areas therebetween.

9. The information processing device according to claim 1, wherein the three-dimensional information generating section corrects positions of the feature points through bundle adjustment.

10. The information processing device according to any one of claims 1 to 9, wherein the three-dimensional information generating section executes simultaneous localization and mapping (SLAM) in which localization of a camera and mapping of a neighborhood map (i.e., an environmental map) are performed at the same time.

11. An information processing method executed in an information processing device, the method comprising the steps of:

generating, by a submap generating section, submaps by dividing an area for which the three-dimensional position is to be calculated; and

generating three-dimensional information by a three-dimensional information generating section which obtains position and attitude of a camera or three-dimensional positions of feature points by successively receiving images from two or more different viewpoints captured with the camera which moves in a space, and updating status data using observation information which includes tracking information of the feature points, the status data including three-dimensional positions of the feature points within the images and position and attitude information of the camera,

wherein the step of generating three-dimensional information is to obtain the position and attitude of the camera or the three-dimensional positions of the feature points by generating status data corresponding to the submaps which do not include information about feature points outside of a submap area for each of the submaps generated by the submap generating section and updating the generated status data corresponding to the submaps.

12. A program which causes an information processing device to process information, the program comprising the steps of:

causing a submap generating section to generate submaps by dividing an area for which the three-dimensional position is to be calculated; and

causing a three-dimensional information generating section to generate three-dimensional information by obtaining position and attitude of a camera or three-dimensional positions of feature points by successively receiving images from two or more different viewpoints captured with the camera which moves in a space, and updating status data using observation information which includes tracking information of the feature points, the status data including three-dimensional positions of the feature points within the images and position and attitude information of the camera,

wherein the step of generating three-dimensional information is to obtain the position and attitude of the camera or the three-dimensional positions of the feature points by generating status data corresponding to the submaps which do not include information about feature points outside of a submap area for each of the submaps generated by the submap generating section and updating the generated status data corresponding to the submaps.