INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND NON-TRANSITORY COMPUTER READABLE STORAGE MEDIUM

• US 20170148167A1
• Filed: 11/17/2016
• Published: 05/25/2017
• Est. Priority Date: 11/20/2015
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Aratani, Shinichi
• Granted Patent: US 10,839,544 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Position/orientation information that indicates the position and orientation of an image sensing device is estimated on the basis of three-dimensional position information of a registered feature and a feature in a frame image captured by the image sensing device. Three-dimensional position information of an unregistered feature is generated on the basis of the estimated position/orientation information, and the generated three-dimensional position information is registered. The generation and registration of the three-dimensional position information of the unregistered feature are controlled in accordance with execution of the estimation.

Claims

1. An information processing apparatus, comprising:

an estimation unit configured to estimate position/orientation information on the basis of three-dimensional position information of a registered feature and a feature in a frame image captured by an image sensing device, the position/orientation information indicating a position and an orientation of the image sensing device; and

a generation unit configured to generate three-dimensional position information of an unregistered feature on the basis of the position/orientation information estimated by the estimation unit, and register the generated three-dimensional position information, wherein

the generation unit controls the generation and the registration of the three-dimensional position information of the unregistered feature in accordance with execution of the estimation.

2. The information processing apparatus according to claim 1, wherein

the generation unit executes the generation and the registration of the three-dimensional position information of the unregistered feature until a prescribed period elapses since a start of processing for estimating the position/orientation information of the image sensing device by the estimation unit, and stops the generation and the registration of the three-dimensional position information of the unregistered feature when the prescribed period has elapsed.

3. The information processing apparatus according to claim 2, wherein

after the generation and the registration of the three-dimensional position information of the unregistered feature have been started, when a continuous period in which the generation and the registration of the three-dimensional position information of the unregistered feature are not executed has exceeded a threshold, the generation unit stops the generation and the registration of the three-dimensional position information of the unregistered feature.

4. The information processing apparatus according to claim 1, wherein

after the generation and the registration of the three-dimensional position information of the unregistered feature have been started, even when a continuous period in which the generation and the registration of the three-dimensional position information of the unregistered feature are not executed has exceeded a threshold, the generation unit does not stop the generation and the registration of the three-dimensional position information of the unregistered feature on a condition that a change amount of the position and/or the orientation indicated by the position/orientation information estimated by the estimation unit does not exceed a threshold.

5. The information processing apparatus according to claim 2, further comprising:

a unit configured to acquire a distance from an initial position of the image sensing device; and

a unit configured to update the prescribed period in accordance with the distance.

6. The information processing apparatus according to claim 2, further comprising

a unit configured to update the prescribed period in accordance with the number of frame images for which the position or the orientation indicated by the position/orientation information estimated by the estimation unit exhibits a change that is equal to or larger than a threshold.

7. The information processing apparatus according to claim 2, further comprising

a unit configured to, when a user inputs a movement range of the image sensing device, set a period corresponding to the movement range as the prescribed period.

8. The information processing apparatus according to claim 2, further comprising

a unit configured to update the prescribed period in accordance with a change amount of the position and/or the orientation indicated by the position/orientation information estimated by the estimation unit.

9. The information processing apparatus according to claim 1, wherein

the generation unit starts the generation and the registration of the three-dimensional position information of the unregistered feature in accordance with capturing of an image of an index performed by the image sensing device.

10. The information processing apparatus according to claim 1, wherein

the generation unit uses, as a keyframe, a frame image for which the position indicated by the position/orientation information estimated by the estimation unit exhibits a change that is equal to or larger than a threshold, and generates three-dimensional position information of a feature on the basis of the keyframe and position/orientation information that has been estimated on the basis of the keyframe and indicates a position and an orientation of the image sensing device.

11. The information processing apparatus according to claim 1, wherein

the generation unit uses, as a keyframe, a frame image for which the orientation indicated by the position/orientation information estimated by the estimation unit exhibits a change that is equal to or larger than a threshold, and generates three-dimensional position information of a feature on the basis of the keyframe and position/orientation information that has been estimated on the basis of the keyframe and indicates a position and an orientation of the image sensing device.

12. The information processing apparatus according to claim 1, further comprising

a unit configured to notify a user of a progress of the generation and the registration of the three-dimensional position information of the unregistered feature.

13. An information processing apparatus, comprising:

an acquisition unit configured to acquire an image captured by an image sensing device;

an estimation unit configured to estimate position/orientation information on the basis of a feature whose three-dimensional position information has been registered and a feature included in the captured image, the position/orientation information indicating a position and an orientation of the image sensing device;

a registration unit configured to register three-dimensional position information of an unregistered feature included in the captured image on the basis of the position/orientation information; and

a notification unit configured to notify a user of a progress of the registration of the three-dimensional position information of the unregistered feature executed by the registration unit.

14. An information processing method executed by an information processing apparatus, the information processing method comprising:

estimating position/orientation information on the basis of three-dimensional position information of a registered feature and a feature in a frame image captured by an image sensing device, the position/orientation information indicating a position and an orientation of the image sensing device;

generating three-dimensional position information of an unregistered feature on the basis of the estimated position/orientation information, and registering the generated three-dimensional position information; and

controlling the generation and the registration of the three-dimensional position information of the unregistered feature in accordance with execution of the estimation.

15. An information processing method, comprising:

acquiring an image captured by an image sensing device;

estimating position/orientation information on the basis of a feature whose three-dimensional position information has been registered and a feature included in the captured image, the position/orientation information indicating a position and an orientation of the image sensing device;

registering three-dimensional position information of an unregistered feature included in the captured image on the basis of the position/orientation information; and

notifying a user of a progress of the registration of the three-dimensional position information of the unregistered feature.

16. A non-transitory computer readable storage medium storing a computer program for causing a computer to function as:

an estimation unit configured to estimate position/orientation information on the basis of three-dimensional position information of a registered feature and a feature in a frame image captured by an image sensing device, the position/orientation information indicating a position and an orientation of the image sensing device; and

a generation unit configured to generate three-dimensional position information of an unregistered feature on the basis of the position/orientation information estimated by the estimation unit, and register the generated three-dimensional position information, wherein

the generation unit controls the generation and the registration of the three-dimensional position information of the unregistered feature in accordance with execution of the estimation.

17. A non-transitory computer readable storage medium storing a computer program for causing a computer to function as:

an acquisition unit configured to acquire an image captured by an image sensing device;

an estimation unit configured to estimate position/orientation information on the basis of a feature whose three-dimensional position information has been registered and a feature included in the captured image, the position/orientation information indicating a position and an orientation of the image sensing device;

a registration unit configured to register three-dimensional position information of an unregistered feature included in the captured image on the basis of the position/orientation information; and

a notification unit configured to notify a user of a progress of the registration of the three-dimensional position information of the unregistered feature executed by the registration unit.