Image processing apparatus and method for obtaining position and orientation of imaging apparatus

• US 9,767,563 B2
• Filed: 08/11/2015
• Issued: 09/19/2017
• Est. Priority Date: 10/30/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Takemoto, Kazuki, Satoh, Kiyohide
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US14/823,263
• Publication Number:
  US 20150348272A1
• Time in Patent Office: 770 Days
• Field of Search: 382103, 382118, 382128, 382190, 382225, 382154, 348208, 348231, 348333, 348565, 348700
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 13/207 using a single 2D image sensor

Abstract

An image processing apparatus obtains location information of each image feature in a captured image based on image coordinates of the image feature in the captured image. The image processing apparatus selects location information usable to calculate a position and an orientation of the imaging apparatus among the obtained location information. The image processing apparatus obtains the position and the orientation of the imaging apparatus based on the selected location information and an image feature corresponding to the selected location information among the image features included in the captured image.

Claims

1. An image processing apparatus comprising:

a processor and memory;

an acquisition unit configured to acquire an image from an imaging apparatus that captures an image of a physical space;

a detection unit configured to detect at least one image feature from the acquired image;

a generation unit configured to generate a combined image by combining a predetermined object which indicates the detected image feature with the captured image;

a display unit configured to display the combined image to a user; and

an obtaining unit configured to obtain a position and an orientation of the imaging apparatus based on a three-dimensional position of the detected image feature and a user'"'"'s instruction indicating whether the detected image feature is a disabled image feature or an enabled image feature,

wherein the display unit displays a feedback image to the user on the basis of the position and the orientation of the imaging apparatus, and

wherein at least one of the units is implemented upon execution of instructions stored in the memory by the processor.

2. The image processing apparatus according to claim 1, wherein the detected image feature includes at least one of a feature detected as a point, a feature detected as a line, and a feature detected as a polygonal shape on the captured image.

3. The image processing apparatus according to claim 1, wherein the captured image is an image of each frame constituting a moving image of the physical space.

10. The image processing apparatus according to claim 1, further comprising a derivation unit configured to derive location information of the detected image feature based on an image coordinate of the detected image feature,

wherein the obtain unit obtains the position and the orientation of the imaging apparatus based on the location information of the image feature except for the disable image feature.

4. A method for processing an image comprising:

acquiring an image from an imaging apparatus that captures an image of a physical space;

detecting at least one image feature from the acquired image;

generating a combined image by combining a predetermined object which indicates the detected image feature with the captured image;

displaying the combined image to a user;

obtaining a position and an orientation of the imaging apparatus based on a three-dimensional position of the detected image feature and a user'"'"'s instruction indicating whether the detected image feature is a disabled image feature or an enabled image feature; and

displaying a feedback image to the user on the basis of the position and the orientation of the imaging apparatus.

6. The method for processing an image according to claim 4, wherein the detected image feature includes at least one of a feature detected as a point, a feature detected as a line, and a feature detected as a polygonal shape on the captured image.

7. The method for processing an image according to claim 4, wherein the captured image is an image of each frame constituting a moving image of the physical space.

5. A non-transitory computer readable medium containing computer-executable instructions utilized in an image processing apparatus for processing an image, the medium comprising:

computer-executable instructions for acquiring an image from an imaging apparatus that captures an image of a physical space;

computer-executable instructions for detecting at least one image feature from the acquired image;

computer-executable instructions for generating a combined image by combining a predetermined object which indicates the detected image feature with the captured image;

computer-executable instructions for displaying the combined image to a user;

computer-executable instructions for obtaining a position and an orientation of the imaging apparatus based on three-dimensional position of the detected image feature and the user'"'"'s instruction indicating whether the detected image feature is a disabled image feature or an enabled image feature; and

computer-executable instructions for displaying a feedback image to the user on the basis of the position and the orientation of the imaging apparatus.

8. The medium for processing an image according to claim 5, wherein the detected image feature includes at least one of a feature detected as a point, a feature detected as a line, and a feature detected as a polygonal shape on the captured image.

9. The medium for processing an image according to claim 5, wherein the captured image is an image of each frame constituting a moving image of the physical space.