POSITION AND ORIENTATION MEASURING APPARATUS AND POSITION AND ORIENTATION MEASURING METHOD, MIXED-REALITY SYSTEM, AND COMPUTER PROGRAM

• US 20080031490A1
• Filed: 07/30/2007
• Published: 02/07/2008
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kobayashi, Kazuhiko
• Granted Patent: US 7,965,904 B2
• Time in Patent Office: Days
• US Class Current: 382/101
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A position and orientation measuring apparatus comprising, a storage unit adapted to store character region specifying information and position information in association with a character region place in a physical space, a detection unit adapted to detect the character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit, and an estimation unit adapted to estimate a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the character region, detected by the detection unit, in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected region.

Claims

1. A position and orientation measuring apparatus comprising:

a storage unit adapted to store character region specifying information and position information in association with a character region placed in a physical space;

a detection unit adapted to detect the character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in said storage unit; and

an estimation unit adapted to estimate a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the character region, detected by said detection unit, in the first captured image data, and the position information which is stored in said storage unit and corresponds to the detected region.

2. The apparatus according to claim 1, wherein said detection unit comprises:

a character region detection unit adapted to detect a character region from the captured image data; and

an image position information detection unit adapted to detect the image position information of an index corresponding to the character region from the detected character region.

3. The apparatus according to claim 2, wherein said character region detection unit comprises:

a contour extraction unit adapted to extract contours in the captured image data;

a setting unit adapted to set a character region candidate using the contours;

a normalization processing unit adapted to normalize a size of the character region candidate; and

a character recognition processing unit adapted to apply character recognition processing to the normalized character region candidate.

4. The apparatus according to claim 1, wherein when the character region is included in a poster placed in the physical space,

said storage unit stores, as information associated with a character, image information around the character region in the poster, and

said detection unit detects the character region further using the image information around the character region.

5. The apparatus according to claim 1, wherein said detection unit comprises:

a save unit adapted to save position information of a character region detected from captured image data captured by the image sensing apparatus at an immediately preceding timing; and

a region estimation unit adapted to estimate an image region corresponding to the character region in the first captured image data based on the saved position information.

6. The apparatus according to claim 1, further comprising a registration unit adapted to calculate position information associated with a character region, which is not registered in said storage unit, using a different parallax image, and to register the calculated position information in said storage unit together with image data of the non-registered character region.

7. A mixed reality system comprising:

an image sensing apparatus;

a position and orientation measuring apparatus according to claim 1;

a virtual object composition unit adapted to generate composite image data by combining a virtual object and captured image data obtained by capturing an image of a physical space by said image sensing apparatus, based on the captured image data and an orientation and position of said image sensing apparatus estimated based on the captured image data; and

a display unit adapted to display the composite image data.

9. A computer program, stored in a computer-readable storage medium, for making a computer function as a position and orientation measuring apparatus according to claim 1.

8. A position and orientation measuring method comprising:

storing, in a storage unit, character region specifying information and position information in association with a character region place in a physical space;

detecting the character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit; and

estimating a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the character region, detected in the detection step, in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected region.