Position and orientation measuring apparatus and position and orientation measuring method, mixed-reality system, and computer program

• US 7,965,904 B2
• Filed: 07/30/2007
• Issued: 06/21/2011
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kobayashi, Kazuhiko
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Thirugnanam; Gandhi
• Application Number:
  US11/830,654
• Publication Number:
  US 20080031490A1
• Time in Patent Office: 1,422 Days
• Field of Search: 382/103, 382/154, 382291-292, 345/633
• US Class Current: 382/292
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A position and orientation measuring apparatus comprising, a storage unit adapted to store character region specifying information and position information in association with a character region place in a physical space, a detection unit adapted to detect the character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit, and an estimation unit adapted to estimate a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the character region, detected by the detection unit, in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected region.

Claims

1. A position and orientation measuring apparatus comprising:

a storage unit constructed to store character region specifying information and position information in association with a character region placed in a physical space;

a character region detection unit constructed to detect a character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit;

an image position information detection unit constructed to detect image position information of an index corresponding to the detected character region; and

an estimation unit constructed to estimate a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the detected character region, in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected character region,

wherein the character region detection unit comprises;

a contour extraction unit constructed to extract contours in the captured image data;

a setting unit constructed to set a character region candidate using the contours;

a normalization processing unit constructed to normalize a size of the character region candidate; and

a character recognition processing unit constructed to apply character recognition processing to the normalized character region candidate.

2. The apparatus according to claim 1, wherein when the character region is included in a poster placed in the physical space,

the storage unit stores, as information associated with a character, image information around the character region in the poster, and

the character region detection unit detects the character region further using the image information around the character region.

3. The apparatus according to claim 1, wherein the image position information detection unit comprises:

a save unit constructed to save position information of a character region detected from captured image data captured by the image sensing apparatus at an immediately preceding timing; and

a region estimation unit constructed to estimate an image region corresponding to the character region in the first captured image data based on the saved position information.

4. The apparatus according to claim 1, further comprising a registration unit constructed to calculate position information associated with a character region, which is not registered in the storage unit, using a different parallax image, and to register the calculated position information in the storage unit together with image data of the non-registered character region.

5. A mixed reality system comprising:

an image sensing apparatus;

a position and orientation measuring apparatus comprising;

a storage unit constructed to store character region specifying information and position information in association with a character region placed in a physical space;

a character region detection unit constructed to detect a character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit;

an image position information detection unit constructed to detect the image position information of an index corresponding to the detected character region; and

an estimation unit constructed to estimate a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the detected character region in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected character region,

wherein the character region detection unit comprises;

a contour extraction unit constructed to extract contours in the captured image data;

a setting unit constructed to set a character region candidate using the contours;

a normalization processing unit constructed to normalize a size of the character region candidate; and

a character recognition processing unit constructed to apply character recognition processing to the normalized character region candidate;

a virtual object composition unit constructed to generate composite image data by combining a virtual object and captured image data obtained by capturing an image of a physical space by the image sensing apparatus, based on the captured image data and an orientation and position of the image sensing apparatus estimated based on the captured image data; and

a display unit constructed to display the composite image data.

6. A position and orientation measuring method comprising:

storing, in a storage unit, character region specifying information and position information in association with a character region placed in a physical space;

detecting by a character region detection unit a character region fron first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit;

detecting by an image position information detection unit image position information of an index corresponding to the detected character region; and

estimating by an estimation unit a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the detected character region in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected character region,

wherein the character region detection unit comprises;

a contour extraction unit constructed to extract contours in the captured image data;

a setting unit constructed to set a character region candidate using the contours;

a normalization processing unit constructed to normalize a size of the character region candidate; and

a character recognition processing unit constructed to apply character recognition processing to the normalized character region candidate.

7. A non-transitory computer-readable storage medium on which is stored a computer program for causing a computer to execute a position and orientation measuring method, the program comprising:

storing, in a storage unit, character region specifying information and position information in association with a character region placed in a physical space;

detecting by a character region detection unit a character region from first captured image data obtained by capturing an image of the physical space by an image sensing apparatus, using the character region specifying information stored in the storage unit;

detecting by an image position information detection unit image position information of an index corresponding to the detected character region; and

estimating by an estimation unit a position and orientation of the image sensing apparatus upon capturing the captured image data based on image position information of the detected character region in the first captured image data, and the position information which is stored in the storage unit and corresponds to the detected character region,

wherein the character region detection unit comprises;

a contour extraction unit constructed to extract contours in the captured image data;

a setting unit constructed to set a character region candidate using the contours;

a normalization processing unit constructed to normalize a size of the character region candidate; and

a character recognition processing unit constructed to apply character recognition processing to the normalized character region candidate.