SYSTEM

• US 20080266386A1
• Filed: 03/31/2008
• Published: 10/30/2008
• Est. Priority Date: 04/25/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Maeda, Sonoko
• Granted Patent: US 8,049,775 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/373 for tracking forward-backward translational head movements, i.e. longitudinal movements

Abstract

An unit (317, 318) acquires identification information from a camera (101R) (left camera 101L). The unit (303, 312) checks a connection state based on the identification information, and determines the subsequent operation. A unit (305) transmits position and orientation information of the camera (101R) to an apparatus (206). A unit (308) outputs a composite image of a virtual space image generated based on the position and orientation information and a captured image input by a unit (302) to an HMD (100). A unit (313) calculates the position and orientation of the camera (101L) using position and orientation relationship information and the position and orientation information. An unit (316) outputs a composite image of a virtual space image generated based on the position and orientation of the camera (101L), and a captured image input by a unit (311) to the HMD (100).

Claims

1. A system which comprises a first image processing apparatus that receives an image of a physical space captured by a first camera of a head mounted display device, and a second image processing apparatus that receives an image of the physical space captured by a second camera of the head mounted display device,

said first image processing apparatus comprises:

a determination unit adapted to determine whether or not the first camera is a master;

a unit adapted to acquire, when said determination unit determines that the first camera is a master, position and orientation information of the first camera, and to acquire, when said determination unit determines that the first camera is not a master, position and orientation information of the second camera from said second image processing apparatus;

a unit adapted to output, when said determination unit determines that the first camera is a master, position and orientation information of the first camera to said second image processing apparatus;

a unit adapted to acquire, when said determination unit determines that the first camera is not a master, position and orientation relationship information indicating a position and orientation relationship between the first camera and the second camera;

a unit adapted to calculate, when said determination unit determines that the first camera is not a master, position and orientation information of the first camera based on the position and orientation information of the second camera and the position and orientation relationship information;

a unit adapted to generate a virtual space image based on the position and orientation information of the first camera;

a unit adapted to acquire an image of the physical space captured by the first camera;

a generation unit adapted to generate a composite image formed by compositing the image of the physical space and the virtual space image; and

a unit adapted to output the composite image to the head mounted display device, and

said second image processing apparatus comprises;

a second determination unit adapted to determine whether or not the second camera is a master;

a unit adapted to acquire, when said second determination unit determines that the second camera is a master, position and orientation information of the second camera, and to acquire, when said second determination unit determines that the second camera is not a master, position and orientation information of the first camera from said first image processing apparatus;

a unit adapted to output, when said second determination unit determines that the second camera is a master, the position and orientation information of the second camera to said first image processing apparatus;

a unit adapted to acquire, when said second determination unit determines that the second camera is not a master, position and orientation relationship information indicating a position and orientation relationship between the first camera and the second camera;

a unit adapted to calculate, when said second determination unit determines that the second camera is not a master, position and orientation information of the second camera based on the position and orientation information of the first camera and the position and orientation relationship information;

a unit adapted to generate a virtual space image based on the position and orientation information of the second camera;

a unit adapted to acquire an image of the physical space captured by the second camera;

a second generation unit adapted to generate a composite image formed by compositing the image of the physical space captured by the second camera and the virtual space image generated based on the position and orientation information of the second camera; and

a unit adapted to output the composite image to the head mounted display device.

2. The system according to claim 1, wherein the head mounted display device comprises:

a first display device adapted to display the composite image generated by said generation unit of said first image processing apparatus; and

a second display device adapted to display the composite image generated by said second generation unit of said second image processing apparatus.

3. The system according to claim 1, wherein said determination unit of said first image processing apparatus and said second determination unit of said second image processing apparatus, respectively, determine, based on identification information transmitted from the first camera and the second camera, that one of the first camera and the second camera is a master.