Image processing method and apparatus

• US 20050174361A1
• Filed: 01/28/2005
• Published: 08/11/2005
• Est. Priority Date: 02/10/2004
• Status: Abandoned Application

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Suzuki, Masahiro, Ohshima, Toshikazu, Kobayashi, Toshihiro
• Application Number:
  US11/044,555
• Publication Number:
  US 20050174361A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus

Abstract

In step S1030, the position and orientation of a stylus operated by the user on the physical space are calculated, and it is detected if the stylus is located on the surface of a real object on the physical space. In step S1040, a virtual index is laid out at the position on the virtual space, which corresponds to the position calculated upon detection. In step S1060, an image of the virtual space including the laid-out virtual index is superimposed on the physical space.

Claims

1. An information processing method for generating an image by combining a virtual image and a physical space image, characterized by comprising:

a designation portion position acquisition step of acquiring a position of a designation portion operated by the user;

a user position/orientation acquisition step of acquiring a position and orientation of the user;

a detection step of detecting if the designation portion is located on a surface of a real object on a physical space;

a virtual index generation step of acquiring the position of the designation portion in response to the detection, and generating a virtual index on the basis of the position of the designation portion;

a virtual image generation step of generating an image of a virtual object corresponding to the real object from virtual space data on the basis of the position and orientation of the user; and

an adjustment step of adjusting the position and orientation of the virtual object in accordance with a user'"'"'s instruction.

2. The method according to claim 1, characterized by further comprising a designation portion virtual index generation step of generating a designation portion virtual index at a tip portion of the designation portion on the basis of the position of the designation portion.

8. A program characterized by making a computer execute an information processing method of claim 1.

3. An information processing method for generating an image by combine a virtual image and a physical space image, characterized by comprising:

a designation portion position acquisition step of acquiring a position of a designation portion operated by the user;

a user position/orientation acquisition step of acquiring a position and orientation of the user;

a detection step of detecting if the designation portion is located on a surface of a real object on a physical space;

an adjustment step of acquiring the position of the designation portion in response to the detection, and adjusting a position and orientation of the virtual object on the basis of the position of the designation portion; and

a virtual image generation step of generating an image of a virtual object corresponding to the real object from virtual space data on the basis of the adjusted position/orientation and the position/orientation of the user.

4. The method according to claim 3, characterized in that the adjustment step includes a step of adjusting the position and orientation of the virtual object along a straight line having the orientation of the designation portion as a direction vector.

5. The method according to claim 3, characterized in that the adjustment step includes steps of:

selecting a surface which forms the virtual object on the basis of the position of the designation portion; and

adjusting the position and orientation of the virtual object in a normal direction of the surface.

6. The method according to claim 3, characterized in that the adjustment step includes a step of rotating the position and orientation of the virtual object using a specific point as a fulcrum or a line segment that connects two specific points as an axis.

7. The method according to claim 3, characterized in that the adjustment step includes a step of adjusting the position and orientation of the virtual object on the basis of positions of a plurality of feature points which are set in advance on the real object.

9. A program characterized by making a computer execute an information processing method of claim 3.

10. An information processing apparatus for generating an image by combining a virtual image and a physical space image, characterized by comprising:

designation portion position acquisition unit configure to acquire a position of a designation portion operated by the user;

user position/orientation acquisition unit configure to acquire a position and orientation of the user;

detection unit configure to detect if the designation portion is located on a surface of a real object on a virtual space;

virtual index generation unit configure to acquire the position of the designation portion in response to the detection, and generating a virtual index on the basis of the position of the designation portion;

virtual image generation unit configure to generate an image of a virtual object corresponding to a real object from virtual space data on the basis of the position and orientation of the user; and

adjustment unit configure to adjust the position and orientation of the virtual object in accordance with a user'"'"'s instruction.

11. An information processing apparatus for generating an image by combining a virtual image and a physical space image, characterized by comprising:

designation portion position acquisition unit configure to acquire a position of a designation portion operated by the user;

user position/orientation acquisition unit configure to acquire a position and orientation of the user;

detection unit configure to detect if the designation portion is located on a surface of a real object on a physical space;

adjustment unit configure to acquire the position of the designation portion in response to the detection, and adjusting a position and orientation of a virtual object on the basis of the position of the designation portion; and

virtual image generation unit configure to generate an image of a virtual object corresponding to a real object from virtual space data on the basis of the adjusted position and orientation and the position and orientation of the user.