Information processing method and apparatus

• US 20060071946A1
• Filed: 09/26/2005
• Published: 04/06/2006
• Est. Priority Date: 09/28/2004
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Anabuki, Mahoro, Kotake, Daisuke, Uchiyama, Shinji
• Granted Patent: US 7,671,875 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

In a case where a position and/or orientation of a shooting viewpoint is calculated by using information about image coordinates of markers placed in a scene, the present invention enables a user to easily determine positions of the markers so that the position and orientation can be calculated more accurately. Information about markers placed in a physical space is obtained and area information about mixing accuracy between a physical space image and a virtual space image is obtained based on the information about the markers, so that a virtual space image is generated in accordance with the obtained area information.

Claims

1. An information processing method for displaying a physical space and a virtual space by mixing the physical space and the virtual space, the information processing method comprising:

an obtaining step of obtaining information about markers placed in the physical space;

a position/orientation calculating step of calculating a position and orientation of a viewpoint;

an area calculating step of calculating area information about mixing accuracy between the physical space and the virtual space based on the information about the markers; and

an image display step of displaying an image by visually representing the area information based on the position and orientation and by mixing the area information on the physical space.

2. The information processing method according to claim 1, wherein the obtaining step obtains position information of the markers placed in the physical space and the area calculating step calculates the area information based on the position information of the markers.

3. The information processing method according to claim 1, wherein the obtaining step obtains position information and normal vectors of the markers placed in the physical space and the area calculating step calculates the area information based on the position information and the normal vectors of the markers.

4. The information processing method according to claim 1, wherein the position/orientation calculating step includes:

a detecting step of detecting a marker position from an image captured from the position of the viewpoint; and

a calculating step of calculating the position and/or orientation of the viewpoint based on the marker position in the captured image and position information of the markers.

5. The information processing method according to claim 1, wherein the area calculating step calculates the area information based on the position of the viewpoint and area information of the markers.

6. The information processing method according to claim 1, wherein the area calculating step calculates the area information based on the number of markers included in an image of the physical space.

7. The information processing method according to claim 1, wherein the area calculating step calculates the area information based on the number of markers included in an image of the physical space and the number of markers in which a distance in the physical space to the markers included in the physical space image is within a predetermined range.

8. The information processing method according to claim 1, wherein the obtaining step obtains information about a marker to be added, and the area calculating step calculates the area information by using the information about the marker to be added.

9. The information processing method according to claim 8, wherein the image display step displays an image by mixing the area information on a physical space image of the marker to be added.

10. The information processing method according to claim 8, further comprising:

a marker type determining step of determining a type of marker to be added,

wherein the area calculating step calculates the area information based on the type of marker determined in the marker type determining step.

11. The information processing method according to claim 1, wherein the information about the markers is marker area information in an image of the physical space, the obtaining step detects a marker area from the physical space image, and the area calculating step calculates the area information in accordance with the position and size of the markers in the physical space image based on the marker area information.

12. The information processing method according to claim 1, further comprising:

a marker selecting step of selecting a marker placed in the physical space,

wherein the area calculating step calculates the area information based on information about the markers except the marker selected in the marker selecting step.

13. The information processing method according to claim 1, wherein the area calculating step calculates the area information only when at least one of addition of a marker, deletion of a marker, and change of information is performed.

14. A program allowing a computer to realize the information processing method according to claim 1.

15. An information processing method for aiding placement of markers used to calculate a position to be measured in a physical space, the information processing method comprising:

a detecting step of detecting a marker area placed in the physical space from a captured image of the physical space;

an area calculating step of calculating area information from which a position and orientation of a viewpoint can be calculated based on the marker area; and

a display step of visually displaying the calculated area information.

16. A program allowing a computer to realize the information processing method according to claim 15.

17. An information processing apparatus for displaying a physical space and a virtual space by mixing the physical space and the virtual space, the information processing apparatus comprising:

an obtaining unit configured to obtain information about markers placed in the physical space;

a position/orientation calculating unit configured to calculate a position and orientation of a viewpoint;

an area calculating unit configured to calculate area information about mixing accuracy between the physical space and the virtual space based on the information about the markers; and

an image display unit configured to display an image by visually representing the area information based on the position and orientation and by mixing the area information on the physical space.

18. An information processing apparatus for aiding placement of markers used to calculate a position to be measured in a physical space, the information processing apparatus comprising:

a detecting unit configured to detect a marker area placed in the physical space from a captured image of the physical space;

an area calculating unit configured to calculate area information from which a position and orientation of a viewpoint can be calculated based on the marker area; and

a display unit configured to visually display the calculated area information.