Image composition system, image composition method, and image composition apparatus

• US 20050256396A1
• Filed: 05/13/2005
• Published: 11/17/2005
• Est. Priority Date: 05/17/2004
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Takemoto, Rika
• Granted Patent: US 7,627,137 B2
• Time in Patent Office: Days
• US Class Current: 600/414
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

When the user designates an additional marker using an operation input unit in a sense image of the physical world sensed by an image sensing unit, a marker extraction/management unit calculates the world coordinates of the additional marker using known marker group information in the sense image, and registers and manages the world coordinates as new known marker group information.

Claims

1. An information processing method comprising:

acquiring a sense image of an image sensing unit;

acquiring coordinate values of registered indices;

detecting an image position of an additional index in the sense image in accordance with a user'"'"'s instruction;

detecting image positions of the registered indices in the sense image;

calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, and the image position of the additional index; and

registering the additional index as a registered index using the calculated coordinate value.

2. The method according to claim 1, wherein the image sensing unit comprises a video see-through Head Mounted Display.

3. The method according to claim 1, wherein a position and orientation of the image sensing unit are calculated using the image positions of the registered indices and coordinate values of registered indices.

4. The method according to claim 1, further comprising:

generating a virtual image on the basis of a position and orientation of the image sensing unit; and

compositing the sense image and the virtual image.

5. The method according to claim 1, wherein when the coordinate value of the additional index cannot be calculated, the additional index is temporarily registered.

6. The method according to claim 5, wherein when the coordinate value of the temporarily registered additional index is calculated from another sense image, the temporarily registered additional index is regularly registered using the calculated coordinate value.

7. The method according to claim 1, wherein the image position of the additional index is detected based on a position of an operation unit which indicates an additional index on a physical space.

8. The method according to claim 7, wherein information of the registered index includes information for specifying a shape, color information, and ID information in addition to the coordinate value, and

said method further comprises:

detecting a shape region corresponding to the information for specifying the shape in an image region set based on the position of the operation unit; and

obtaining the color information on the basis of a color in the detected shape region.

9. The method according to claim 7, wherein the operation unit comprises a stylus, and a position of the stylus is measured.

10. The method according to claim 7, wherein the operation unit has a predetermined frame, and designates the additional index by locating the additional frame within the predetermined frame.

11. The method according to claim 1, further comprising:

detecting a position of a predetermined portion of a user who indicates an additional index on a physical space; and

detecting the image position of the additional index on the basis of the detection result.

12. An information processing method comprising:

acquiring a sense image of an image sensing unit;

acquiring coordinate values of registered indices;

detecting an image position of an additional index in the sense image;

detecting image positions of the registered indices in the sense image;

calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, and the image position of the additional index;

calculating a feature amount of the additional index from the sense image; and

registering the coordinate value and the feature amount of the additional index.

13. The method according to claim 12, wherein the feature amount is color information.

14. An information processing apparatus comprising:

first acquisition means for acquiring a sense image of an image sensing unit;

second acquisition means for acquiring coordinate values of registered indices;

first detection means for detecting an image position of an additional index in the sense image in accordance with a user'"'"'s instruction;

second detection means for detecting image positions of the registered indices in the sense image;

calculation means for calculating a coordinate value of the additional index using image positions of the registered indices, the coordinate values of the registered indices, and the image position of the additional index; and

registration means for registering the additional index as a registered index using the calculated coordinate value.

15. An information processing apparatus comprising:

first acquisition means for acquiring a sense image of an image sensing unit;

second acquisition means for acquiring coordinate values of registered indices;

first detection means for detecting an image position of an additional index in the sense image;

second detection means for detecting image positions of the registered indices in the sense image;

first calculation means for calculating a coordinate value of the additional index using image positions of the registered indices, the coordinate values of the registered indices, and the image position of the additional index;

second calculation means for calculating a feature amount of the additional index from the sense image; and

registration means for registering the coordinate value and the feature amount of the additional index.

16. A program for controlling an information processing apparatus, comprising:

a program code of a first acquisition step of acquiring a sense image of an image sensing unit;

a program code of a second acquisition step of acquiring coordinate values of registered indices;

a program code of a first detection step of detecting an image position of an additional index in the sense image in accordance with a user'"'"'s instruction;

a program code of a second detection step of detecting image positions of the registered indices in the sense image;

a program code of a calculation step of calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, and the image position of the additional index; and

a program code of a registration step of registering the additional index as a registered index using the calculated coordinate value.

17. A program for controlling an information processing apparatus, comprising:

a program code of a first acquisition step of acquiring a sense image of an image sensing unit;

a program code of a second acquisition step of acquiring coordinate values of registered indices;

a program code of a first detection step of detecting an image position of an additional index in the sense image;

a program code of a second detection step of detecting image positions of the registered indices in the sense image;

a program code of a first calculation step of calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, and the image position of the additional index;

a program code of a second calculation step of calculating a feature amount of the additional index from the sense image; and

a program code of a registration step of registering the coordinate value and the feature amount of the additional index.