Compound reality presentation apparatus, method therefor, and storage medium

• US 20020037768A1
• Filed: 03/21/2001
• Published: 03/28/2002
• Est. Priority Date: 09/27/2000
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ohshima, Toshikazu
• Granted Patent: US 6,685,566 B2
• Time in Patent Office: Days
• US Class Current: 463/30
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

When S=0, and a user waves his or her hand five times or more from side to side (S1502), the head of a dolphin appears from the water, and the number L1 of times of status changes is increased by 1 (S1504). When the user waves his or her hand twice or more up and down (S1503), the dolphin jumps from the water, and the number L1 is increased by 1 (S1505). When S=1, and a user waves his or her hand three times or more from side to side, the dolphin disappears into the water, and the number L1 is increased by 1 (S1507). When S=2, and the dolphin jumps into the water, then S=0 (S1509). If S=3, and the dolphin disappears into the water, then S=0 (S1511). Then, it is determined whether or not L1≧3 (S1512). If yes, the process is passed to the communications level 2 (S1513).

Claims

1. A compound reality presentation apparatus which generates an image of a virtual object, superposes the image on real space, and presents a result to a user, comprising:

measurement means for measuring a position and a posture of a predetermined portion of a body of a user;

detection means for detecting a predetermined movement of the predetermined portion of the body based on the measurement result of said measurement means;

change means for changing an internal status of the virtual object into another internal status based on the detection result of said detection means; and

presentation means for presenting a user with a picture of a virtual object based on the internal status.

2. The compound reality presentation apparatus according to claim 1, further comprising:

capture means for capturing a picture of real space; and

compound means for compounding a picture of the real space obtained by said capture means and the virtual object, and generating a compound picture, wherein

said presentation means presents the user with a compound picture generated by said compound means contains the picture of the virtual object.

3. The compound reality presentation apparatus according to claim 1, further comprising

management means for managing the predetermined movement of the predetermined portion of the body associated with the information about the status change of the virtual object, wherein

said change means changes the status of the virtual object by referring to the information, which is managed by said management means, about a status change of the virtual object corresponding to the predetermined movement by the predetermined portion of the body.

4. The compound reality presentation apparatus according to claim 1, further comprising

contents storage means for storing the information about contents issued by the virtual object, wherein

said change means selects information corresponding to a state of the virtual object from the information stored in said contents storage means when a status of the virtual object is changed.

5. The compound reality presentation apparatus according to claim 4, wherein

said information about the contents issued by the virtual object contains either data of voice indicating the contents, or data of a character indicating the contents.

6. A compound reality presentation apparatus which provides virtual communications with a virtual creature by presenting a user with a picture of the virtual creature, whose visible status such as appearance, movement, etc. and/or internal status are determined by a plurality of status determination parameters, and/or a picture of environment of the virtual creature both of which are superimposed on real space or its picture, comprising:

measurement means for measuring a position and a posture of a predetermined portion of a body of a user;

detection means for detecting a movement of the predetermined portion of the body based on a measurement result of said measurement means;

status change means for changing a value of a predetermined parameter in the plurality of status determination parameters based on a detection result of said detection means;

picture generation means for generating a picture of the virtual creature based on the plurality of status determination parameters; and

presentation means for presenting a user with the picture of the virtual creature superposed on the real space or a picture of the real space.

7. A compound reality presentation method for presenting compound reality after generating an image of a virtual object, superposing the image on real space, and presenting a result to a user, comprising:

a measuring step of measuring a position and a posture of a predetermined portion of a body of a user;

a detecting step of detecting a predetermined movement of the predetermined portion of the body based on the measurement result of said measuring step;

a changing step of changing an internal status of the virtual object into another internal status based on the detection result of said detecting step; and

a presenting step of presenting a user with a picture of a virtual object based on the internal status.

8. The compound reality presentation method according to claim 7, further comprising:

a capturing step of capturing a picture of real space by predetermined capturing means; and

a compounding step of compounding a picture of the real space obtained in said capturing step and the virtual object, and generating a compound picture, wherein

in said presenting step, the user is presented with a compound picture generated in said compounding step contains the picture of the virtual object.

9. The compound reality presentation method according to claim 7, further comprising

a managing step of managing the predetermined movement of the predetermined portion of the body associated with the information about the status change of the virtual object by the predetermined management means, wherein

in said changing step, the status of the virtual object is changed by referring to the information, which is managed by the predetermined management means, about a status change of the virtual object corresponding to the predetermined movement by the predetermined portion of the body.

10. The compound reality presentation method according to claim 7, further comprising

a contents storing step of storing the information about contents issued by the virtual object in a predetermined storage means, wherein

in said changing step, information corresponding to a state of the virtual object is selected from the information stored in the predetermined storage means when a status of the virtual object is changed.

11. A compound reality presentation method for providing virtual communications with a virtual creature by presenting a user with a picture of the virtual creature, whose visible status such as appearance, movement, etc. and/or internal status are determined by a plurality of status determination parameters, and/or a picture of environment of the virtual creature both of which are superimposed on real space or its picture, comprising:

a measuring step of measuring a position and a posture of a predetermined portion of a body of a user;

a detecting step of detecting a movement of the predetermined portion of the body based on a result of said measuring step;

a status changing step of changing a value of a predetermined parameter in the plurality of status determination parameters based on a detection result of said detecting step;

a picture generating step of generating a picture of the virtual creature based on the plurality of status determination parameters; and

a presenting step of presenting a user with the picture of the virtual creature superposed on the real space or a picture of the real space.

13. The storage medium according to claim 12, further comprising:

a program code of a capturing step of capturing a picture of real space by predetermined capturing means; and

a program code of a compounding step of compounding a picture of the real space obtained in said capturing step and the virtual object, and generating a compound picture, wherein

in said presenting step, the user is presented with a compound picture generated in said compounding step contains the picture of the virtual object.

14. The storage medium according to claim 12, further comprising

a program code of a managing step of managing the predetermined movement of the predetermined portion of the body associated with the information about the status change of the virtual object by the predetermined management means, wherein

in said changing step, the status of the virtual object is changed by referring to the information, which is managed by the predetermined management means, about a status change of the virtual object corresponding to the predetermined movement by the predetermined portion of the body.

15. The storage medium according to claim 12, further comprising

a program code of a contents storing step of storing the information about contents issued by the virtual object in a predetermined storage means, wherein

in said changing step, information corresponding to a state of the virtual object is selected from the information stored in the predetermined in the predetermined storage means when a status of the virtual object is changed.

12. A storage medium storing a program code for use in presenting compound reality after generating an image of a virtual object, superposing the image on real space, and presenting a result to a user, comprising:

a program code of a measuring step of measuring a position and a posture of a predetermined portion of a body of a user;

a program code of a detecting step of detecting a predetermined movement of the predetermined portion of the body based on the measurement result of said measuring step;

a program code of a changing step of changing an internal status of the virtual object into another internal status based on the detection result of said detecting step; and

a program code of a presenting step of presenting a user with a picture of a virtual object based on the internal status.

16. A storage medium storing a program code for use in presenting compound reality which provides virtual communications with a virtual creature by presenting a user with a picture of the virtual creature, whose visible status such as appearance, movement, etc. and/or internal status are determined by a plurality of status determination parameters and/or a picture of environment of the virtual creature both of which are superimposed on real space or its picture, comprising:

a program code of a measuring step of measuring a position and a posture of a predetermined portion of a body of a user;

a program code of a detecting step of detecting a movement of the predetermined portion of the body based on a result of said measuring step;

a program code of a status changing step of changing a value of a predetermined parameter in the plurality of status determination parameters based on a detection result of said detecting step;

a program code of a picture generating step of generating a picture of the virtual creature based on the plurality of status determination parameters; and

a program code of a presenting step of presenting a user with the picture of the virtual creature superposed on the real space or a picture of the real space.