Mixed-reality presentation system and control method therefor

• US 7,834,893 B2
• Filed: 07/24/2007
• Issued: 11/16/2010
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ohshima, Toshikazu, Wanda, Koichiro
• Primary Examiner(s): Good Johnson; M
• Application Number:
  US11/782,194
• Publication Number:
  US 20080030499A1
• Time in Patent Office: 1,211 Days
• Field of Search: 345/633, 345/619, 345/650, 345/661, 345/676
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour

Abstract

In a mixed-reality presentation system, a handler for manipulating a virtual object is displayed as a virtual object. Then, manipulation of the handler by the hand(s) of an observer is detected as manipulation of the virtual object associated with the handler, and those detection results are reflected in a display of the manipulated virtual object. By dynamically changing virtual objects displaying the handler according to the size of the virtual objects displayed in a view volume of the observer presented with a mixed reality space, the observer can select a desired virtual object with ease.

Claims

1. A control method for a mixed-reality presentation system that presents an observer with a mixed reality by superimposing a virtual object rendered according to a position and orientation of a viewpoint of the observer onto a physical space or onto a sensed image of physical space, the control method comprising:

using a processor to perform;

a measuring step of measuring the position and orientation of the viewpoint of the observer;

a virtual object detection step of detecting one or more virtual objects included in a view volume based on the position and orientation of the viewpoint obtained in the measuring step; and

an adding step of adding a manipulation virtual object to each virtual object, from among the one or more virtual objects to be rendered included in the view volume, that satisfies set display conditions,

wherein the display conditions are set as ranges relating to a size of the one or more virtual objects occupying the view volume, a surface area of a projected image of the one or more virtual objects occupying a clipping plane, a number of pixels in the image of the one or more virtual objects, a number of polygons in the one or more virtual objects, a distance between the one or more virtual objects and the viewpoint of the observer, a movement velocity of the viewpoint of the observer and the one or more virtual objects, a range within a color space of the one or more virtual objects, and

wherein the adding step determines whether the manipulation virtual object is to be added or not for each virtual object of the one or more virtual objects included in the view volume, and adds the manipulation virtual object to each of those virtual objects that satisfies the set display conditions.

2. The control method for mixed-reality presentation system according to claim 1, wherein the adding step deletes the manipulation virtual object associated with virtual objects that no longer satisfy the set display conditions from among the one or more virtual objects to be rendered.

3. The control method for mixed-reality presentation system according to claim 1, wherein the adding step sets a position of the manipulation virtual object to a position that does not overlap the virtual object associated with the manipulation virtual object, and further, where the observer can manipulate the manipulation virtual object by hand.

4. The control method for mixed-reality presentation system according to claim 1, further comprising:

a manipulation detection step of detecting a manipulation by the observer of the manipulation virtual object; and

a changing step of changing rendering attributes of the virtual object so that a manipulation detected in the manipulation detection step is reflected in rendering results of the virtual object associated with the manipulation virtual object.

5. The control method for mixed-reality presentation system according to claim 4, wherein the manipulation detection step detects the manipulation based on a region of the observer'"'"'s hand(s) included in an image sensed from a vicinity of the position of the viewpoint of the observer.

6. The control method for mixed-reality presentation system according to claim 1, wherein the display conditions are determined by a distance between the one or more virtual objects and the viewpoint of the observer, a movement velocity of the viewpoint of the observer, or relative velocities of the viewpoint of the observer and the virtual object.

7. A mixed-reality presentation system that presents an observer with a mixed reality by superimposing a virtual object rendered according to a position and orientation of a viewpoint of the observer onto a physical space or onto a sensed image of physical space, the mixed-reality presentation system comprising:

a measuring unit constructed to measure the position and orientation of the viewpoint of the observer;

a virtual object detection unit constructed to detect one or more virtual objects included in a view volume based on the position and orientation of the viewpoint obtained by the measuring unit; and

an adding unit constructed to add a manipulation virtual object to each virtual object, from among the one or more virtual objects to be rendered included in the view volume, satisfies set display conditions,

wherein the display conditions are set as ranges relating to a size of the one or more virtual objects occupying the view volume, a surface area of a projected image of the one or more virtual objects occupying a clipping plane, a number of pixels in the image of the one or more virtual objects, a number of polygons in the one or more virtual objects, a distance between the one or more virtual objects and the viewpoint of the observer, a movement velocity of the viewpoint of the observer and the one or more virtual objects, a range within a color space of the one or more virtual objects, and

wherein the adding unit determines whether the manipulation virtual object is to be added or not for each virtual object of the one or more virtual objects included in the view volume, and adds the manipulation virtual object to each of those virtual objects that satisfies the set display conditions.

8. A computer readable storage medium storing a program for causing a computer to function as the mixed-reality presentation system according to claim 7.