Virtual reality presentation apparatus, virtual reality presentation method, program, image processing method, image processing apparatus, information processing method, and information processing apparatus

• US 20060256110A1
• Filed: 05/05/2006
• Published: 11/16/2006
• Est. Priority Date: 05/11/2005
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Noro, Hideo, Okuno, Yasuhiro, Matsui, Taichi
• Granted Patent: US 7,773,098 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: Y10S 345/958 Collision avoidance

Abstract

A virtual reality presentation apparatus (100) has a CPU (101) which controls the overall apparatus, a memory (103) which stores various programs, a memory (104) which stores various data, an HMD (105) as a video display device, and a position and orientation controller (106) having position and orientation sensors (107, 108, 109), which are connected to each other via a bus (102). After a position and orientation required to render a CG scene for one frame are determined, when CG objects are in an interference state immediately before this processing and the difference between the current time and an interference time (128) set upon determining the interference state does not exceed an interference detection skip time (124), CG image generation means (112) generates a CG scene for one frame based on the determined position and orientation without checking if an interference has occurred between the CG objects.

Claims

1. A virtual reality presentation apparatus which displays an image generated based on three-dimensionally modeled CG objects, and displays an image -according to a-change in line of sight position of an observer, comprising:

interference detection unit adapted to detect an interference between the CG objects; and

interference detection suppression unit adapted to suppress the interference detection within a detection suppression time period after detection of the interference by said interference detection unit.

2. The apparatus according to claim 1, wherein said interference detection suppression unit comprises:

processing time measuring unit adapted to measure a processing time period required for immediately preceding interference detection; and

detection suppression time period setting unit adapted to set the detection suppression time period in accordance with the measured processing time period.

3. A virtual reality presentation method which displays an image generated based on three-dimensionally modeled CG objects, and displays an image according to a change in line of sight position of an observer, comprising:

an interference detection step of detecting an interference between the CG objects; and

an interference detection suppression step of suppressing the interference detection within a detection suppression time period after detection of the interference in the interference detection step.

4. A program for making a computer execute a virtual reality presentation method which displays an image generated based on three-dimensionally modeled CG objects, and displays an image according to a change in line of sight position of an observer, comprising:

an interference detection module for detecting an interference between the CG objects; and

an interference detection suppression module for suppressing the interference detection within a detection suppression time period after detection of the interference by said interference detection module.

5. A virtual reality presentation apparatus which displays an image generated based on three-dimensionally modeled CG objects, and displays an image according to a change in line of sight position of an observer, comprising:

moving unit adapted to move a CG object to be manipulated based on a manipulation of the observer;

check unit adapted to check an interference between the CG object to be moved and another CG object; and

setting unit adapted to set a condition upon executing interference check processing again after said check unit checks the interference, and

in that said check unit checks the interference based on the condition set by said setting unit.

6. A virtual reality presentation method which displays an image generated based on three-dimensionally modeled CG objects, and displays an image according to a change in line of sight position of an observer, comprising:

a moving step of moving a CG object to be manipulated based on a manipulation of the observer;

a check step of checking an interference between the CG object to be moved and another CG object; and

a setting step of setting a condition upon executing interference check processing again after the interference is checked in the check step, and

in that the interference is checked in the check step based on the condition set in the setting step.

7. An image processing method comprising:

an acquisition step of acquiring a position and orientation of a viewpoint of an observer;

a layout step of laying out a plurality of virtual objects on a virtual space;

a generation step of generating an image viewed when the virtual space on which the plurality of virtual objects are-laid out is seen from the viewpoint having the position and orientation acquired in the acquisition step;

an output step of externally outputting the image generated in the generation step; and

a recording step of recording, when a collision between the virtual objects is detected, information associated with the collision in a memory.

8. The method according to claim 7, further comprising:

a second acquisition step of acquiring an image of a physical space which is seen from the viewpoint, and

in that the output step includes a step of superimposing the image generated in the generation step on the image acquired in the second acquisition step, and outputting the superimposed image.

9. The method according to claim 8, wherein the recording step includes a step of recording, in the memory, a detection time of the collision, the position and orientation of the viewpoint acquired in the acquisition step, positions and orientations of the colliding virtual objects, information used to identify the colliding virtual objects, information indicating colliding parts, and the image obtained by superimposing the image generated in the generation step on the image acquired in the second acquisition step.

11. A program making a computer execute an image processing method of claim 7.

12. A computer-readable storage medium storing a program of claim 11.

10. An image processing apparatus comprising:

acquisition unit adapted to acquire a position and orientation of a viewpoint of an observer;

layout unit adapted to lay out a plurality of virtual objects on a virtual space;

generation unit adapted to generate an image viewed when the virtual space on which the plurality of virtual objects are laid out is seen from the viewpoint having the position and orientation acquired by said acquisition unit;

output unit adapted to externally output the image generated by said generation unit; and

recording unit adapted to, when a collision between the virtual objects is detected, record information associated with the collision.

13. An information processing method which comprises:

a first acquisition step of acquiring a position and orientation of a viewpoint of an observer;

a second acquisition step of acquiring a position and orientation of a pointing tool held by a hand of the observer;

a layout step of laying out a first virtual object on a virtual space at the position and orientation acquired in the second acquisition step; and

a generation step of generating an image viewed when the virtual space including the first virtual object is seen from the viewpoint having the position and orientation acquired in the first acquisition step,

said method further comprising;

a calculation step of calculating a moving velocity of the pointing tool;

a determination step of determining based on the moving velocity of the pointing tool whether or not an interference state between the first virtual object and a second virtual object laid out on the virtual space is to be checked; and

a notification step of notifying occurrence of an interference when the first virtual object and the second virtual object interfere with each other as a result of checking the interference state between the first virtual object and the second virtual object.

14. The method according to claim 13, wherein the calculation step includes a step of calculating a difference vector between a position acquired in the second acquisition step at time t and a position acquired in the second acquisition step at time (t-1), and obtaining a size of the difference vector as a moving velocity of the pointing tool at time t.

15. The method according to claim 13, wherein the determination step includes a step of determining, when the moving velocity of the pointing tool is not less than a predetermined value, that the interference state between the first virtual object and the second virtual object is not to be checked.

21. The method according to claim 13, further comprising:

a third acquisition step of acquiring an image of a physical space which is seen from the viewpoint; and

an output step of superimposing the image generated in the generation step on the image acquired in the third acquisition step, and externally outputting the superimposed image.

25. A program making a computer execute an information processing method of claim 13.

26. A computer-readable storage medium storing a program of claim 25.

16. An information processing method which comprises:

a first acquisition step of acquiring a position and orientation of a viewpoint of an observer;

a second acquisition step of acquiring a position and orientation of a pointing tool held by a hand of the observer;

a layout step of laying out a first virtual object on a virtual space at the position and orientation acquired in the second acquisition step; and

a generation step of generating an image viewed when the virtual space including the first virtual object is seen from the viewpoint having the position and orientation acquired in the first acquisition step, said method further comprising;

a determination step of determining based on a moving direction of the pointing tool and a positional relationship between the first virtual object and a second virtual object laid out on the virtual space whether or not an interference state between the first virtual object and the second virtual object is to be checked; and

a notification step of notifying occurrence of an interference when the first virtual object and the second virtual object interfere with each other as a result of checking the interference state between the first virtual object and the second virtual object.

17. The method according to claim 16, wherein the determination step includes a step of determining, when the first virtual object and the second virtual object have a predetermined positional relationship, that the interference state between the first virtual object and the second virtual object is to be checked.

18. The method according to claim 16, wherein the determination step includes a step of calculating, when the first virtual object and the second virtual object do not have a predetermined positional relationship, an inner product value of a vector from a position of the pointing tool toward a position of the second virtual object and a moving direction vector of the pointing tool, and determining, when the calculated inner product value is positive, that the interference state between the first virtual object and the second virtual object is not to be checked.

19. An information processing method which comprises:

a first acquisition step of acquiring a position and orientation of a viewpoint of an observer;

a second acquisition step of acquiring a position and orientation of a pointing tool held by a hand of the observer;

a layout step of laying out a first virtual object on a virtual space at the position and orientation acquired in the second acquisition step; and

a generation step of generating an image viewed when the virtual space including the first virtual object is seen from the viewpoint having the position and orientation acquired in the first acquisition step,

said method further comprising;

an estimation step of estimating a load on processing for checking an interference state between the first virtual object and a second virtual object laid out on the virtual space;

a determination step of determining based on the load estimated in the estimation step whether or not the check processing is to be executed; and

a notification step of notifying occurrence of an interference when the first virtual object and the second virtual object interfere with each other as a result of checking the interference state between the first virtual object and the second virtual object.

20. The method according to claim 19, wherein the determination step includes a step of determining, when the load estimated in the estimation step is not less than a predetermined value, that the interference state between the first virtual object and the second virtual object is not to be checked.

22. An information processing apparatus which comprises:

first acquisition unit adapted to acquire a position and orientation of a viewpoint of an observer;

second acquisition unit adapted to acquire a position and orientation of a pointing tool held by a hand of the observer;

layout unit adapted to lay out a first virtual object on a virtual space at the position and orientation acquired by said second acquisition unit; and

generation unit adapted to generate an image viewed when the virtual space including the first virtual object is seen from the viewpoint having the position and orientation acquired by said first acquisition unit,

said apparatus further comprising;

calculation unit adapted to calculate a moving velocity of the pointing tool;

determination unit adapted to determine based on the moving velocity of the pointing tool whether or not an interference state between the first virtual object and a second virtual object laid out on the virtual space is to be checked; and

notification unit adapted to notify occurrence of an interference when the first virtual object and the second virtual object interfere with each other as a result of checking the interference state between the first virtual object and the second virtual object.

23. An information processing apparatus which comprises:

first acquisition unit adapted to acquire a position and orientation of a viewpoint of an observer;

second acquisition unit adapted to acquire a position and orientation of a pointing tool held by a hand of the observer;

layout unit adapted to lay out a first virtual object on a virtual space at the position and orientation acquired by said second acquisition unit; and

generation unit adapted to generate an image viewed when the virtual space including the first virtual object is seen from the viewpoint having the position and orientation acquired by said first acquisition unit,

said apparatus further comprising;

determination unit adapted to determine, based on a moving direction of the pointing tool and a positional relationship between the first virtual object and a second virtual object laid out on the virtual space, whether or not an interference state between the first virtual object and the second virtual object is to be checked; and

notification unit adapted to notify occurrence of an interference when the first virtual object and the second virtual object interfere with each other as a result of checking the interference state between the first virtual object and the second virtual object.

24. An information processing apparatus which comprises:

first acquisition unit adapted to acquire a position and orientation of a viewpoint of an observer;

second acquisition unit adapted to acquire a position and orientation of a pointing tool held by a hand of the observer;

layout unit adapted to lay out a first virtual object on a virtual space at the position and orientation acquired by said second acquisition unit; and

generation unit adapted to generate an image viewed when the virtual space including the first virtual object is seen from the viewpoint having the position and orientation acquired by said first acquisition unit,

said apparatus further comprising;

estimation unit adapted to estimate a load on processing for checking an interference state between the first virtual object and a second virtual object laid out on the virtual space;

determination unit adapted to determine based on the load estimated by said estimation unit whether or not the check processing is to be executed; and

notification unit adapted to notify occurrence of an interference when the first virtual object and the second virtual object interfere with each other as a result of checking the interference state between the first virtual object and the second virtual object.

27. An information processing method having:

an acquisition step of acquiring a position of a pointing tool manipulated by an observer;

a layout step of laying out a first virtual object according to the position of the pointing tool; and

a notification step of determining an interference state between the first virtual object and a second virtual object different from the first virtual object, and making notification according to the determination result, characterized in that

the notification step is controlled in accordance with a moving velocity of the pointing tool.

28. A virtual reality presentation apparatus which displays an image generated based on three-dimensionally modeled CG objects, and displays an image according to a change in line of sight position of an observer, comprising:

detection unit adapted to detect occurrence of a predetermined condition; and

interference detection suppression unit adapted to suppress detection of an interference between the CG objects in accordance with the detection by said detection unit.

29. The apparatus according to claim 28, wherein said detection unit detects an interference between the CG objects, and

said interference detection suppression unit suppresses detection of the interference within a detection suppression time period after detection of the interference by said detection unit.

30. The apparatus according to claim 28, wherein said detection unit detects if a moving velocity of a pointing tool is not more than a predetermined value, wherein one CG object is laid out at a position and orientation of the pointing tool, and

said interference detection suppression unit suppresses detection of the interference when said detection unit detects that the moving velocity is not more than the predetermined value.

31. The apparatus according to claim 29, further comprising recording unit adapted to, when said detection unit detects the interference between the CG objects, record information associated with the interference in a memory.

32. A virtual reality presentation method executed by a virtual reality presentation apparatus which displays an image generated based on three-dimensionally modeled CG objects, and displays an image according to a change in line of sight position of an observer, comprising:

a detection step of detecting occurrence of a predetermined condition; and

an interference detection suppression step of suppressing detection of an interference between the CG objects in accordance with the detection in the detection step.

33. A program making a computer to execute a virtual reality presentation method of claim 32.