SPATIAL ASSOCIATION BETWEEN VIRTUAL AND AUGMENTED REALITY

• US 20110316845A1
• Filed: 06/25/2010
• Published: 12/29/2011
• Est. Priority Date: 06/25/2010
• Status: Abandoned Application

• Current Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Roberts, Michael, Begole, James M.A., Chu, Maurice K., Kletter, Doron
• Application Number:
  US12/823,939
• Publication Number:
  US 20110316845A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

One embodiment of the present invention provides a system that facilitates interaction between two entities located away from each other. The system includes a virtual reality system, an augmented reality system, and an object-state-maintaining mechanism. During operation, the virtual reality system displays an object associated with a real-world object. The augmented reality system displays the object based on a change to the state of the object. The object-state-maintaining mechanism determines the state of the object and communicates a state change to the virtual reality system, the augmented reality system, or both. A respective state change of the object can be based on one or more of: a state change of the real-world object; a user input to the virtual reality system or the augmented reality system; and an analysis of an image of the real-world object.

Claims

1. A system, comprising:

a virtual reality system configured to display an object associated with a real-world object;

an augmented reality system configured to display the object based on a change to the state of the object; and

an object-state-maintaining mechanism configured to determine the state of the object and communicate a state change to the virtual reality system, the augmented reality system, or both;

wherein a respective state change of the object can be based on one or more of;

a state change of the real-world object;

a user input to the virtual reality system or the augmented reality system; and

an analysis of an image of the real-world object.

2. The system of claim 1, wherein the virtual reality system is configured to allow a user to manipulate the displayed object and communicate information indicative of a state change of the object corresponding to the user manipulation to the object-state-maintaining mechanism;

wherein the object-state-maintaining mechanism is configured to communicate the received information indicative of the state change to the augmented reality system; and

wherein the augmented reality system is configured to update the displayed object based on the received information indicative of the state change.

3. The system of claim 2, wherein the information indicative of the state change comprises a set of parameters which can be used to identify the object and a state of the object.

4. The system of claim 1, wherein the object-state-maintaining mechanism further comprises a scene recognition engine configured to determine a model for the object and communicate information indicative of the model to the virtual reality system, the augmented reality system, or both.

5. The system of claim 1, wherein the augmented reality system comprises a machine vision mechanism configured to capture an image of the real-world object;

and

wherein the augmented reality system is further configured to communicate the image or a set of parameters that can be used to identify the real-world object and a state of the real-world object to the object-state-maintaining mechanism.

6. The system of claim 1, wherein the virtual reality system and the augmented reality system are both configured to display the object in a three-dimension (3D) display environment.

7. The system of claim 1, wherein the object-state-maintaining mechanism resides in a server which is coupled to the virtual reality system and the augmented reality system via a network.

8. A method, comprising:

displaying at a virtual reality system an object associated with a real-world object;

determining the state of the object by an object-state-maintaining mechanism based on one or more of;

a state change of the real-world object;

a user input; and

an analysis of an image of the real-world object;

communicating a state change of the object to an augmented reality system; and

displaying at the augmented reality system the object based on the state change.

9. The method of claim 8, further comprising:

allowing a user to manipulate the displayed object at the virtual reality system;

communicating information indicative of a state change of the object corresponding to the user manipulation from the virtual reality system to the object-state-maintaining mechanism;

communicating the received information indicative of the state change from the object-state-maintaining mechanism to the augmented reality system; and

updating the object which is displayed in the augmented reality system based on the received information indicative of the state change.

10. The method of claim 9, wherein the information indicative of the state change comprises a set of parameters which can be used to identify the object and a state of the object.

11. The method of claim 8, further comprising determining a model for the object and communicating information indicative of the model to the virtual reality system, the augmented reality system, or both.

12. The method of claim 8, further comprising:

capturing an image of the real-world object by a machine vision mechanism associated with the augmented reality system; and

communicating the image or a set of parameters which can be used to identify the real-world object and a state of the real-world object to the object-state-maintaining mechanism.

13. The method of claim 8, wherein the object is displayed in a three-dimension (3D) display environment.

14. The method of claim 8, wherein the object-state-maintaining mechanism resides in a server which is coupled to the virtual reality system and the augmented reality system via a network.

15. A computer readable non-transitory storage medium storing instructions which when executed by one or more computers cause the computer(s) to execute a method, the method comprising:

displaying at a virtual reality system an object associated with a real-world object;

determining the state of the object by an object-state-maintaining mechanism based on one or more of;

a state change of the real-world object;

a user input; and

an analysis of an image of the real-world object;

communicating a state change of the object to an augmented reality system; and

displaying at the augmented reality system the object based on the state change.

16. The computer readable non-transitory storage medium of claim 15, wherein the method further comprises:

allowing a user to manipulate the displayed object at the virtual reality system;

communicating information indicative of a state change of the object corresponding to the user manipulation from the virtual reality system to the object-state-maintaining mechanism;

communicating the received information indicative of the state change from the object-state-maintaining mechanism to the augmented reality system; and

updating the object which is displayed in the augmented reality system based on the received information indicative of the state change.

17. The computer readable non-transitory storage medium of claim 16, wherein the information indicative of the state change comprises a set of parameters which can be used to identify the object and a state of the object.

18. The computer readable non-transitory storage medium of claim 15, wherein the method further comprises determining a model for the object and communicating information indicative of the model to the virtual reality system, the augmented reality system, or both.

19. The computer readable non-transitory storage medium of claim 15, wherein the method further comprises:

capturing an image of the real-world object by a machine vision mechanism associated with the augmented reality system; and

communicating the image or a set of parameters which can be used to identify the real-world object and a state of the real-world object to the object-state-maintaining mechanism.

20. The computer readable non-transitory storage medium of claim 15, wherein the object is displayed in a three-dimension (3D) display environment.

21. The computer readable non-transitory storage medium of claim 15, wherein the object-state-maintaining mechanism resides in a server which is coupled to the virtual reality system and the augmented reality system via a network.