Systems and Methods for Interaction With a Virtual Environment

• US 20110084983A1
• Filed: 06/24/2010
• Published: 04/14/2011
• Est. Priority Date: 09/29/2009
• Status: Abandoned Application

• Current Assignee: Experience Proximity, Inc.
• Original Assignee: Wavelength & Resonance, LLC
• Inventors: Demaine, Kent
• Application Number:
  US12/823,089
• Publication Number:
  US 20110084983A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/302 specially adapted for receiving control signals not targeted to a display device or game input means, e.g. vibrating driver's seat, scent dispenser
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Systems and methods for interaction with a virtual environment are disclosed. In some embodiments, a method comprises generating a virtual representation of a user'"'"'s non-virtual environment, determining a viewpoint of a user in a non-virtual environment relative to a display, and displaying, with the display, the virtual representation in a spatial relationship with the user'"'"'s non-virtual environment based on the viewpoint of the user.

Claims

1. A method comprising:

generating a virtual representation of a user'"'"'s non-virtual environment;

determining a viewpoint of a user in a non-virtual environment relative to a display; and

displaying, with the display, the virtual representation in a spatial relationship with the user'"'"'s non-virtual environment based on the viewpoint of the user.

2. The method of claim 1, further comprising placing the display relative to the user'"'"'s non-virtual environment.

3. The method of claim 1, wherein the display is not transparent.

4. The method of claim 1, wherein generating the virtual representation of the user'"'"'s non-virtual environment comprises taking one or more digital photographs of the user'"'"'s non-virtual environment and generating the virtual representation based on the one or more digital photographs.

5. The method of claim 1, wherein a camera directed at the user is used to determine the viewpoint of the user in the non-virtual environment relative to the display.

6. The method of claim 1, wherein determining the viewpoint of the user comprises performing facetracking of the user to determine the viewpoint.

7. The method of claim 1, further comprising displaying virtual content within the virtual representation.

8. The method of claim 7, wherein the method further comprises displaying an interaction between the virtual content and the virtual representation.

9. The method of claim 7, wherein the user interacts with the display to change the virtual content.

10. A system comprising:

a virtual representation module configured to generate a virtual representation of a user'"'"'s non-virtual environment;

a viewpoint module configured to determine a viewpoint of a user in a non-virtual environment; and

a display configured to display the virtual representation in a spatial relationship with a user'"'"'s non-virtual environment based, at least in part, on the determined viewpoint.

11. The system of claim 10, wherein the display is not transparent.

12. The system of claim 10, wherein the virtual representation module configured to generate the virtual representation of the user'"'"'s non-virtual environment comprises the virtual representation module configured to receive one or more digital photographs of the user'"'"'s non-virtual environment and to generate the virtual representation based on the one or more digital photographs.

13. The system of claim 10, wherein the viewpoint module comprises one or more cameras configured to determine the viewpoint of the user in the non-virtual environment relative to the display.

14. The system of claim 10, wherein the one or more cameras are configured to perform facetracking of the user to determine the viewpoint.

15. The system of claim 10, further comprising a virtual content module configured to display virtual content within the virtual representation.

16. The method of claim 15, wherein the virtual content module is further configured to interact the virtual content with the virtual representation.

17. The method of claim 15, further comprising a display interface module configured to receive interaction with the user and to display a change to the virtual content based on the interaction.

18. A computer readable medium configured to store executable instructions, the instructions being executable by a processor to perform a method, the method comprising:

generating a virtual representation of a user'"'"'s non-virtual environment;

determining a viewpoint of a user in a non-virtual environment relative to a display; and

displaying, with the display, the virtual representation in a spatial relationship with the user'"'"'s non-virtual environment based on the viewpoint of the user.

19. The computer readable medium of claim 18, wherein the method further comprises displaying virtual content within the virtual representation.

20. The computer readable medium of claim 19, wherein the method further comprises displaying an interaction between the virtual content and the virtual representation.