Multi-Sensor Proximity-Based Immersion System and Method

• US 20120188279A1
• Filed: 08/10/2011
• Published: 07/26/2012
• Est. Priority Date: 09/29/2009
• Status: Abandoned Application

• Current Assignee: Experience Proximity, Inc.
• Original Assignee: Wavelength & Resonance, LLC
• Inventors: Demaine, Kent
• Application Number:
  US13/207,312
• Publication Number:
  US 20120188279A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/203 Image generating hardware
• CPC Class Codes: A63F 2300/302 specially adapted for receiving control signals not targeted to a display device or game input means, e.g. vibrating driver's seat, scent dispenser
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6653 for altering the visibility of an object, e.g. preventing the occlusion of an object, partially hiding an object
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Systems and methods for interaction with a virtual environment are disclosed. In some embodiments, a method comprises generating a virtual representation of a user'"'"'s non-virtual environment, determining a viewpoint of a user in a non-virtual environment relative to a display, and displaying, with the display, the virtual representation in a spatial relationship with the user'"'"'s non-virtual environment based on the viewpoint of the user.

Claims

1. A method comprising:

generating a virtual representation of a non-virtual environment;

determining a position of a user relative to the display using a first sensor when the user is within a first predetermined proximity to a display;

determining a viewpoint of the user relative to the display using a second sensor when the user is within a second predetermined proximity to the display; and

displaying the virtual representation on the display in a spatial relationship with the non-virtual environment based on the viewpoint of the user relative to the display, and the position of the user relative to the display.

2. The method of claim 1, wherein displaying the virtual representation on the display further comprises increasing or decreasing a degree of detail, size, light, or texture of the virtual representation based on the position of the user relative to the display.

3. The method of claim 1, further comprising when the user is within a third predetermined proximity to the display, determining a position of an element of the user'"'"'s body.

4. The method of claim 3, wherein determining the position of the user relative to the display using the first sensor comprises interpolating between data from the first sensor relating to a position of the user'"'"'s body and data from the first sensor relating to the position of the element of the user'"'"'s body.

5. The method of claim 1, further comprising when the user is within a fourth predetermined proximity to the display:

determining a head position of the user relative to the display using the first sensor;

determining the head position of the user relative to the display using the second sensor; and

interpolating between head position data from the first sensor and head position data from the second sensor to determine the viewpoint of the user relative to the display.

6. The method of claim 1, wherein each predetermined proximity is defined by a tracking zone monitored by the first sensor or the second sensor, each tracking zone being an area surrounding the display that is defined relative to the display.

7. The method of claim 6, further comprising adjusting the dimension or the location of the area of a tracking zone based on the position of the display relative to the non-virtual environment, on an orientation of the display relative to the non-virtual environment, a time of day where the display is located, or a date where the display is located.

8. The method of claim 1, further comprising determining a rate of approach by the user toward the display based on the position of the user relative to the display over a range of time, and wherein displaying the virtual representation on the display further comprises displaying the virtual representation based on the rate of approach.

9. The method of claim 1, further comprising displaying virtual content in the virtual representation.

10. The method of claim 9, wherein displaying the virtual content in the virtual representation comprises displaying the virtual content within the virtual representation based on the position of the display relative to the non-virtual environment, the viewpoint of the user relative to the display, or the position of the user relative to the display.

11. The method of claim 10, wherein displaying the virtual content in the virtual representation based on the position of the user comprises increasing or decreasing a degree of detail, size, light, or texture of the virtual content based on the position of the user.

12. The method of claim 1, further comprising selecting the user from a plurality of users viewing the display based on the position of the user in comparison to a position of another in the plurality of users relative to the display.

13. The method of claim 1, further comprising determining a position or a orientation of a display relative to the non-virtual environment, wherein displaying the virtual representation on the display in the spatial relationship with the non-virtual environment is further based on the position or the orientation of the display relative to the non-virtual environment.

14. A system, comprising:

a first sensor;

a second sensor;

a virtual representation module configured to generate a virtual representation of a non-virtual environment;

a user position module configured to determine a position of a user relative to a display using the first sensor when the user is within a first predetermined proximity to the display;

a viewpoint module configured to determine a viewpoint of the user relative to the display using the second sensor when the user is within a second predetermined proximity to the display; and

the display configured to display the virtual representation in a spatial relationship with the non-virtual environment based on the viewpoint of the user relative to the display, and the position of the user relative to the display.

15. The system of claim 14, wherein the display is not transparent.

16. The system of claim 14, wherein the display is further configured to display the virtual representation on the display by increasing or decreasing a degree of detail, size, light, or texture of the virtual representation based on the position of the user relative to the display.

17. The system of claim 14, wherein the user position module is further configured to determine a position of an element of the user'"'"'s body when the user is within a third predetermined proximity to the display.

18. The system of claim 17, wherein the user position module is further configured to determine the position of the user relative to the display using the first sensor by interpolating data from the first sensor relating to a position of the user'"'"'s body and data from the first sensor relating to the position of the element of the user'"'"'s body.

19. The system of claim 14, wherein the viewpoint module is further configured such that when the user is within a fourth predetermined proximity to the display, the viewpoint of the user relative to the display is determined by:

determining a head position of the user relative to the display using the first sensor;

determining the head position of the user relative to the display using the second sensor; and

interpolating between head position data from the first sensor and head position data from the second sensor to determine the viewpoint of the user relative to the display.

20. The system of claim 14, wherein each predetermined proximity is defined by a tracking zone monitored by the first sensor or the second sensor, each tracking zone being an area surrounding the display that is defined relative to the display.

21. The system of claim 20, further comprising adjusting the dimension or the location of the area of a tracking zone based on the position of the display relative to the non-virtual environment, on an orientation of the display relative to the non-virtual environment, a time of day where the display is located, or a date where the display is located.

22. The system of claim 14, wherein the user position module is further configured to determine a rate of approach by the user toward the display based on the position of the user relative to the display over a range of time, and wherein the display is further configured to display the virtual representation based on the rate of approach.

23. The system of claim 14, wherein the display is further configured to display virtual content in the virtual representation.

24. The system of claim 23, wherein the display is further configured to display the virtual content within the virtual representation based on the position of the display relative to the non-virtual environment, the viewpoint of the user relative to the display, or the position of the user relative to the display.

25. The system of claim 24, wherein the display is further configured to increase or decrease a degree of detail, size, light, or texture of the virtual content based on the position of the user.

26. The system of claim 14, further comprising a selection module configured to select the user from a plurality of users viewing the display based on the position of the user in comparison to a position of another in the plurality of users relative to the display.

27. The system of claim 14, further comprising a display position module configured to determine a position of the display relative to the non-virtual environment, wherein the display is further configured to display the virtual representation in the spatial relationship with the non-virtual environment further based on the position of the display relative to the non-virtual environment.

28. The system of claim 14, further comprising a display orientation module configured to determine a position of the display relative to the non-virtual environment, wherein the display is further configured to display the virtual representation in the spatial relationship with the non-virtual environment further based on the orientation of the display relative to the non-virtual environment.

29. A computer readable medium configured to store executable instructions, the instructions being executable by a processor to perform a method, the method comprising:

generating a virtual representation of a non-virtual environment;

determining a position of a display relative to the non-virtual environment;

determining a position of a user relative to the display using a first sensor when the user is within a first predetermined proximity to the display;

determining a viewpoint of the user relative to the display using a second sensor when the user is within a second predetermined proximity to the display; and

displaying the virtual representation on the display in a spatial relationship with the non-virtual environment based on the position of the display relative to the non-virtual environment, the viewpoint of the user relative to the display, and the position of the user relative to the display.