HEAD AND ARM DETECTION FOR VIRTUAL IMMERSION SYSTEMS AND METHODS

• US 20120200600A1
• Filed: 10/04/2011
• Published: 08/09/2012
• Est. Priority Date: 06/23/2010
• Status: Abandoned Application

• Current Assignee: Experience Proximity, Inc.
• Original Assignee: Wavelength & Resonance, LLC
• Inventors: DEMAINE, Kent
• Application Number:
  US13/252,949
• Publication Number:
  US 20120200600A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/203 Image generating hardware
• CPC Class Codes: A63F 2300/302 specially adapted for receiving control signals not targeted to a display device or game input means, e.g. vibrating driver's seat, scent dispenser
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Systems and methods for detection of the head and arms of a user to interact with an immersive virtual environment are disclosed. In some embodiments, a method comprises generating a virtual representation of a non-virtual environment, determining a position of a user relative to the display using an overhead sensor when the user is within a predetermined proximity to a display, determining a position of a user'"'"'s head relative to the display using the overhead sensor, and displaying the virtual representation on the display in a spatial relationship with the non-virtual environment based on the position of the user'"'"'s head relative to the display.

Claims

1. A method comprising:

generating a virtual representation of a non-virtual environment;

determining a position of a user relative to the display using an overhead sensor when the user is within a predetermined proximity to a display;

determining a position of a user'"'"'s head relative to the display using the overhead sensor; and

displaying the virtual representation on the display in a spatial relationship with the non-virtual environment based on the position of the user'"'"'s head relative to the display.

2. The method of claim 1, wherein determining the position of the user'"'"'s head relative to the display using an overhead sensor comprises detecting the user and determining a z-depth between the overhead camera and a position closest to the overhead sensor.

3. The method of claim 2, further comprising adding ½

of the length of an average user'"'"'s head to the position closest to the overhead sensor to define a head region of interest.

4. The method of claim 3, further comprising determining a position of the user head within the head region of interest.

5. The method of claim 4, adjusting the virtual representation to display an image over a position in the virtual representation that correlates with a position of the user'"'"'s head using the head region of interest.

6. The method of claim 1, further comprising detecting the user and determining a z-depth between the overhead camera and the user'"'"'s arms.

7. The method of claim 6, further comprising adding a length of an average user'"'"'s aims to the bottom of the z-depth between the overhead camera and the user'"'"'s arms to define an arms region of interest.

8. The method of claim 7, further comprising determining a position of a user'"'"'s arms and hands in the arms region of interest.

9. The method of claim 8, further comprising adjusting the virtual representation to display an image over a position in the virtual representation that correlates with a position of the user'"'"'s arms using the arms region of interest.

10. The method of claim 1, wherein the overhead sensor is a camera.

11. The method of claim 1, wherein determining the position of the user'"'"'s head comprises interpolating between data from the overhead sensor relating to a position of the user'"'"'s head and data from the first sensor relating to the position of the user'"'"'s head.

12. A system comprising:

a display configured to display a virtual representation of a non-virtual environment;

an overhead sensor that scans an area in front of the display;

one or more display sensors coupled to the display; and

a processor configured to determine a position of a user relative to the display using the overhead sensor when the user is within a predetermined proximity to a display, determining a position of a user'"'"'s head relative to the display using the overhead sensor, and displaying the virtual representation on the display in a spatial relationship with the non-virtual environment based on the position of the user'"'"'s head relative to the display.

13. The system of claim 12, wherein the processor configured to determine the position of the user'"'"'s head relative to the display using the overhead sensor comprises the processor configured to detect the user and determining a z-depth between the overhead camera and a position closest to the overhead sensor.

14. The system of claim 13, wherein the processor is further configured to add ½

of the length of an average user'"'"'s head to the position closest to the overhead sensor to define a head region of interest.

15. The system of claim 14, wherein the processor is further configured to further comprising determining a position of the user head within the head region of interest.

16. The system of claim 15, wherein the processor is further configured to adjust the virtual representation to display an image over a position in the virtual representation that correlates with a position of the user'"'"'s head using the head region of interest.

17. The system of claim 12, wherein the processor is further configured to detect the user and determining a z-depth between the overhead camera and the user'"'"'s arms.

18. The system of claim 17, further comprising adding a length of an average user'"'"'s arms to the bottom of the z-depth between the overhead camera and the user'"'"'s arms to define an arms region of interest.

19. The system of claim 18, wherein the processor is further configured to determine a position of a user'"'"'s arms and hands in the arms region of interest.

20. The system of claim 19, wherein the processor is further configured to adjust the virtual representation to display an image over a position in the virtual representation that correlates with a position of the user'"'"'s arms using the arms region of interest.

21. The system of claim 12, wherein the overhead sensor is a camera.

22. The system of claim 12, wherein the processor configured to determine the position of the user'"'"'s head comprises the processor configured to interpolate between data from the overhead sensor relating to a position of the user'"'"'s head and data from the first sensor relating to the position of the user'"'"'s head.

23. A computer readable medium configured to store executable instructions, the instructions being executable by a processor to perform a method, the method comprising:

generating a virtual representation of a non-virtual environment;

determining a position of a user relative to the display using an overhead sensor when the user is within a predetermined proximity to a display;

determining a position of a user'"'"'s head relative to the display using the overhead sensor; and

displaying the virtual representation on the display in a spatial relationship with the non-virtual environment based on the position of the user'"'"'s head relative to the display.