Method and system to present immersion virtual simulations using three-dimensional measurement

• US 20020140633A1
• Filed: 02/05/2001
• Published: 10/03/2002
• Est. Priority Date: 02/03/2000
• Status: Abandoned Application

• Current Assignee: Canesta Incorporated (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Bamji, Cyrus, Rafii, Abbas, Sze, Cheng-Feng
• Application Number:
  US09/777,778
• Publication Number:
  US 20020140633A1
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: B60K 2360/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: G01S 17/48 Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

A virtual simulation system generates an image of a virtual control on a display that may be a heads-up-display in a vehicle. The system uses three-dimensional range finding data to determine when a user is sufficiently close to the virtual control to “manipulate” the virtual control. The user “manipulation” is sensed non-haptically by the system, which causes the displayed control image to move in response to user manipulation. System output is coupled, linearly or otherwise, to an actual device having a parameter that is adjusted substantially in real-time by user-manipulation of the virtual image. System generated displays can be dynamic and change appearance when a user'"'"'s hand is in close proximity. displays can disappear until needed, or can include menus and icons to be selected by the user who points towards or touches the virtual images. System generate images can include representation of the user for use in a training or gaming system.

Claims

1. A method of presenting a virtual simulation to control an actual device, the method comprising the following steps:

(a) generating a display including an image of a control to change a parameter of said device,

(b) sensing (x,y,z) axes proximity of a user to said image on said display;

(c) determining non-haptically from data sensed at step (b), user intended movement of said image of said control; and

(d) outputting a signal coupleable to said actual device to control said parameter as a function of sensed user intended movement of said image of said control.

2. The method of claim 1, wherein at step (a), said display is a heads-up-display.

3. The method of claim 1, wherein step (b) includes sensing using time-of-flight data.

4. The method of claim 1, wherein step (c) includes modifying said display to represent movement of said control created by said user.

5. The method of claim 1, wherein step (a) includes generating an image of a slider control.

6. The method of claim 1, wherein step (a) includes generating an image of a rotary control.

7. The method of claim 1, wherein step (a) includes generating an image including a menu of icons selectable by said user.

8. The method of claim 1, wherein said actual device is selected from a group consisting of (i) an electronic entertainment device, (ii) radio, (iii) a cellular telephone, (iv) a heater system, (v) a cooling system, (vi) a motorized system.

9. The method of claim 1, wherein at step (a) said display is generated only after detection of a user in close proximity to an area whereon said display is presentable.

10. The method of claim 9, further including displaying a user-alert warning responsive to a parameter of said device, independently of user proximity to said area.

11. The method of claim 1, wherein said display is a heads-up-display in a motor vehicle operable by a user, and said device is selected from a group consisting of (i) said motor vehicle, and (ii) an electronic accessory disposed in said motor vehicle.

12. The method of claim 11, wherein said device is a global position satellite system, said display includes a map, and said control is user-operable to change displayed appearance of said map.

14. The method of claim 13, wherein said manipulating is part of a regime to train said user to manipulate a real object represented by said virtual image.

16. The system of claim 15, wherein said image is a representation of a control, said object is a portion of a user'"'"'s hand, and said proximity includes user manipulation of said image;

further including;

a system outputting a signal coupleable to a real device having a parameter variable in response to said user manipulation of said image.

17. The system of claim 15, wherein:

said system is a heads-up-system;

said display is presentable on a windshield of a motor vehicle; and

said image includes an image of a control.

18. The system of claim 17, wherein:

said system includes a circuit outputting a command signal responsive to said detection of said proximity, said command signal coupleable to a device selected from a group consisting of (a) an electrically-controllable component of said motor vehicle, (b) an electrically-controllable electronic device disposed in said motor vehicle.

19. The system of claim 18, wherein said device is a global positioning satellite (GPS) system, wherein said image is a map generated by said GPS system, and said image is a control to change appearance of said image of said map.

20. The system of claim 17, wherein said detection sub-system operates independently of ambient light.

13. A method of presenting a virtual simulation, the method comprising the following steps:

(a) generating a display including a virtual image of an object;

(b) non-haptically sensing in three-dimensions proximity of at least a portion of a user'"'"'s body to said display;

(c) modifying said display substantially in real-time to include a representation of said user'"'"'s body; and

(d) modifying said display to depict substantially in real-time said representation of said user'"'"'s body manipulating said object.

15. A virtual simulation system, comprising:

an imaging sub-system to generate a display including an image;

a detection sub-system to non-haptically detect in three-dimensions proximity of a portion of an object to a region of said display; and

said imaging sub-system modifying said image in response to detected proximity of said portion of said object.