User interface for augmented and virtual reality systems

• US 7,190,378 B2
• Filed: 08/16/2002
• Issued: 03/13/2007
• Est. Priority Date: 08/16/2001
• Status: Expired due to Term

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Sauer, Frank, Khamene, Ali, Schimmang, Lars
• Primary Examiner(s): Sajous; Wesner
• Application Number:
  US10/222,284
• Publication Number:
  US 20030063132A1
• Time in Patent Office: 1,670 Days
• Field of Search: 345/633, 345/418, 345619-621, 345/632, 345/638, 345771-773, 345810-811, 345835-836, 345856-860, 345862-863, 345156-158, 345 7- 8, 345169-173, 715856-860, 715862-863, 715810-811, 715835-836, 715/800, 715771-773
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A user interface for augmented reality (AR) systems and virtual reality (VR) systems which include a computer and virtual graphics and in which a user experiences a virtual viewing direction, includes: a virtual line pointer included in the virtual graphics and fixed with respect to the virtual viewing direction; a representation of the virtual line pointer to the user; and a user-operable device coupled to the computer for optionally marking an object intersecting the virtual line pointer.

Claims

1. A user interface for augmented reality (AR) systems and virtual reality (VR) systems which includes a computer and virtual graphics containing virtual objects positioned in three dimensional (3D) space and in which a user experiences a view of a 3D virtual or augmented scene which includes the virtual objects, and determines a viewpoint with the position of his head and a viewing direction with the orientation of his head, said interface comprising:

a graphical representation of said virtual line pointer to said user, the virtual line pointer extending from the user'"'"'s viewpoint into a 3D virtual scene oriented along a user'"'"'s line of sight and fixed to the user'"'"'s head, the virtual line pointer controlled with regard to position and orientation by the user'"'"'s head movements;

an intersection processor for calculating in the 3D virtual scene an intersection point of the virtual line pointer and a virtual object'"'"'s surface, the intersection point accordingly being selected by the user'"'"'s head movement and orientation; and

a user operable device for triggering manipulation of the virtual scene at the selected intersection point.

2. A user interface as recited in claim 1, wherein said representation of said virtual line pointer is a search-light-like ray or beam.

3. A user interface as recited in claim 1, wherein said representation of said virtual line pointer is a mark at a perceived distance along the line of said virtual line pointer.

4. A user interface as recited in claim 3, wherein said perceived distance is variable so as to make said mark appear at a surface of said object intersecting said virtual line pointer.

5. A user interface as recited in claim 1, wherein said representation of said virtual line pointer is a mark positioned in the line of said virtual viewing direction.

6. A user interface as recited in claim 1, wherein said user-operable device is a switch.

7. A user interface as recited in claim 1, wherein said user-operable device is a speech recognition device.

8. A user interface as recited in claim 1, wherein said user-operable device is a virtual button included in said virtual graphics.

9. A user interface as recited in claim 8, wherein said virtual line pointer exhibits a tip at a perceived fixed length from said user'"'"'s head and wherein said user-operable device is a virtual button included in said virtual graphics, said virtual button being triggered when said tip is placed thereupon and said user'"'"'s head is moved forward such that said tip appears to move forward and press said virtual button.

10. A user interface as recited in claim 1, where said intersection is calculated as a 3D point.

11. A user interface as recited in claim 1, wherein the user-operable device confirms said intersection.

12. A user interface as recited in claim 11, wherein said confirming of the intersection selects the intersected said virtual object, for potential further manipulation.

13. A user interlace as recited in claim 11, where said confirming of the intersection places a graphical marker at the location of said intersection.

14. A user interface as recited in claim 8, wherein said virtual line pointer exhibits a tip at a perceived fixed length from said user'"'"'s head and wherein said user-operable device is a virtual button included in said virtual graphics, said virtual button being triggered when said tip is placed thereupon and some pre-defined small head-movement is executed.

15. A user interface according to claim 1 wherein a manipulation performed by a user operable device comprises the steps of:

selecting a particular intersection point by recording 3D coordinates of the selected point;

placing markers at these coordinates in 3D space;

fixing the selected point in the virtual scene'"'"'s world coordinate system; and

having the selected point attach to the object so it moves along with the object.

16. A user interface according to claim 1 wherein a manipulation performed by a user operable device comprises the steps of:

selecting a complete object containing a selected intersection point; and

changing the appearance of the whole object.

17. The method of claim 16 wherein the step of changing the appearance of the whole object comprises the step of:

deleting the whole object.

18. The method of claim 16 wherein the step of changing the appearance of the whole object comprises the step of:

highlighting the whole object.

19. The method of claim 16 wherein the step of changing the appearance of the whole object comprises the step of:

moving the whole object in drag and drop fashion.