User-controlled linkage of information within an augmented reality system

• US 20040046711A1
• Filed: 06/18/2003
• Published: 03/11/2004
• Est. Priority Date: 12/18/2000
• Status: Abandoned Application

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Triebfuerst, Gunthard
• Application Number:
  US10/463,695
• Publication Number:
  US 20040046711A1
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G05B 2219/35482 Eyephone, head-mounted 2-D or 3-D display, also voice and other control
• CPC Class Codes: G05B 2219/35487 Display and voice output incorporated in safety helmet of operator
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A system and a method for use, in particular, in an augmented reality environment, which improve the representation of information in terms of its user friendliness. The system includes a display unit (2) displaying information (3), an image detection unit (7) detecting objects (9) in a field of vision (8) of a user (1), a command detection unit (10) detecting the commands (4) given by the user (1), and a control unit (11) that controls the display unit (2), recognizes the objects (9) detected by the image detection unit (7) and processes the commands (4) of the user (1) detected by the command detection unit (10). The system additionally establishes a linkage between the displayed information (3) and the contemporaneously detected objects (9), wherein the linkage is controlled by the commands (4) given by the user (1).

Claims

1. System comprising:

a display unit displaying information,

an image detection unit detecting objects in a field of vision of a user,

a command detection unit detecting commands given by the user, and

a control unit controlling the display unit, recognizing the objects detected by the image detection unit, and processing the commands of the user detected by the command detection unit,

wherein the control unit further provides a linkage, controlled by the commands given by the user, between the displayed information and the contemporaneously detected objects.

2. The system as claimed in claim 1, wherein:

the control unit reversibly interrupts the linkage between the display information and the contemporaneously detected objects in accordance with the commands of the user, whereby the display unit displays the information independently of the contemporaneously detected objects.

3. The system as claimed in claim 1, wherein the command detection unit detects voice commands of the user.

4. The system as claimed in claim 1, further comprising feedback devices transmitting feedback to the user, wherein the control unit generates the feedback.

5. The system as claimed in claim 4, wherein the feedback comprises acoustic feedback.

6. The system as claimed in claim 1, wherein the objects are provided with at least one marker, enabling the control unit to recognize the objects detected by the image detection unit, and wherein the information displayed is associated with the at least one marker.

7. The system as claimed in claim 1, wherein the objects are provided respectively with at least one marker, causing the control unit to recognize the objects detected by the image detection unit, and wherein the respective items of information are associated with the respective markers.

8. The system as claimed in claim 1, wherein the display unit is a head-mounted display that superimposes the information on the field of vision of the user.

9. The system as claimed in claim 1, in an augmented reality environment, wherein the control of the linkage causes an object-independent display of the information on the display unit of a previously object-dependent display of the information on the display unit.

10. The system as claimed in claim 9, wherein commands of the user initiate and terminate the object-independent representation.

11. A method comprising:

displaying information,

detecting objects in a field of vision of a user,

detecting commands given by the user,

recognizing the detected objects,

processing the detected commands of the user, and

controlling a linkage between the displayed information and the detected objects in accordance with the commands given by the user.

12. The method as claimed in claim 11, further comprising:

reversibly interrupting the linkage between the display information and the detected objects in accordance with the commands of the user, wherein the information is displayed independently of the detected objects.

13. The method as claimed in claim 11, wherein the commands given by the user comprise voice commands.

14. The method as claimed in claim 11, further comprising:

generating feedback and transmitting the feedback to the user.

15. The method as claimed in claim 14, wherein the feedback comprises acoustic feedback.

16. The method as claimed in claim 11, further comprising:

providing the objects with at least one marker,

recognizing the detected objects, and

associating the information displayed with the at least one marker.

17. The method as claimed in claim 11, further comprising:

providing the objects each with at least one marker,

recognizing the detected objects, and

associating the respective items of information with the respective markers.

18. The method as claimed in claim 11, wherein the information is superimposed on the field of vision of the user via a head-mounted display.

19. The method as claimed in claim 11, in an augmented reality environment, wherein the controlling of the linkage comprises displaying an object-independent display of the information of a previously object-dependent display of the information.

20. The method as claimed in claim 19, wherein commands of the user initiate and terminate the object-independent representation.

21. Computer program product for programming a control unit in a system comprising:

a display unit displaying information,

an image detection unit detecting objects in a field of vision of a user,

a command detection unit detecting commands given by the user, and

a control unit controlling the display unit, recognizing the objects detected by the image detection unit, and processing the commands of the user detected by the command detection unit,

wherein the control unit further provides a linkage, controlled by the commands given by the user, between the displayed information and the contemporaneously detected objects.

22. A system comprising:

a display means for displaying information,

an image detection means for detecting objects in a field of vision of a user,

a command detection means for detecting commands given by the user, and

a control means for controlling the display unit, recognizing the objects detected by the image detection unit, and processing the commands of the user detected by the command detection unit,

wherein the control means further provides a linkage, controlled by the commands given by the user, between the displayed information and the contemporaneously detected objects.

23. A component of an augmented reality system, comprising:

an object recognition device configured to associate a plurality of predetermined objects with respective sets of information,

a visual display unit configured to display the respective sets information in accordance with associations by the object recognition device,

a processor configured to control the visual display unit to operate selectively in an object-dependent mode and an object-independent mode, and

a user interface configured to receive a signal for the processor indicative of a user'"'"'s selection of one of the object-dependent mode and the object-independent mode.