Augmented-reality system for situation-related support of the interaction between a user and an engineering apparatus

• US 7,324,081 B2
• Filed: 09/04/2001
• Issued: 01/29/2008
• Est. Priority Date: 03/02/1999
• Status: Expired due to Term

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Friedrich, Wolfgang, Wohlgemuth, Wolfgang
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Shapiro; Leonid
• Application Number:
  US09/945,773
• Publication Number:
  US 20020044104A1
• Time in Patent Office: 2,338 Days
• Field of Search: 345/156, 345/700, 345/7, 345/8, 359/13, 359/630, 700/83, 700/100, 700/111
• US Class Current: 345/156
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G05B 2219/37567 3-D vision, stereo vision, with two cameras

Abstract

The invention relates to an augmented-reality system having a mobile apparatus for the context-dependent insertion of assembly instructions. The context-dependent insertion of assembly instructions with process-optimized allocation of the necessary work steps provides situation-related support by for work sequences.

Claims

1. A system for real-time situation related support of the interaction between a user and various different apparatuses, said system comprising:

a mobile device to be worn by the user, said mobile device comprising a camera for collecting visual data including dynamic data, a screen for displaying collected and received data, a microphone for sending auditory data including non-visual dynamic data observed by the user and a transceiver unit for the wireless transmission and receipt of data relating to various different apparatuses,

a remotely located augmented-reality database system, said database system comprising (i) an information module for storing visual and textual data relative to the various different apparatuses, (ii) a communication module including a transceiver unit for the wireless transmission and receipt of apparatus data to and from the mobile device, said information module routinely updated with additional apparatus data not received from the mobile device, said database system via the communications module capable of connecting to the internet, and communicating with a remote PC to obtain additional data and to process data, and (iii) an analyzer system comparing the collected visual data with data stored in the database system to determine additional information, wherein the additional information is visually overlayed on apparatus images transmitted from the mobile user and a created visually overlaid image is transmitted back to the user for display on the screen.