System and method for communication between a mobile data processing device and a stationary data processing device

• US 8,612,511 B2
• Filed: 08/09/2001
• Issued: 12/17/2013
• Est. Priority Date: 08/22/2000
• Status: Active Grant

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Friedrich, Wolfgang, Hamadou, Mehdi, Jahn, Dirk, Kiesel, Bruno
• Primary Examiner(s): CHRISTENSEN, SCOTT B
• Application Number:
  US10/362,624
• Publication Number:
  US 20040105427A1
• Time in Patent Office: 4,513 Days
• Field of Search: 709/201, 709/203, 709/213, 709/217, 709/223, 709/224
• US Class Current: 709/203
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/31162 Wireless lan
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G05B 2219/32126 Hyperlink, access to program modules and to hardware modules in www, web server, browser
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability

Abstract

A stationary data processing device communicates with a mobile data processing device, such as a client in the field of augmented-reality techniques in an automation system, or a production machine, or a machine tool or parts thereof. The stationary data processing device includes a server for communication with a browser on the mobile data processing device; an information broker for providing context-related information; legacy systems that contain and store information; and an information model that represents the information space and that allows navigation within the information and access to the information by the information broker independent of the legacy systems. The information broker utilizes the actual context, especially location, work situation, EA devices used and the information model.

Claims

1. A stationary data processing device to communicate with a mobile data processing device, comprising:

at least one computer including;

a server wirelessly communicating with the mobile data processing device operating as a client, an empty Internet browser being installed on the mobile data processing device to selectively load or reload a standard plug-in and a special plug-in in correspondence with a specific hardware profile of the mobile data processing to implement an augmented reality functionality;

legacy systems, including at least one of a Plant Maintenance Management System and a Product Data Management System, that contain and store information; and

an information broker providing information corresponding to a current context of a physical object under consideration, location, work situation, input and output units used, and

where the information broker provides an information model providing a representation of an information space and navigates within the information of the current context suited to the client independent of access to the legacy systems, and the client accesses data maintained in said legacy systems using the information broker, and

the information broker augments the information space in correspondence with a specific hardware profile of any of user interaction units to provide the augmented reality functionality in at least one of an industrial automation system, a production machine, machine tool, and parts thereof, and

the information space being changed to meet the specific hardware profile of the mobile data processing device based on the current context by loading or reloading one of the standard plug-in and the special plug-in to said empty browser of the mobile data processing device.

2. The stationary data processing device as claimed in claim 1, wherein the information model is a representation of the information space and the information model is enhanced by supplementary meta-information while information of the legacy systems is maintained.

3. The stationary data processing device as claimed in claim 1, wherein a function of the augmented reality technologies causes a plug-in to be automatically reloaded in accordance with information of the current context provided by the information broker.

4. A method of communication between a mobile data processing device and a stationary data processing device, comprising:

providing a server that wirelessly communicates with the mobile data processing device operating as a client, an empty Internet browser being installed on the mobile data processing device to selectively load or reload a standard plug-in and a special plug-in in correspondence with a specific hardware profile of the mobile data processing to implement an augmented reality functionality;

providing information by an information broker, based on a current context of a physical object under consideration, location, work situation, input and output units used; and

providing, via the information broker, an information model providing a representation of an information space and enabling the information broker to navigate within information of the current context independent of the legacy systems, and

displaying, in the mobile data processing device acting as a client and using the information provided by the information broker, and

where the legacy systems includes at least one of a Plant Maintenance Management System and a Product Data Management System; and

the information broker augments the information space in correspondence with a specific hardware profile of any of user interaction units to provide the augmented reality functionality in at least one of an industrial automation system, a production machine, machine tool, and parts thereof, and

the information space being changed to meet the specific hardware profile of the mobile data processing device based on the current context by loading or reloading one of the standard plug-in and the special plug-in to said empty browser of the mobile data processing device.

5. The method as claimed in claim 4, wherein the input and output units of the stationary data processing device are adapted for information for user interactions having a nonrepresentative structure, and

wherein said method further comprises mapping the user interactions onto a specific hardware profile based on the current context.

6. The method as claimed in claim 5, further comprising describing nonrepresentative information using XML.

7. The method as claimed in claim 6, further comprising:

presenting augmented information by a component augmented reality plugin in the client, and

controlling said presenting using voice interactions, and

wherein said communicating includes communication of the augmented information with reloadable components required for said controlling of said presenting.

8. The method as claimed in claim 4,

wherein the information model is a representation of the information space, and

wherein said method further comprises enhancing the information model by supplementary information.

9. The method as claimed in claim 8,

wherein said communicating between the browser and the server takes place wirelessly using different technologies respectively matched to an area of use, including at least one of Bluetooth for a local area, a wireless LAN, and for a remote area, at least one of GSM, GPRS, and UMTS, and wherein information to be transmitted to the browser is determined based on available bandwidth by the augmented reality technologies.

10. A stationary data processing device to communicate with a mobile data processing device, comprising:

at least one computer including;

a server wirelessly communicating with the mobile data processing device operating as a client, an empty Internet browser being installed on the mobile data processing device to selectively load or reload a standard plug-in and a special plug-in in correspondence with a specific hardware profile of the mobile data processing to implement an augmented reality functionality;

legacy systems, including at least one of a Plant Maintenance Management System and a Product Data Management System, that contain and store information; and

an information broker providing information corresponding to a current context of a physical object under consideration, location, work situation, input and output units used, and

where the information broker provides an information model providing a representation of an information space and navigates within the information of the current context suited to the client independently of and without having to access said legacy systems, and the client accesses data maintained in said legacy systems using the information broker, and

the information broker augments and the information space in correspondence with a specific hardware profile of any of user interaction units to provide the augmented reality functionality in at least one of an industrial automation system, a production machine, machine tool, and parts thereof, and

the information space being changed to meet the specific hardware profile of the mobile data processing device based on the current context.

11. The stationary data processing device as claimed in claim 10, wherein the input and output units are adapted and said information broker provides a nonrepresentative structure for information for user interactions that are current-context-dependent mapped onto a specific hardware profile.

12. The stationary data processing device as claimed in either of claim 11, wherein nonrepresentative information may be described using XML.

13. The stationary data processing device as claimed in claim 12, wherein the client features a component augmented reality plugin for presenting the augmented information and communicates using reloadable components required for controlling presentation using voice interactions.

14. The stationary data processing device as claimed in claim 13, wherein the information model is an information space representation enhanced by supplementary information.

15. The stationary data processing device as claimed in claim 14,

wherein said server includes communication interfaces for wireless communication with the browser matched to an area of use, using at least one of Bluetooth for a local area, a wireless LAN, and, for a remote area, GSM, GPRS, and UMTS, and

wherein said server determines information to be transmitted to the client for the augmented reality technologies based on available bandwidth.