ASSEMBLING METHOD, OPERATING METHOD, AUGMENTED REALITY SYSTEM AND COMPUTER PROGRAM PRODUCT

• US 20130038633A1
• Filed: 04/26/2011
• Published: 02/14/2013
• Est. Priority Date: 06/10/2010
• Status: Active Grant

• Current Assignee: Sartorius Stedim Biotech GmbH (Sartorius AG)
• Original Assignee: Sartorius Stedim Biotech GmbH (Sartorius AG)
• Inventors: Maggiore, Frank
• Granted Patent: US 8,982,156 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: B01D 65/104 Detection of leaks in membrane apparatus or modules
• CPC Class Codes: B01J 19/0033 Optimalisation processes, i.e. processes with adaptive control systems adaptive control systems per se G05B13/00
• CPC Class Codes: B01L 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: C12M 41/48 Automatic or computerized control automatic analysis G01N35/00
• CPC Class Codes: G01N 15/0826 and measuring fluid flow rate, i.e. permeation rate or pressure change
• CPC Class Codes: G05B 19/41805 characterised by assembly
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G05B 2219/35506 Camera images overlayed with graphics, model
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
• CPC Class Codes: Y10T 29/49764 with testing or indicating

Abstract

An assembling method for assembling a measurement or production set-up includes providing an augmented reality system with a processing device, an output device and a sensing device. The sensing device captures sensing data belonging to a working space. The method then includes providing first and second set-up components having first and second markers at the working space where the second set-up component is connectable to the first set-up component. The method captures the first and second markers by the sensing device and identifies the first and second marker. The processing device retrieves respective digital information assigned to the identified first and second markers from a database and makes a decision on the compatibility of the first set-up component with the second set-up component based on the retrieved digital information. An augmented representation of at least part of the captured sensing data and the decision on the compatibility is output.

Claims

1. An assembling method for assembling a measurement or production set-up comprising the steps:

providing an augmented reality system with a processing device, an output device and at least one sensing device, whereby the at least one sensing device is capable of capturing sensing data belonging to a working space;

providing a first set-up component having at least one first marker at the working space;

providing a second set-up component having at least one second marker at the working space, wherein the second set-up component is connectable to the first set-up component;

capturing the first marker and the second marker by the at least one sensing device;

identifying the first and second marker, whereby the processing device retrieves respective digital information assigned to the identified first marker and second marker from a database and whereby the processing device makes a decision on the compatibility of the first set-up component with the second set-up component based on the retrieved digital information;

outputting an augmented representation of at least part of the captured sensing data and the decision on the compatibility.

2. An assembling method according to claim 1, wherein the first set-up component is an integrity testing device or a container controlling unit.

3. An assembling method according to claim 1, wherein the second set-up component is any one of an integrity testable product, a container, a disposable container, a disposable bag, a bioreactor, a disposable bioreactor, a spinner flask, a filter device, a pump, a valve, a hose, and a supply terminal.

4. An assembling method according to claim 1, comprising the step:

generating an enabling instruction in case the first set-up component is compatible to the second set-up component.

5. An assembling method according to claim 1, comprising the steps:

determining the spatial distance between the first marker and the second marker based on the sensing data captured by the at least on sensing device, wherein the processing device makes a decision on a correct connection of the first set-up component with the second set-up component based on the determined spatial distance;

output an augmented representation comprising a representation of the decision on the correct connection.

6. An assembling method according to claim 1, comprising the steps:

determining the location of a component to be assembled by an operator;

display the location of the component to the operator.

24. A computer program product for a computer-aided assembly of a measurement or production set-up and/or for automatically operating a measurement or production set-up, wherein the computer program comprises coding segments that when loaded and executed on a suitable system can execute an operating method according to claim 1.

7. An operating method for operating a measurement or production set-up comprising the steps:

providing an augmented reality system with a processing device, an output device and at least one sensing device, whereby the at least one sensing device is capable of capturing sensing data belonging to a working space;

providing a first set-up component with a first marker at the working space;

capturing the first marker by the at least one sensing device;

identifying the first marker, whereby the processing device retrieves digital information assigned to the identified first marker from a database and whereby the processing device provides a set of instructions for performing the measurement or production using the first set-up component based on the retrieved digital information;

outputting an augmented representation of at least part of the captured sensing data and at least part of the retrieved digital information;

performing the measurement or production operation according to the set of instructions.

8. An operating method according to claim 7, wherein the first set-up component is any one of an integrity testing device, a container controlling unit, a hose, a pump, a valve, a tank, a piping, an integrity testable product, a container, and an incubated container, such as a bioreactor, a disposable bioreactor, a fermentation tank, a fermentation equipment, an incubator, a medical incubator, an animal/livestock incubator, an incubated shaker, a cell/virus culture vessel, a tissue culture vessel, a disposable incubated vessel, or a disposable bag.

9. An operating method according to of claim 7, wherein the first set-up component is monitored by the at least one sensing device during the measurement or production operation, wherein the processing device generates an alert notice if the monitored data deviates from expected data beyond a predetermined variation, wherein the expected data and/or the tolerable variation is retrieved from the database.

10. An operating method according to claim 9, wherein the monitored data is recorded to a database.

11. An operating method according to claim 7, wherein the operating method comprises the steps:

providing a second set-up component connectable to the first set-up component;

capturing a second marker of the second set-up component by the at least one sensing device;

identifying the second marker, whereby the processing device retrieves digital information assigned to the identified second marker from a database and whereby the processing device makes a decision on the compatibility of the first set-up component with the second set-up component based on the retrieved digital information;

outputting an augmented representation of at least part of the captured sensing data and the decision on the compatibility;

performing the measurement or production in case the first set-up component is compatible with the second set-up component.

12. An operating method according to claim 7, wherein the second set-up component is an integrity testing device or a container controlling unit and wherein the operating method comprises the step:

sending an enabling instruction to the integrity testing device or to the container controlling unit in case the first set-up component is compatible to the second set-up component.

13. An operating method according to claim 7, wherein the operating method further comprises the steps:

determining the spatial distance between the first marker and the second marker based on the sensing data captured by the at least one sensing device, wherein the processing device makes a decision on a correct connection of the first set-up component with the second set-up component based on the determined spatial distance;

outputting an augmented representation comprising a representation of the decision on the correct connection.

14. An operating method according to claim 7, wherein the operating method further comprises the steps:

capturing a time series of images of the working space comprising at least the first set-up component;

comparing at least two images of the time series of images;

determining a change between the at least two images;

outputting a message in case a change between the at least two images occur near the first set-up component.

15. A testing method for testing the integrity of an integrity testable product comprising the steps:

providing an augmented reality system with a processing device, an output device and at least one sensing device, whereby the at least one sensing device is capable of capturing sensing data belonging to a working space;

providing an integrity testable product with at least one product marker at the working space;

providing at least one test set-up component connectable to the integrity testable product;

capturing the product marker by the at least one sensing device;

identifying the product marker, whereby the processing device retrieves digital information assigned to the identified product marker from a database and whereby the processing device makes a decision on the compatibility of the integrity testable product with the test setup component based on the retrieved digital information;

outputting an augmented representation of at least part of the captured sensing data and the decision on the compatibility;

performing the integrity test in case the integrity testable product is compatible with the test setup component.

16. A testing method according to claim 15, wherein one of the at least one test setup component is an integrity testing device and wherein the testing method further comprises the step:

sending an enabling instruction to the integrity testing device in case the at least one testing setup component is compatible to the integrity testable product.

17. Augmented reality system for operating a measurement or production set-up, the augmented reality system comprising:

at least one sensing device capable of capturing sensing data belonging to a working space;

a processing device, which is in communicatively connected to the at least one sensing device, and which is capable of

detecting the presence of a first marker of a first set-up component in the sensing data captured by the at least one sensing device,

identifying the first marker,

retrieving a digital information assigned to the identified first marker, and

making a decision on the compatibility of the first set-up component with a second set-up component based on the retrieved digital information and/or operating the measurement or production set-up according to a set of instructions contained in the retrieved digital information;

an output device for outputting an augmented representation of at least part of the captured sensing data as well as the decision on the compatibility and/or at least a part of the retrieved digital information.

18. Augmented reality system according to claim 17, wherein at least one sensing device is a camera, preferably a mobile camera.

19. Augmented reality system according to claim 17, further comprising an integrity testing device or a container controlling unit.

20. Augmented reality system according to claim 17, wherein the at least one sensing device and the output device is part of a mobile device.

21. Augmented reality system according to claim 20, wherein the processing device is part of the mobile device.

22. Augmented reality system according to claim 20, wherein the mobile device is connected to an integrity testing device or a container controlling unit of the system via a wireless connection.

23. Augmented reality system according to claim 17, wherein the output device is a projector projecting the augmented representation onto the working space or onto a set-up component, whereby the augmented representation is adjusted and displayed in accordance with the spatial distribution of the component.