Interactive and shared augmented reality system and method having local and remote access
First Claim
1. A system for remote programming of an industrial robot, the system comprising:
- a camera for capturing an image, the camera being movably located on the robot at a local site,a first registering unit configured to generate graphics and register the graphics generated by the first registering unit on the image from the camera, to provide a composite augmented reality image comprising graphical information overlaid on an image captured at the local site,a remote display device carried by or arranged on an operator located at a remote site, physically separated from the local site, for displaying a view comprising the composite augmented reality image,a tracking unit associated with the remote display device and configured to determine a remote position and an orientation of the remote display device in the remote site in relation to a fixed remote coordinate system, wherein the first registering unit is adapted to register the generated graphics and image captured at the local site to the augmented reality image in dependence on the position and orientation specified by the tracking unit, and wherein the remote display device is adapted to display the generated graphics to the augmented reality image and image captured at the local site in dependence on the position and orientation specified by the first specifying unit,a specifying unit configured to specify a position and an orientation of the robot at the local site in relation to a local coordinate system, wherein a position and orientation of the robot is dependent on the remote position and orientation specified by the tracking unit in the remote coordinate system,a second registering unit configured to generate graphics and register the generated graphics on an environment at the local site or an image of the environment of the local site, in dependence on the position and the orientation specified by the specifying unit, to provide a composite augmented reality image comprising the generated graphics overlaid on the image of the environment at the local site, anda communication link configured to communicate information between the local site and the remote site, and to communicate to the robot positions and orientations specified by the tracking unit.
2 Assignments
0 Petitions
Accused Products
Abstract
An augmented reality system including a camera movably located at a local site captures an image. A registering unit generates graphics and registers the generated graphics to the image from the camera to provide a composite augmented reality image. A display device located at a remote site, physically separated from the local site, displays a view including the composite augmented reality image. A communication link communicates information between the local and the remote site. A specifying unit specifies a position and an orientation in the remote site. The registering unit is adapted to register the generated graphical representation to the image in dependence of the specified position and orientation. The camera is arranged such that its position and orientation is dependent on the specified position and orientation.
-
Citations
30 Claims
-
1. A system for remote programming of an industrial robot, the system comprising:
-
a camera for capturing an image, the camera being movably located on the robot at a local site, a first registering unit configured to generate graphics and register the graphics generated by the first registering unit on the image from the camera, to provide a composite augmented reality image comprising graphical information overlaid on an image captured at the local site, a remote display device carried by or arranged on an operator located at a remote site, physically separated from the local site, for displaying a view comprising the composite augmented reality image, a tracking unit associated with the remote display device and configured to determine a remote position and an orientation of the remote display device in the remote site in relation to a fixed remote coordinate system, wherein the first registering unit is adapted to register the generated graphics and image captured at the local site to the augmented reality image in dependence on the position and orientation specified by the tracking unit, and wherein the remote display device is adapted to display the generated graphics to the augmented reality image and image captured at the local site in dependence on the position and orientation specified by the first specifying unit, a specifying unit configured to specify a position and an orientation of the robot at the local site in relation to a local coordinate system, wherein a position and orientation of the robot is dependent on the remote position and orientation specified by the tracking unit in the remote coordinate system, a second registering unit configured to generate graphics and register the generated graphics on an environment at the local site or an image of the environment of the local site, in dependence on the position and the orientation specified by the specifying unit, to provide a composite augmented reality image comprising the generated graphics overlaid on the image of the environment at the local site, and a communication link configured to communicate information between the local site and the remote site, and to communicate to the robot positions and orientations specified by the tracking unit. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A method for remote programming of an industrial robot arranged at a local site, the method comprising:
-
obtaining an image at the local site from a camera mounted on the robot, generating first graphics, registering the first graphics with the image at the local site, generating a composite augmented reality image with a registering unit based on the image, the generated first graphics, and the specified position and orientation, displaying the augmented reality image on a remote display device carried by or arranged on an operator at a remote site that is physically separated from the local site, specifying a remote position and an orientation at the remote site with a tracking unit carried by or arranged on an operator at the remote site, wherein the position and orientation are in relation to a remote coordinate system, specifying a local position and an orientation at the local site in relation to a local coordinate system based on the remote position and orientation specified at the remote site in the remote coordinate system, positioning and orienting the robot in the specified local position and orientation such that a camera arranged on the robot assumes the specified local position and orientation, displaying a view comprising an environment of the local site and the generated first graphics projected on the environment in dependence of the specified local position and orientation, and remotely controlling movements of the robot at the local site and remotely teaching the robot one or more waypoints at the local site to carry out a task. - View Dependent Claims (19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
-
-
30. A computer program product, comprising:
-
a computer readable medium; and computer program instructions recorded on the computer readable medium and executable by a processor for performing a method for remote programming of an industrial robot by remotely displaying an augmented reality view comprising graphical information overlaid an image captured at a local site, the method comprising obtaining an image at the local site from a camera mounted on the robot, generating first graphics, registering the first graphics with the image at the local site, generating a composite augmented reality image with a registering unit based on the image, the generated first graphics, and the specified position and orientation, displaying the augmented reality image on a remote display device carried by or arranged on an operator at a remote site that is physically separated from the local site, specifying a remote position and an orientation at the remote site with a tracking unit carried by or arranged on an operator at the remote site, wherein the position and orientation are in relation to a remote coordinate system, specifying a local position and an orientation at the local site in relation to a local coordinate system based on the remote position and orientation specified at the remote site in the remote coordinate system, positioning and orienting the robot in the specified local position and orientation such that a camera arranged on the robot assumes the specified local position and orientation, displaying a view comprising an environment of the local site and the generated first graphics projected on the environment in dependence of the specified local position and orientation, and remotely controlling movements of the robot at the local site and remotely teaching the robot one or more waypoints at the local site to carry out a task.
-
Specification