Method and device for visualizing computer-generated informations

• US 20040189631A1
• Filed: 02/10/2004
• Published: 09/30/2004
• Est. Priority Date: 02/11/2003
• Status: Active Grant

• Current Assignee: KUKA Laboratories GmbH (Midea Group Co., Ltd)
• Original Assignee: KUKA Roboter GmbH (Midea Group Co., Ltd)
• Inventors: Kazi, Arif, Bischoff, Rainer, Kuepeli, Bahadir
• Granted Patent: US 7,298,385 B2
• Time in Patent Office: Days
• US Class Current: 345/418
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: G05B 2219/35422 Unit freely movable in space, detect its position, orientation by triangulation
• CPC Class Codes: G05B 2219/35506 Camera images overlayed with graphics, model
• CPC Class Codes: G05B 2219/36167 Use camera of handheld device, pda, pendant, head mounted display
• CPC Class Codes: G05B 2219/39443 Portable, adapted to handpalm, with joystick, function keys, display
• CPC Class Codes: G05B 2219/39449 Pendant, pda displaying camera images overlayed with graphics, augmented reality
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

To improve the operation of robots in installations reprogramming and emptying an operating sequence to be performed by them, the invention provides a method and a device for visualizing computer-assisted information in an image of the real environment on a viewing device, in which there is a determination of the position and orientation or pose of the image receiving device and that robot-specific informations corresponding to this determination are faded over the image of the real environment on the viewing device.

Claims

1. Method for fading computer-generated informations into an image of the real environment detected by an image receiving unit located on a viewing device, wherein there is a determination of a position and an orientation or pose of the image receiving unit and that robot-specific information corresponding to this determination is faded over the image of the real environment on the viewing device.

2. Method according to claim 1, wherein at least one robot-specific coordinate system is faded in.

3. Method according to claim 1, wherein a hand flange-fixed coordinate system is faded in.

4. Method according to claim 1, wherein robot axes are faded in.

5. Method according to claim 1, wherein an image of a control element of a robot manual programmer movable in at least two dimensions is faded in.

6. Method according to claim 5, wherein an image of the control unit in association and orientation of the robot hand flange is faded in.

7. Method according to claim 1, wherein at least one tool moved by a robot, preferably several robot elements are faded into a working environment of a robot.

8. Method according to claim 1, wherein a robot path to be covered by a robot in connection with a working process, particularly that of the tool centre point, is faded in, optionally including additional information.

9. Method according to claim 1, wherein path points are faded in, wherein the position of the path points in space is variable and in each case a robot path curve corresponding to the position of all the path points is faded in.

10. Method according to claim 1, wherein the image of a workpiece to be machined by a robot, optionally together with axes associated therewith is faded in.

11. Method according to claim 1, wherein for adapting a robot path to the position of a detected, real workpiece, a virtual image of the workpiece with a robot path adapted thereto is faded in, so that by superimposing the virtual workpiece image with the image of the real workpiece it is possible to adapt the robot path to be performed to the position of the real workpiece.

12. Method according to claim 1, wherein a working area reachable by a robot and/or a permitted operating area is visualized on the viewing device.

13. Method according to claim 1, wherein movement corridors of a robot tool, robot hand and/or further robot elements are visualized on the viewing device.

14. Method according to claim 1, wherein permanent and/or instantaneous associations of at least one manual programmer of at least one robot are visualized.

15. Method according to claim 1, wherein the position and orientation of the display are detected by fixed markings in space.

16. Method according to claim 15, wherein the position and orientation of the viewing device are determined by radio.

17. Method according to claim 1, wherein the position and orientation of the viewing device are determined optically.

18. Method according to claim 1, wherein the robot-specific, computer-generated information is faded into data spectacles to be worn by a user.

19. Method according to claim 1, wherein the robot-specific, computer-generated information is faded onto a transparent screen as the viewing device.

20. Method according to claim 1, wherein the robot-specific, computer-generated information is displayed together with optoelectronically recorded information on a graphicable screen as the viewing device.

21. Method according to claim 1, wherein the robot-specific, computer-generated information is displayed together with optoelectronically recorded information on a screen in the form of a graphicable screen of a manual programmer for a robot.

22. Method according to claim 1, wherein the robot-specific, computer-generated information is displayed together with optoelectronically recorded information on a graphicable screen as the viewing device connectable to a manual programmer for a robot.

23. Device for visualizing computer-generated informations in an image of the real environment having an image receiving device and a viewing device, characterized by a means for determining the position and orientation or pose of the image receiving device and by a means for fading the determination of corresponding robot-specific information over the image of the real environment on the viewing device.

24. Device according to claim 23, comprising a display of at least one robot-specific coordinate system.

25. Device according to claim 23, comprising a display of a hand flange-fixed coordinate system.

26. Device according to claim 23, comprising a display of robot axes.

27. Device according to claim 23, comprising a display of the image of an operating unit of a robot manual programmer movable in at least two dimensions.

28. Device according to claim 27, comprising a display of an image of the operating unit in association and orientation of the robot hand flange.

29. Device according to claim 23, comprising a display of at least one tool moved by a robot and preferably several robot elements, in an operating environment of a robot.

30. Device according to claim 23, comprising a display of a robot path to be covered during an operating process and optionally with additional information for a robot, particularly the hand flange of a robot.

31. Device according to claim 23, comprising a display of path points with respect to a variability of the position of the path points in space and in each case a display of a position of a robot path curve corresponding to all the path points.

32. Device according to claim 23, comprising a display of an image of a workpiece to be machined by a robot and optionally with axes associated therewith.

33. Device according to claim 23, comprising a display of a virtual image of a workpiece with a robot path adapted thereto for adapting the robot path to the position of a detected, real workpiece, so that by superimposing the virtual workpiece image with the image of the real workpiece it is possible to adapt the robot path to be performed to the position of the real workpiece.

34. Device according to claim 23, comprising a display of an operating area reachable by a robot and/or a permitted operating area on the viewing device.

35. Device according to claim 23, comprising a display of movement corridors of a robot tool, a robot hand and/or further robot elements on the viewing device.

36. Device according to claim 23, comprising a display of permanent and/or instantaneous associations of at least one manual programmer with at least one robot.

37. Device according to claim 23, comprising fixed markings in space for detecting the position and orientation of the display.

38. Device according to claim 37, wherein the markings are radio receivers, particularly transmitters.

39. Device according to claim 23, wherein the markings can be detected by an optical receiver.

40. Device according to claim 23, comprising data spectacles to be worn by a user for displaying robot-specific, computer-generated information.

41. Device according to claim 23, comprising a transparent screen for displaying robot-specific, computer-generated information.

42. Device according to claim 23, comprising a graphicable screen for displaying robot-specific, computer-generated information together with optoelectronically recorded information.

43. Device according to claim 23, comprising a viewing device in the form of a graphicable screen of a manual programmer for a robot for displaying robot-specific, computer-generated information together with optoelectronically recorded information.

44. Device according to claim 23, comprising a graphicable screen connectable to a manual programmer of a robot as a viewing device for displaying robot-specific, computer-generated information together with optoelectronically recorded information.