SELECTION OF A DEVICE OR OBJECT USING A CAMERA

• US 20160346936A1
• Filed: 05/23/2016
• Published: 12/01/2016
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: KUKA Roboter GmbH (Midea Group Co., Ltd)
• Original Assignee: KUKA Roboter GmbH (Midea Group Co., Ltd)
• Inventors: HIETMANN, Gerhard
• Granted Patent: US 10,095,216 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/39444 Display of position, of shape of robot and tool
• CPC Class Codes: G05B 2219/39449 Pendant, pda displaying camera images overlayed with graphics, augmented reality
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/02 Arm motion controller
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A method for actuating a manipulator system by means of a portable end device, a corresponding portable end device, and a robot system. The portable end device includes a 3-D camera that is a plenoptic camera. A three-dimensional image is acquired by means of the 3-D camera of the portable end device of at least part the manipulator system to be actuated. The image identifies at least one actuatable object in the manipulator system. A task is selected for the identified object to actuate. By selecting the identified actuatable object, the object is actuated to perform the task.

Claims

1. A method for actuating a manipulator system by means of a portable end device, wherein the portable end device includes a camera, the method having the following steps of

acquiring an image, by means of the camera in the portable end device, of at least some of the manipulator system;

determining a feature of the manipulator system based on the acquired image;

identifying at least one actuatable object of the manipulator system based on the determined feature; and

actuating the manipulator system.

2. The method in accordance with claim 1, furthermore having the step of:

providing reference data sets for the manipulator system to be actuated,

wherein the step of determining the feature of the manipulator system furthermore includes the step of performing an image recognition process that detects the feature based on the acquired image and the provided reference data sets.

3. The method in accordance with claim 1, wherein the camera is a 3-D camera and the acquired image is a three-dimensional image.

4. The method in accordance with claim 2, wherein the step of providing reference data sets for the manipulator includes teaching the at least one actuatable object of the manipulator system by means of the portable end device.

5. The method in accordance with claim 1,

wherein the portable end device includes a rendering device that includes a touch panel, and

wherein the method furthermore includes the step of rendering the acquired image on the rendering device of the portable end device.

6. The method in accordance with claim 5, further having the step of rendering an optical marking on the rendering device of the portable end device, which optical marking specifies the identified object of the manipulator system.

7. The method in accordance with claim 6, wherein the step of actuating the manipulator system includes the step of selecting the at least one identified object of the manipulator system.

8. The method in accordance with claim 7, wherein the step of selecting the identified object of the manipulator system includes selecting the optical marking on the rendering device of the portable end device.

9. The method in accordance with claim 5, wherein at least two actuatable objects of the manipulator system are identified and at least two optical markings are rendered on the rendering device of the portable end device, wherein each optical marking specifies one of the identified objects of the manipulator system (10).

10. The method in accordance with claim 1, wherein the actuation of the manipulator system includes actuation of the identified object of the manipulator system.

11. The method in accordance with claim 1, wherein the step of actuating of the manipulator system includes a step of querying, providing, creating, or modifying data of the manipulator system, which data describe at least one of the following:

adjusting or orienting a manipulator;

parameterizing an end effector;

calibrating the manipulator and/or the end effector;

defining a work area;

parameterizing the object of the manipulator system;

defining a coordinate system;

or

a program sequence.

12. The method in accordance with claim 1, wherein the step of actuating of the manipulator system includes a step of creating, modifying, or and/of executing a function or a program directed to at least one of the following:

action of an end effector;

moving the object of the manipulator system;

controlling the object of the manipulator system;

moving a manipulator;

controlling the manipulators;

or

changing a program sequence.

13. The method in accordance with claim 5, further including the step of rendering of context-dependent data, taking into account the determined feature, based on operator input.

14. The method in accordance with claim 1, wherein the camera is a plenoptic camera, and wherein the acquisition of the image includes acquisition of a four-dimensional light field, by means of the plenoptic camera, of the at least one object of the manipulator system to be actuated.

15. A portable end device including a camera and a device control unit, wherein the device control unit is configured to:

acquire an image, by means of the camera in the portable end device, of at least some of a manipulator system;

determine a feature of the manipulator system based on the acquired image;

identify at least one actuatable object of the manipulator system based on the determined feature; and

actuate the manipulator system.

17. The portable end device in accordance with claim 15, wherein the camera is a 3-D camera.

18. The portable end device in accordance with claim 17, wherein the camera is a plenoptic camera.

16. A manipulator system, including a manipulator and a portable end device that includes a camera, wherein the manipulator system furthermore has a manipulator control unit that is configured to:

acquire an image, by means of the camera in the portable end device, of at least some of the manipulator;

determine a feature of the manipulator based on the acquired image;

identify at least one actuatable object of the manipulator based on the determined feature; and

actuate the manipulator.

19. The manipulator system in accordance with claim 16, wherein the camera is a 3-D camera.

20. The manipulator system in accordance with claim 16, wherein the camera is a plenoptic camera.