Selection of a device or object using a camera
First Claim
1. A method for actuating objects in a manipulator system by means of a portable end device, wherein the portable end device includes a camera, a device control unit and a rendering device, wherein the device control unit performs the steps of:
- acquiring an image, by means of the camera in the portable end device, imaging at least a portion of the manipulator system;
obtaining access to reference data sets for at least the manipulator system wherein reference data sets include camera image data of at least the portion of the manipulator system;
identifying at least one feature of the manipulator system by performing an image recognition process using the camera image and the reference data sets, wherein the at least one feature comprises a visible aspect of the manipulator system that is apparent in the acquired camera image and characteristic of the manipulator system or an actuatable object of the manipulator system;
classifying the imaged manipulator system based on the at least one characteristic feature;
displaying the acquired camera image of the manipulator system on the rendering device of the portable end device;
identifying the at least one actuatable object of the manipulator system based on the at least one characteristic feature and the reference data sets for the identified manipulator system;
rendering an optical marking the camera image diplayed on the rendering device of the portable end device, the optical marking indicating the at least ;
one actuatable object of the manipulator system; and
actuating the actuatable object of the manipulator system upon receiving an input via a user interface of the portable end device to select the optical marking.
1 Assignment
0 Petitions
Accused Products
Abstract
A method for actuating a manipulator system by means of a portable end device, a corresponding portable end device, and a robot system. The portable end device includes a 3-D camera that is a plenoptic camera. A three-dimensional image is acquired by means of the 3-D camera of the portable end device of at least part the manipulator system to be actuated. The image identifies at least one actuatable object in the manipulator system. A task is selected for the identified object to actuate. By selecting the identified actuatable object, the object is actuated to perform the task.
34 Citations
20 Claims
-
1. A method for actuating objects in a manipulator system by means of a portable end device, wherein the portable end device includes a camera, a device control unit and a rendering device, wherein the device control unit performs the steps of:
-
acquiring an image, by means of the camera in the portable end device, imaging at least a portion of the manipulator system; obtaining access to reference data sets for at least the manipulator system wherein reference data sets include camera image data of at least the portion of the manipulator system; identifying at least one feature of the manipulator system by performing an image recognition process using the camera image and the reference data sets, wherein the at least one feature comprises a visible aspect of the manipulator system that is apparent in the acquired camera image and characteristic of the manipulator system or an actuatable object of the manipulator system; classifying the imaged manipulator system based on the at least one characteristic feature; displaying the acquired camera image of the manipulator system on the rendering device of the portable end device; identifying the at least one actuatable object of the manipulator system based on the at least one characteristic feature and the reference data sets for the identified manipulator system; rendering an optical marking the camera image diplayed on the rendering device of the portable end device, the optical marking indicating the at least ;
one actuatable object of the manipulator system; andactuating the actuatable object of the manipulator system upon receiving an input via a user interface of the portable end device to select the optical marking. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A portable end device including a camera, a rendering device and a device control unit, wherein the device control unit is configured to:
-
acquire an image, by means of the camera in the portable end device, of at least a portion of a manipulator system; obtain access to reference data sets for the manipulator system, wherein the reference data sets include camera image data of at least the portion of the manipulator system; determine a feature of the manipulator system by performing an image recognition process using the camera image and the reference data sets, wherein the at least one feature comprises a visible aspect of the manipulator system that is apparent in the acquired camera image and characteristic of the manipulator system or an actuatable object of the manipulator system; classifying the imaged manipulator system based on the at least one characteristic feature; display the acquired camera image of the manipulator system on the rendering device of the portable end device; identify the at least one actuatable object of the manipulator system based on the at least one characteristic feature and the reference data sets for the manipulator system; render an optical marking on the camera image diplayed on the rendering device of the portable end device, the optical marking indicating the at least one actuatable object of the manipulator system; and actuate the actuatable object of the manipulator system upon receiving an input via a user interface of the portable end device to select the optical marking. - View Dependent Claims (14, 15, 16)
-
-
17. A manipulator system, including a manipulator and a portable end device that includes a camera and a rendering device, wherein the manipulator system furthermore has a manipulator control unit that is configured to:
-
acquire an image, by means of the camera in the portable end device, of at least a portion of the manipulator system; obtain access to reference data sets for the manipulator system, wherein the reference data sets include camera image data of at least the portion of the manipulator system; identify at least one feature of the manipulator system by performing an image recognition process using the camera image and the reference data sets, wherein the at least one feature comprises a visible aspect of the manipulator system that is apparent in the acquired camera image and characteristic of the manipulator system or an actuatable object of the manipulator system; classifying the imaged manipulator system based on the at least one characteristic feature; display the acquired camera image of the manipulator system on the rendering device of the portable end device; identify the at least one actuatable object of the manipulator system based on the at least one characteristic feature and the reference data sets for the manipulator system; render an optical marking on the camera image diplayed on the rendering device of the portable end device, the optical marking indicating the at least one actuatable object of the manipulator system; and actuate the actuatable object of the manipulator system upon receiving an input via a user interface of the portable end device to select the optical marking. - View Dependent Claims (18, 19, 20)
-
Specification