METHOD AND SYSTEM FOR DETERMINING A CONFIGURATION OF A VIRTUAL ROBOT IN A VIRTUAL ENVIRONMENT

• US 20170091999A1
• Filed: 09/25/2015
• Published: 03/30/2017
• Est. Priority Date: 09/25/2015
• Status: Abandoned Application

• Current Assignee: Siemens Industry Software Limited (Siemens AG)
• Original Assignee: Siemens Industry Software Limited (Siemens AG)
• Inventors: Blumenfeld, Rafael, Schwimmer, Moshe
• Application Number:
  US14/865,226
• Publication Number:
  US 20170091999A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1605 Simulation of manipulator lay-out, design, modelling of manipulator
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

Systems and a method for determining a configuration of a virtual robot in a virtual environment include identifying a real robot from a 3D image of a 3D camera. The identified real robot it is linked to its corresponding virtual robot. The configuration data of the specific configuration in the real scene of the at least one identified real robot is extracted. The extracted location data is used to determine a specific virtual configuration of the virtual robot in the virtual environment reflecting the configuration of the corresponding real robot in the real scene.

Claims

1. A method for determining a configuration of a virtual robot in a virtual environment by a data processing system, wherein the virtual environment is provided with at least one virtual robot representing at least one real robot;

the at least one virtual robot being defined by a three-dimensional (3D) virtual model;

the method comprising;

receiving a positioning of a 3D camera in a real scene;

registering the 3D camera in the virtual environment;

receiving a positioning of at least one real robot at a specific configuration in the real scene;

capturing, by the 3D camera, at least one 3D image of the at least one real robot positioned in the real scene;

identifying, from the at least one captured 3D image, the at least one real robot;

linking the at least one identified real robot to a corresponding virtual robot;

extracting configuration data of the specific configuration in the real scene of the at least one identified real robot; and

determining, from the extracted configuration data, a specific virtual configuration of the virtual robot in the virtual environment reflecting the configuration of the corresponding real robot in the real scene.

2. The method of claim 1, further comprising:

positioning in the virtual environment the virtual robot at a determined specific virtual pose based on the determined specific virtual configuration.

3. The method of claim 2, further comprising:

making at least one adjustment to the specific determined virtual pose of the virtual robot so as to determine an adjusted virtual pose of the virtual robot.

4. The method of claim 3, further comprising:

receiving a positioning of the real robot at an adjusted configuration reflecting the determined adjusted virtual pose of the virtual robot.

5. The method of claim 1, wherein the 3D virtual model is selected from the group consisting of:

a computer-aided design (CAD) model;

a CAD-like model;

a point cloud model; and

a 3D computer model.

6. The method of claim 1, further comprising:

receiving a variation of positioning of a second specific configuration of the at least one identified real robot;

repeating the capturing, the extracting, and the determining so as to re-determine the configuration of the linked virtual robot at a specific second virtual configuration in the virtual environment reflecting the second specific configuration of the corresponding real robot in the real scene.

7. The method of claim 1, wherein the linking is performed automatically via picking.

8. The method of claim 1, wherein the linking is adapted to be performed manually.

9. A data processing system comprising:

a processor; and

an accessible memory, the data processing system particularly configured to;

receive a positioning of a three-dimensional (3D) camera in a real scene;

register the 3D camera in the virtual environment;

receive a positioning of at least one real robot at a specific configuration in the real scene;

receive inputs, by the 3D camera, of capturing at least one 3D image of the at least one real robot positioned in the real scene;

identify, from the at least one captured 3D image, the at least one real robot;

link the at least one identified real robot to a corresponding virtual robot;

extract configuration data of the specific configuration in the real scene of the at least one identified real robot; and

determine, from the extracted configuration data, a specific virtual configuration of the virtual robot in the virtual environment reflecting the configuration of the corresponding real robot in the real scene.

10. The data processing system of claim 9, further configured to:

position in the virtual environment the virtual robot at a determined specific virtual pose based on the determined specific virtual configuration.

11. The data processing system of claim 9, further configured to:

make at least one adjustment to the specific determined virtual pose of the virtual robot so as to determine an adjusted virtual pose of the virtual robot.

12. The data processing system of claim 11, further configured to:

receive a positioning of the real robot at an adjusted configuration reflecting the determined adjusted virtual pose of the virtual robot.

13. The data processing system of claim 9, wherein the 3D virtual model is selected from the group consisting of:

a computer-aided design (CAD) model;

a CAD-like model;

a point cloud model; and

a 3D computer model.

14. A non-transitory computer-readable medium encoded with executable instructions that, when executed, cause one or more data processing systems to:

receive a positioning of a three-dimensional (3D) camera in a real scene;

register the 3D camera in the virtual environment;

receive a positioning of at least one real robot at a specific configuration in the real scene;

receive inputs, by the 3D camera, of capturing at least one 3D image of the at least one real robot positioned in the real scene;

identify, from the at least one captured 3D image, the at least one real robot;

link the at least one identified real robot to a corresponding virtual robot;

extract configuration data of the specific configuration in the real scene of the at least one identified real robot; and

determine, from the extracted configuration data, a specific virtual configuration of the virtual robot in the virtual environment reflecting the configuration of the corresponding real robot in the real scene.

15. The non-transitory computer-readable medium of claim 14 further comprising:

position in the virtual environment the virtual robot at a determined specific virtual pose based on the determined specific virtual configuration.

16. The non-transitory computer-readable medium of claim 14 further comprising:

make at least one adjustment to the specific determined virtual pose of the virtual robot so as to determine an adjusted virtual pose of the virtual robot.

17. The non-transitory computer-readable medium of claim 16 further comprising:

receive a positioning of the real robot at an adjusted configuration reflecting the determined adjusted virtual pose of the virtual robot.

18. The non-transitory computer-readable medium of claim 16, wherein the 3D virtual model is selected from the group consisting of:

a computer-aided design (CAD) model;

a CAD-like model;

a point cloud model; and

a 3D computer model.