ROBOTIC CAMERA CONTROL VIA MOTION CAPTURE

• US 20180021956A1
• Filed: 07/21/2016
• Published: 01/25/2018
• Est. Priority Date: 07/21/2016
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Atherton, Evan Patrick, Thomasson, David, Conti, Maurice Ugo, Kerrick, Heather
• Granted Patent: US 10,427,305 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1679 characterised by the tasks executed
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A motion capture setup records the movements of an operator, and a control engine then translates those movements into control signals for controlling a robot. The control engine may directly translate the operator movements into analogous movements to be performed by the robot, or the control engine may compute robot dynamics that cause a portion of the robot to mimic a corresponding portion of the operator.

Claims

1. A computer-implemented method for controlling a robot, the method comprising:

generating motion capture data based on one or more movements of an operator;

processing the motion capture data to generate control signals for controlling the robot; and

transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator.

2. The computer-implemented method of claim 1, wherein generating the motion capture data comprises tracking at least one of head, body, and limb movements associated with the operator via a motion capture setup.

3. The computer-implemented method of claim 1, wherein processing the motion capture data to generate the control signals comprises:

modeling a first set of dynamics of the operator that corresponds to the one or more movements;

determining a second set of dynamics of the robot that corresponds to the one or more movements; and

generating the control signals based on the second set of dynamics.

4. The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements.

5. The computer-implemented method of claim 4, wherein the robot comprises a robotic arm that includes the set of joints.

6. The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements comprises adjusting a speed of at least one motor coupled to a propeller.

7. The computer-implemented method of claim 6, wherein the robot comprises a flying drone.

8. The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a subset of the one or more movements.

9. The computer-implemented method of claim 1, further comprising capturing sensor data while the robot mimics the one or more movements.

10. The computer-implemented method of claim 1, wherein the one or more movements comprise a target trajectory for a sensor array coupled to the robot.

11. A non-transitory computer-readable medium that, when executed by a processor, causes the processor to control a robot by performing the steps of:

generating motion capture data based on one or more movements of an operator;

processing the motion capture data to generate control signals for controlling the robot; and

transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator.

12. The non-transitory computer-readable medium of claim 11, wherein generating the motion capture data comprises tracking at least one of head, body, and limb movements associated with the operator via a motion capture setup.

13. The non-transitory computer-readable medium of claim 11, wherein processing the motion capture data to generate the control signals comprises:

modeling a first set of dynamics of the operator that corresponds to the one or more movements;

determining a second set of dynamics of the robot that corresponds to the one or more movements; and

generating the control signals based on the second set of dynamics.

14. The non-transitory computer-readable medium of claim 11, wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements.

15. The non-transitory computer-readable medium of claim 11, wherein causing the robot to mimic the one or more movements comprises adjusting a speed of at least one motor coupled to a propeller.

16. The non-transitory computer-readable medium of claim 11, wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a subset of the one or more movements.

17. The non-transitory computer-readable medium of claim 11, further comprising capturing sensor data while the robot mimics the one or more movements.

18. The non-transitory computer-readable medium of claim 11, wherein the one or more movements comprise a target trajectory for a sensor array coupled to the robot.

19. A system for controlling a robot, comprising:

a memory storing a control application; and

a processor coupled to the memory and configured to;

generate motion capture data based on one or more movements of an operator,

process the motion capture data to generate control signals for controlling the robot, and

transmit the control signals to the robot to cause the robot to mimic the one or more movements of the operator.

20. The system of claim 19, wherein the processor is configured to execute the control application to:

generate the motion capture data;

process the motion capture data to generate the control signals; and

transmit the control signals to the robot.