Robotic augmentation of creative tasks

• US 10,642,244 B2
• Filed: 12/19/2016
• Issued: 05/05/2020
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Atherton, Evan Patrick, Thomasson, David, Conti, Maurice Ugo, Kerrick, Heather
• Primary Examiner(s): Karim, Ziaul
• Application Number:
  US15/384,236
• Publication Number:
  US 20180173192A1
• Time in Patent Office: 1,233 Days
• Field of Search: 700173
• CPC Class Codes: G05B 19/188 characterised by special applications and not provided for in the relevant subclasses, (e.g. making dies, filament winding)
• CPC Class Codes: G05B 19/402 characterised by control arrangements for positioning, e.g. centring a tool relative to a hole in the workpiece, additional detection means to correct position G05B19/19 takes precedence
• CPC Class Codes: G05B 2219/37582 Position, angle of workpiece surface
• CPC Class Codes: G05B 2219/39004 Assisted by automatic control system for certain functions

Abstract

A robot is configured to assist an end-user with creative tasks. While the end-user modifies the work piece, the robot observes the modifications made by the end-user and determines one or more objectives that the end-user may endeavor to accomplish. The robot then determines a set of actions to perform that assist the end-user with accomplishing the objectives.

Claims

1. A computer-implemented method for controlling a robot, the method comprising:

determining, based on visual sensor data, a first modification that has been applied to a work piece by an end-user;

determining, based on the first modification, a second modification to apply to the work piece, wherein the second modification corresponds to the first modification; and

causing a robot to apply the second modification to the work piece via a separate implement in order to assist the end-user in modifying the work piece.

2. The computer-implemented method of claim 1, wherein determining the first modification comprises determining a first set of actions executed by the end-user relative to the work piece and determining the second modification comprises determining a second set of actions that are derived from the first set of actions.

3. The computer-implemented method of claim 2, wherein:

the first set of actions comprises a sequence of repeated operations applied to a first region of the work piece, and

the second set of actions comprises the sequence of repeated operations applied to a second region of the work piece.

4. The computer-implemented method of claim 1, wherein the first modification is associated with a first theme.

5. The computer-implemented method of claim 4, further comprising determining the second modification by:

comparing the work piece to one or more other work pieces;

identifying a subset of the one or more other work pieces that is associated with the first theme; and

extracting the second modification from the subset of the one or more other work pieces.

6. The computer-implemented method of claim 1, wherein the first modification is associated with a first theme, and the second modification is associated with a second theme.

7. The computer-implemented method of claim 6, further comprising determining the second modification by:

comparing the work piece to one or more other work pieces;

identifying a subset of the one or more other work pieces that includes at least one feature common to the work piece; and

extracting the second modification from the subset of the one or more other work pieces.

8. The computer-implemented method of claim 1, further comprising processing additional information received from the end-user to determine a third modification to the work piece.

9. The computer-implemented method of claim 8, wherein:

the third modification comprises a first number of actions,

the second modification comprises a second number of actions,

the first number of actions is greater than the second number of actions, and

further comprising strengthening, based on the additional information received from the end-user, at least one weight included in a neural network associated with the robot.

10. The computer-implemented method of claim 8, wherein:

the third modification comprises a first number of actions,

the second modification comprises a second number of actions,

the first number of actions is less than the second number of actions, and

further comprising weakening, based on the additional information received from the end-user, at least one weight included in a neural network associated with the robot.

11. The computer-implemented method of claim 1, wherein the visual sensor data includes a video feed depicting one or more modifications made to the work piece.

12. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to control a robot by performing the steps of:

determining, based on visual sensor data, a first modification that has been applied to a work piece by an end-user;

determining, based on the first modification, a second modification to apply to the work piece, wherein the second modification corresponds to the first modification; and

causing a robot to apply the second modification to the work piece via a separate implement in order to assist the end-user in modifying the work piece.

13. The one or more non-transitory computer-readable media of claim 12, wherein:

determining the first modification comprises determining a first set of actions executed by the end-user relative to the work piece; and

determining the second modification comprises determining a second set of actions that are derived from the first set of actions.

14. The one or more non-transitory computer-readable media of claim 13, wherein:

the first set of actions comprises a sequence of repeated operations applied to a first region of the work piece, and

the second set of actions comprises the sequence of repeated operations applied to a second region of the work piece.

15. The one or more non-transitory computer-readable media of claim 12, wherein the first modification is associated with a first theme.

16. The non-transitory computer-readable media of claim 15, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the step of determining the second modification by:

comparing the work piece to one or more other work pieces;

identifying a subset of the one or more other work pieces that is associated with the first theme; and

extracting the second modification from the subset of the one or more other work pieces.

17. The one or more non-transitory computer-readable media of claim 12, wherein the first modification is associated with a first theme, and the second modification is associated with a second theme.

18. The one or more non-transitory computer-readable media of claim 17, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the step of determining the second modification by:

comparing the work piece to one or more other work pieces;

identifying a subset of the one or more other work pieces that includes at least one feature common to the work piece; and

extracting the second modification from the subset of the one or more other work pieces.

19. The non-transitory one or more computer-readable media of claim 12, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to performs the step of receiving feedback from the end-user indicating a correction to be made to the second modification.

20. The non-transitory one or more computer-readable media of claim 19, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to performs the step of adjusting a neural network associated with the robot to incorporate the correction.

21. A system for controlling a robot, comprising:

a memory storing an application; and

a processor that is coupled to the memory and, when executing the application, is configured to;

determine, based on visual sensor data, a first modification that has been applied to a work piece by an end-user,

determine, based on the first modification, a second modification to apply to the work piece, wherein the second modification corresponds to the first modification, and

cause a robot to apply the second modification to the work piece via a separate implement in order to assist the end-user in modifying the work piece.