Runtime controller for robotic manufacturing system

• US 10,509,392 B2
• Filed: 09/19/2017
• Issued: 12/17/2019
• Est. Priority Date: 04/01/2014
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Linnell, Jeffrey, Byrne, Kendra, Bitterman, Matthew
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Li, Ce Li
• Application Number:
  US15/708,864
• Publication Number:
  US 20180004187A1
• Time in Patent Office: 819 Days
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 19/42 Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
• CPC Class Codes: G05B 2219/36056 Modify program, machining order in real time, during operation, dynamically
• CPC Class Codes: G05B 2219/39432 Direct robot control, click on mouse on variety of display command buttons
• CPC Class Codes: G05B 2219/39443 Portable, adapted to handpalm, with joystick, function keys, display

Abstract

Example systems and methods allow for runtime control of robotic devices during a construction process. One example method includes determining at least one sequence of robot operations corresponding to at least one robot actor, causing the at least one robot actor to execute a portion of the at least one sequence of robot operations during a first time period, receiving an interrupt signal from a mobile computing device indicating a modification to the at least one sequence of robot operations, where the mobile computing device is configured to display a digital interface including one or more robot parameters describing the at least one robot actor and one or more tool parameters describing operating characteristics of at least one physical tool, and causing the at least one robot actor to execute a portion of the at least one modified sequence of robot operations during a second time period.

Claims

1. A computer-implemented method comprising:

providing, for output, a user interface that includes (i) a representation of a current state of performance of a pre-programmed sequence of robot operations being performed by a robot in a physical workcell, and (ii) a control that is associated with repositioning at least a portion of the robot to a position that is different than is specified by the pre-programmed sequence for the current state, that skips over respective positions associated with one or more states in the pre-programmed sequence, and that is specified as a set of pre-programmed coordinates within the physical workcell that are associated with a later state in the pre-programmed sequence;

while the robot is performing the pre-programmed sequence of robot operations, receiving, in real time or near real time to performing the pre-programmed sequence of robot operations by the robot in the physical workcell, data indicating a selection of the control that is associated with repositioning the portion of the robot to the position that is different than is specified by the pre-programmed sequence for the current state, that skips over the respective positions associated with the one or more states in the pre-programmed sequence, and that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence;

while the robot is performing the pre-programmed sequence of robot operations, updating, based at least on receiving the selection of the control that is associated with repositioning the portion of the robot to the position that is different than is specified by the pre-programmed sequence for the current state, that skips over the respective positions associated with the one or more states in the pre-programmed sequence, and that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence, (i) performance of one or more selected robot operations in the pre-programmed sequence of robot operations by the robot in the physical workcell to move the portion of the robot directly from the position that is specified by the pre-programmed sequence for the current state to the different position that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state without moving the portion of the robot to the respective positions that are specified by the pre-programmed sequence for the skipped over states, and (ii) the representation of the current state of performance of the pre-programmed sequence of robot operations being performed by the robot in the physical workcell, that is provided for output on the user interface to reflect the positioning of the robot at the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence; and

after the performance has been updated, resuming performance of one or more remaining robot operations that follow the later state in the sequence of robot operations.

2. The method of claim 1, wherein:

the user interface is provided for output on a mobile device, and

the control is selected by way of a touch input to a proximity sensitive display of the mobile device.

3. The method of claim 1, wherein:

the control is further associated with adjusting a pace at which the pre-programmed sequence of robot operations is performed; and

updating the representation of the current state of performance comprises adjusting a location of a cursor on a timeline representation of the current state of performance of the pre-programmed sequence of robot operations.

4. The method of claim 1, wherein:

the control is further associated with adjusting a tool to associate with the robot in performing the pre-programmed sequence of robot operations.

5. The method of claim 1, comprising determining a numeric value that reflects a quantity of robot operations in the pre-programmed sequence of robot operations that have already been executed, wherein the representation of the current state of performance of the pre-programmed sequence of robot operations comprises the numeric value that reflects the quantity of robot operations that have already been executed.

6. The method of claim 1, wherein the representation of the current state of performance of the pre-programmed sequence of robot operations comprises one or more timestamps that are associated with robot operations that have already been executed.

7. The method of claim 1, wherein the control comprises a digital wheel.

8. The method of claim 1, wherein:

receiving the data indicating the selection of the control comprises receiving a parameter value, and

updating performance of the one or more selected robot operations comprises applying the parameter value to a parameter that is associated with the selected robot operations.

9. The method of claim 1, wherein, after the one or more selected robot operations whose performance has been updated have been performed, the representation of the current state of performance of the one or more remaining operations that follow the later state whose performance is resumed is provided for output on the user interface.

10. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

providing, for output, a user interface that includes (i) a representation of a current state of performance of a pre-programmed sequence of robot operations being performed by a robot in a physical workcell, and (ii) a control that is associated with repositioning at least a portion of the robot to a position that is different than is specified by the pre-programmed sequence for the current state, that skips over respective positions associated with one or more states in the pre-programmed sequence, and that is specified as a set of pre-programmed coordinates within the physical workcell that are associated with a later state in the pre-programmed sequence;

while the robot is performing the pre-programmed sequence of robot operations, receiving, in real time or near real time to performing the pre-programmed sequence of robot operations by the robot in the physical workcell, data indicating a selection of the control that is associated with repositioning the portion of the robot to the position that is different than is specified by the pre-programmed sequence for the current state, that skips over the respective positions associated with the one or more states in the pre-programmed sequence, and that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence;

while the robot is performing the pre-programmed sequence of robot operations, updating, based at least on receiving the selection of the control that is associated with repositioning the portion of the robot to the position that is different than is specified by the pre-programmed sequence for the current state, that skips over the respective positions associated with the one or more states in the pre-programmed sequence, and that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence, (i) performance of one or more selected robot operations in the pre-programmed sequence of robot operations by the robot in the physical workcell to move the portion of the robot directly from the position that is specified by the pre-programmed sequence for the current state to the different position that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state without moving the portion of the robot to the respective positions that are specified by the pre-programmed sequence for the skipped over states, and (ii) the representation of the current state of performance of the pre-programmed sequence of robot operations being performed by the robot in the physical workcell, that is provided for output on the user interface to reflect the positioning of the robot at the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence; and

after the performance has been updated, resuming performance of one or more remaining robot operations that follow the later state in the sequence of robot operations.

11. The medium of claim 10, wherein:

the user interface is provided for output on a mobile device, and

the control is selected by way of a touch input to a proximity sensitive display of the mobile device.

12. The medium of claim 10, wherein:

the control is further associated with adjusting a pace at which the pre-programmed sequence of robot operations is performed; and

updating the representation of the current state of performance comprises adjusting a location of a cursor on a timeline representation of the current state of performance of the pre-programmed sequence of robot operations.

13. The medium of claim 10, wherein:

the control is further associated with adjusting a tool to associate with the robot in performing the pre-programmed sequence of robot operations.

14. The medium of claim 10, comprising determining a numeric value that reflects a quantity of robot operations in the pre-programmed sequence of robot operations that have already been executed, wherein the representation of the current state of performance of the pre-programmed sequence of robot operations comprises the numeric value that reflects the quantity of robot operations that have already been executed.

15. The medium of claim 10, wherein the representation of the current state of performance of the pre-programmed sequence of robot operations comprises one or more timestamps that are associated with robot operations that have already been executed.

16. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

providing, for output, a user interface that includes (i) a representation of a current state of performance of a pre-programmed sequence of robot operations being performed by a robot in a physical workcell, and (ii) a control that is associated with repositioning at least a portion of the robot to a position that is different than is specified by the pre-programmed sequence for the current state, that skips over respective positions associated with one or more states in the pre-programmed sequence, and that is specified as a set of pre-programmed coordinates within the physical workcell that are associated with a later state in the pre-programmed sequence;

while the robot is performing the pre-programmed sequence of robot operations, receiving, in real time or near real time to performing the pre-programmed sequence of robot operations by the robot in the physical workcell, data indicating a selection of the control that is associated with repositioning the portion of the robot to the position that is different than is specified by the pre-programmed sequence for the current state, that skips over the respective positions associated with the one or more states in the pre-programmed sequence, and that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence;

while the robot is performing the pre-programmed sequence of robot operations, updating, based at least on receiving the selection of the control that is associated with repositioning the portion of the robot to the position that is different than is specified by the pre-programmed sequence for the current state, that skips over the respective positions associated with the one or more states in the pre-programmed sequence, and that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence, (i) performance of one or more selected robot operations in the pre-programmed sequence of robot operations by the robot in the physical workcell to move the portion of the robot directly from the position that is specified by the pre-programmed sequence for the current state to the different position that is specified as the set of pre-programmed coordinates within the physical workcell that are associated with the later state without moving the portion of the robot to the respective positions that are specified by the pre-programmed sequence for the skipped over states, and (ii) the representation of the current state of performance of the pre-programmed sequence of robot operations being performed by the robot in the physical workcell, that is provided for output on the user interface to reflect the positioning of the robot at the set of pre-programmed coordinates within the physical workcell that are associated with the later state in the pre-programmed sequence; and

after the performance has been updated, resuming performance of one or more remaining robot operations that follow the later state in the sequence of robot operations.

17. The system of claim 16, wherein:

the user interface is provided for output on a mobile device, and

the control is selected by way of a touch input to a proximity sensitive display of the mobile device.

18. The system of claim 16, wherein:

the control is further associated with adjusting a pace at which the pre-programmed sequence of robot operations is performed; and

updating the representation of the current state of performance comprises adjusting a location of a cursor on a timeline representation of the current state of performance of the pre-programmed sequence of robot operations.

19. The system of claim 16, wherein:

the control is further associated with adjusting a tool to associate with the robot in performing the pre-programmed sequence of robot operations.

20. The system of claim 16, wherein the operations comprise determining a numeric value that reflects a quantity of robot operations in the pre-programmed sequence of robot operations that have already been executed, wherein the representation of the current state of performance of the pre-programmed sequence of robot operations comprises the numeric value that reflects the quantity of robot operations that have already been executed.