TRAINABLE CONVOLUTIONAL NETWORK APPARATUS AND METHODS FOR OPERATING A ROBOTIC VEHICLE

• US 20150306761A1
• Filed: 04/29/2014
• Published: 10/29/2015
• Est. Priority Date: 04/29/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: O'Connor, Peter, Izhikevich, Eugene
• Granted Patent: US 9,346,167 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A robotic vehicle may be operated by a learning controller comprising a trainable convolutional network configured to determine control signal based on sensory input. An input network layer may be configured to transfer sensory input into a hidden layer data using a filter convolution operation. Input layer may be configured to transfer sensory input into hidden layer data using a filter convolution. Output layer may convert hidden layer data to a predicted output using data segmentation and a fully connected array of efficacies. During training, efficacy of network connections may be adapted using a measure determined based on a target output provided by a trainer and an output predicted by the network. A combination of the predicted and the target output may be provided to the vehicle to execute a task. The network adaptation may be configured using an error back propagation method. The network may comprise an input reconstruction.

Claims

1. A method of operating a robotic device by a computerized neuron network comprising an input layer, an intermediate layer and an output layer of neurons, the method comprising:

during one operation of a plurality of operations

causing the robotic device to execute an action along a first trajectory in accordance with a first control signal determined based on a sensory input;

determining, by the output layer, a performance measure based on an evaluation of the first trajectory and indication related to a target trajectory provided by a trainer;

conveying information related to the performance measure to the input layer; and

updating one or more learning parameters of the input layer in accordance with the information; and

during a subsequent operation of a plurality of operations;

causing the robotic device to execute the action along a second trajectory in accordance with a second control signal determined based on the sensory input;

wherein;

the execution of the action along the second trajectory is characterized by a second performance measure; and

the updating is configured to displace the second trajectory closer towards the target trajectory relative to the first trajectory.

2. The method of claim 1, wherein:

the first control signal is based on a feature detected in the sensory input;

one or more first nodes of the input layer process are configured to effectuate the detection of the feature; and

the updating is configured to modify one or more parameters associated with the one or more first nodes of the input layer.

3. The method of claim 2, wherein:

one or more second nodes of the output layer are configured to produce the first control signal;

the one or more first nodes of the input layer are coupled to the one or more second nodes of the output layer via a connectivity array of efficacies; and

the updating comprises a plasticity operation configured to modify one or more efficacies of the connectivity array of efficacies.

4. The method of claim 3, wherein:

the one or more first nodes of the input layer are coupled to the one or more second nodes of the output layer via an-all to all connectivity pattern; and

the connectivity array of efficacies is characterized by a first dimension determined based on a first number of the one or more first nodes of the input layer and a second dimension determined based on a second number of the one or more second nodes of the output layer.

5. The method of claim 1, wherein:

the performance measure comprises a first distance between the first trajectory and the target trajectory; and

the second performance measure comprises a second distance between the second trajectory and the target trajectory, the second distance being smaller than the first distance.

6. The method of claim 1, wherein:

the performance measure comprises a first probability parameter between the first trajectory and the target trajectory; and

the second performance measure comprises a second probability between the second trajectory and the target trajectory, the second probability being greater than the first probability.

7. The method of claim 1, wherein:

the computerized neuron network is configured for operation in accordance with a supervised learning process configured based on a teaching signal; and

the first control signal comprises a combination of the second layer output and a teaching signal provided to the robotic device.

8. A method of generating a predicted control output by an adaptive controller of a robotic apparatus comprising a predictor and a combiner, the method comprising:

configuring the adaptive controller apparatus to detect an object in sensory input provided by a sensor of the robotic apparatus, the object detection causing generation of a control output based on a characteristic of the object;

configuring the predictor to determine a predicted control output based on the characteristic of the object;

configuring the combiner to determine a combined output based on a control input and the predicted control output, the combined output being characterized by a transform function;

determining a performance measure based on the predicted control output and the combined output;

updating one or more learning parameters of the adaptive controller in accordance with the performance measure; and

configuring the adaptive controller to provide the combined output to the robotic apparatus, the combined output configured to cause the robotic apparatus to execute a maneuver in accordance with the characteristic of the object.

9. The method of claim 8, wherein

the object detection is effectuated by a first component of the adaptive controller;

the generation of the control output is effectuated by a second component of the adaptive controller; and

the one or more learning parameters update is uses a back propagation operation configured to convey the performance measure from the second component to the first component.

10. The method of claim 9, wherein the control input is provided by a training entity based on an evaluation of a trajectory associated with the maneuver executed by the robotic apparatus versus a target trajectory.

11. The method of claim 10, wherein the transform function is configured to combine the predicted control output and the control input via one or more operations including a union operation.

12. The method of claim 10, wherein the transform function is configured to combine the predicted control output and the control input via one or more operations including an additive operation.

13. The method of claim 10, wherein:

the training entity comprises a computerized apparatus operable in accordance with a reinforcement learning process; and

the predictor is operable in accordance with a supervised learning process configured based on a teaching signal configured based on the combined output.

14. The method of claim 9, wherein:

the robotic apparatus comprises a vehicle;

the sensory input comprises a video stream of the vehicle surroundings;

the characteristic comprises a representation of an obstacle or a target within the surroundings;

and the maneuver comprises one of an obstacle voidance or a target approach.

15. A computerized neuron network apparatus configured to provide a response based on analysis of visual input frames, the apparatus comprising:

an input component comprising first portion of neurons configured to implement a convolutional operation on the visual input frames using a plurality of filter masks, the operation configured to produce convolved input frames;

an output component comprising at least one output neuron configured to provide an output based on the at least one output neuron reaching a target state;

a connection component configured to couple the input component to the at least one output neuron via an efficacy array; and

a cost estimation component configured to determine a first similarity measure between a response and a target response;

wherein;

the response is configured based on the output;

the first similarity measure determined based on a first analysis of a first frame of the visual input frames is configured to cause update of the neuron network, the update of the neuron network configured to increase a second similarity measure determined based on a second analysis of a second frame of the visual input frames subsequent to the first frame.

16. The apparatus of claim 15, wherein the update comprises:

determination of a discrepancy parameter based on the first similarity measure;

modification of one or more efficacies of the efficacy array using the discrepancy parameter;

backward propagation of the discrepancy parameter from the output component to the input component using a gradient operation with respect to a filter parameter; and

modification of the filter mask parameter based on an outcome of the gradient operation.

17. The apparatus of claim 16, further comprising:

an input reconstruction component configured to produce reconstructed input frames based on the convolved input frames and a de-convolution filter mask;

wherein the update further comprises;

a second modification of the filter mask parameter based on an evaluation of the input frame and a first reconstructed input frame, the second modification configured to reduce the discrepancy for another response generated based on a third frame subsequent to the second frame.

18. The apparatus of claim 17, wherein:

a first response generated in an absence of input reconstruction is characterized by a first value of the second similarity measure

a second response generated based on the input reconstruction is characterized by a second value of the second similarity measure, the second value being greater than the first value.

19. The apparatus of claim 18, wherein the apparatus is configured to determine that the response is closer to the target response based at least on the second value being greater than the first value.

20. The apparatus of claim 19, further comprising:

a combiner component configured to determine the response based at least on the output and a correction signal, the combined output being characterized by a transform function;

wherein the determination of the first similarity measure is configured based on the output and the combined output.

21. The apparatus of claim 20, wherein:

the convolved frame is configured to enable detection of an object;

the output is configured based on the detected object;

the response is configured to be provided to the robotic apparatus, the response being configured to cause the robotic apparatus to execute a first action in accordance with the detected object.

22. The apparatus of claim 20, wherein the first action is selected from the group comprising object avoidance actions and object approach actions.

23. A method of operating a robotic device using a computerized neuron network having a plurality of layers of neurons, the method comprising:

causing the robotic device to execute an action along a first trajectory in accordance with a first control signal, the first signal determined based at least on a sensory input;

determining a performance measure based on an evaluation of the first trajectory relative to a target trajectory;

updating one or more learning parameters of a first of the plurality of layers in accordance with information relating to the determined performance measure; and

causing the robotic device to execute the action along a second trajectory in accordance with a second control signal, the second signal determined based at least on a sensory input and the updated one or more learning parameters, the second trajectory being closer to the target trajectory than the first trajectory.