Apparatus and methods for training path navigation by robots

• US 9,604,359 B1
• Filed: 01/27/2015
• Issued: 03/28/2017
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Grotmol, Oyvind, Sinyavskiy, Oleg
• Primary Examiner(s): Mancho, Ronnie
• Application Number:
  US14/607,018
• Time in Patent Office: 791 Days
• Field of Search: 700/245, 700/246, 700/247, 700/249, 700/250, 700/253, 700/254, 700/258, 700/259, 700/255
• US Class Current: 1/1
• CPC Class Codes: B25J 9/0081 with master teach-in means
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/1607 Calculation of inertia, jacobian matrixes and inverses
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/00 Computing arrangements based on biological models
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/03 Teaching system
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/47 Optical

Abstract

An apparatus and methods for training and/or operating a robotic device to follow a trajectory. A robotic vehicle may utilize a camera and stores the sequence of images of a visual scene seen when following a trajectory during training in an ordered buffer. Motor commands associated with a given image may be stored. During autonomous operation, an acquired image may be compared with one or more images from the training buffer in order to determine the most likely match. An evaluation may be performed in order to determine if the image may correspond to a shifted (e.g., left/right) version of a stored image as previously observed. If the new image is shifted left, right turn command may be issued. If the new image is shifted right then left turn command may be issued.

Claims

1. A method of determining a control signal for a robot, the method being performed by a special purpose computing platform having one or more processors executing instructions stored by a non-transitory computer-readable storage medium, the method comprising:

receiving first input features of a first type and second input features of a second type;

determining a subset of features by randomly selecting at least one of the first input features and at least one of the second input features;

comparing individual features of the subset to corresponding features of a plurality of training feature sets, individual ones of the plurality of training feature sets comprising a number of training features, the number being equal to or greater than a quantity of features within the subset of features;

based on the comparison, determining a similarity measure for a given training set of the plurality of training feature sets, the similarity measure characterizing a similarity between individual features of the subset and one or more features of the given training set;

responsive to the similarity measure breaching a threshold, selecting one or more training sets from the plurality of training feature sets;

determining one or more potential control signals for the robot, individual ones of the one or more potential control signals being associated with a corresponding training set of the plurality of training feature sets; and

determining the control signal based on a transformation obtained from the one or more potential control signals;

wherein;

individual ones of the plurality of training feature sets comprise at least one feature of the first type and at least one feature of the second type;

the individual ones of the plurality of training feature sets are obtained during a training operation of the robot, the training operation being performed responsive to receiving a training signal from the robot; and

individual ones of the one or more potential control signals are determined based on the training signal and the one or more features of the given training set.

2. The method of claim 1, wherein the similarity measure is determined based on a difference between values of individual features of the subset and one or more values of the one or more features of the given training set.

3. The method of claim 1, wherein the similarity measure is determined based on a distance metric between the individual features of the subset and the one or more features of the given training set.

4. The method of claim 3, wherein selecting one or more training sets comprises selecting a training set associated with a smallest distance metric.

5. The method of claim 3, wherein selecting one or more training sets comprises selecting N training sets associated with a lowest percentile of the distance metric, N being greater than two.

6. The method of claim 5, wherein the transformation comprises a statistical operation performed on individual ones of the one or more potential control signals associated with the selected N training sets.

7. The method of claim 6, wherein the statistical operation is selected from the group consisting of mean and percentile.

8. The method of claim 5, wherein the transformation comprises a weighted sum of a product of individual ones of the one or more potential control signals and a corresponding distance measure associated with the selected N training sets.

9. The method of claim 1, wherein:

the control signal is configured to cause the robot to execute the action;

the first input type comprises a digital image type comprising a plurality of pixel values; and

the second input type comprises a binary indication type associated with the action being executed.

10. The method of claim 9, wherein:

the training comprises a plurality of iterations configured based on the training signal; and

a given iteration is characterized by a control command and a performance measure associated with the action execution based on the control command.

11. The method of claim 9, wherein:

the plurality of pixels comprises at least 10 pixels; and

the random selection is performed based on a random number generation operation.

12. A self-contained robotic apparatus, the apparatus comprising:

a platform comprising a motor;

a first sensor component configured to provide a signal configured to convey a video frame comprising a plurality of pixels;

a second sensor component configured to provide a binary sensor signal characterized by one of two states;

a memory component configured to store training sets, a given training set comprising an instance of the video frame, an instance of the binary sensor signal, and an instance of a motor control indication configured to cause the apparatus to execute an action; and

one or more physical processors configured to operate a random k-nearest neighbors learning process to determine a motor control indication, the one or more physical processors configured to;

determine a subset of features comprising the binary sensor signal and a set of pixels randomly selected from the plurality of pixels;

scale individual pixels of the set of pixels by a scaling factor;

scale features of the subset of features by a scaling factor;

compare individual scaled features of the subset to corresponding features of individual ones of the training sets;

based on the comparison, determine a similarity measure for a given training set, the similarity measure characterizing a similarity between the individual scaled features of the subset and features of the given training set;

based on an evaluation of the similarity measure, select one or more of the training sets;

determine one or more potential control signals for the robot, individual ones of the one or more potential control signals being associated with a corresponding training set; and

determine the control signal based on a transformation obtained from the one or more potential control signals;

wherein;

individual ones of the plurality of training feature sets comprise at least one feature of the first type and at least one feature of the second type;

individual ones of the plurality of training feature sets are obtained during training operation of the robot, the training operation being performed responsive to receiving a training signal from the robot; and

individual ones of the one or more potential control signals are determined based on the training signal and the features of the given training set.

13. The apparatus of claim 12, wherein:

the one or more physical processors are configured to scale features by a multiplication of a first input by the scaling factor; and

the scaling factor is determined based on a number of pixels in the subset.

14. The apparatus of claim 13, wherein the scaling factor is determined based on a ratio of a range of pixel values to a range of the binary values.

15. The apparatus of claim 12, wherein:

the action comprises target-approach-obstacle-avoidance; and

the one or more physical processors are configured to scale features based on a size of an obstacle or an object as the obstacle or object appears in the video frame.

16. The apparatus of claim 12, wherein the one or more physical processors are configured to scale features by specific pixels.

17. A non-transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by a processor to selecting an outcome of a plurality of outcomes by at least:

determine a history of sensory input;

apply a transformation to an instance of the sensory input, the transformation being configured to produce a scaled input based on an analysis of the history;

determine a set of features comprising features of a first type randomly selected from the scaled input and at least one feature of a second type;

compare individual features of the set of features to corresponding features of a plurality of training feature sets, individual ones of the plurality of training feature sets comprising a number of training features, the number being equal to or greater than the quantity of features within the set of features;

based on the comparison, determine a similarity measure for a given training set of the plurality of training feature sets, the similarity measure characterizing a similarity between features of a subset and features of the given training set;

responsive to the similarity measure breaching a threshold, select one or more training sets from the plurality of training sets;

determine one or more potential control signals for the robot, individual ones of the one or more potential control signals being associated with a corresponding training set of the plurality of training sets; and

determine the control signal based on a transformation obtained from the one or more potential control signals;

wherein;

individual ones of the plurality of training feature sets comprise at least one feature of the first type and at least one feature of the second type;

individual ones of the plurality of training feature sets are obtained during training operation of the robot, the training operation being performed responsive to receiving a training signal from the robot; and

individual ones of the one or more potential control signals being determined based on the training signal and the features of the given training set.

18. The storage medium of claim 17, wherein:

the analysis of the history comprises a determination of a feature mean and a feature standard deviation; and

the transformation comprises subtracting the feature mean and dividing the outcome by the feature standard deviation.

19. The storage medium of claim 17, wherein:

the set of features comprises a plurality of set features, individual ones of the set features characterized by a pointer; and

the feature mean and the feature standard deviation is configured for a respective pointer.

20. The storage medium of claim 19, wherein:

the features of the first type comprise a matrix of values;

the pointer identifies a value within the matrix; and

the feature mean and the feature standard deviation are configured for a given location within the matrix.