Method to minimize collisions of mobile robotic devices

• US 10,207,408 B1
• Filed: 10/06/2016
• Issued: 02/19/2019
• Est. Priority Date: 12/07/2015
• Status: Active Grant

• Current Assignee: Ai Incorprated
• Original Assignee: Ai Incorprated
• Inventors: Ebrahimi Afrouzi, Ali
• Primary Examiner(s): Moyer, Dale
• Application Number:
  US15/286,911
• Time in Patent Office: 866 Days
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/32334 Use of reinforcement learning, agent acts, receives reward
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05B 2219/40499 Reinforcement learning algorithm
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A method for minimizing the rate of collision of a mobile robotic device. A mobile robotic device selects controls comprised of sets of actions to navigate through a workspace. Controls resulting in non-collisions cause the system to earn a positive reward. Controls resulting in collisions cause the system to earn a negative reward. Cumulative rewards over the course of a work session are compared to the cumulative rewards of other work sessions. A policy is defined based on outcomes of prior work sessions to minimize the expected number of collisions.

Claims

1. A method for minimizing the rate of collisions incurred by a mobile robotic device comprising:

selecting controls comprised of sets of actions to navigate through a workspace;

carrying out the selected controls by a mobile robotic device;

detecting whether a collision has been incurred by the mobile robotic device; and

calculating and assigning varying levels of rewards to the mobile robotic device system based on the type of obstacle encountered and the success or failure of controls in avoiding collisions and carrying out controls.

2. The method of claim 1, further comprising:

determining a value of a work session based on the value of rewards earned during the work session; and

iteratively creating policies for navigation to maximize rewards earned during each work session, thereby minimizing collisions.

3. The method of claim 1, wherein assigned rewards are decreased over time within a work session by a decay factor to account for a long term reduction in run time due to deterioration of battery life over time.

4. The method of claim 1, wherein controls are selected, in part, based on input from one or more sensors.

5. A method for minimizing the rate of collisions incurred by a mobile robotic device comprising:

selecting controls comprised of sets of actions to navigate through a workspace;

carrying out the selected controls by a mobile robotic device;

detecting whether a collision has been incurred by the mobile robotic device; and

calculating and assigning rewards to a mobile robotic device system based on the success or failure of controls in avoiding collisions and carrying out controls;

wherein assigned rewards are decreased over time within a work session by a decay factor to account for a long term reduction in run time due to deterioration of battery life over time.

6. The method of claim 5, further comprising:

determining a value of a work session based on the value of rewards earned during the work session; and

iteratively creating policies for navigation to maximize rewards earned during each work session, thereby minimizing collisions.

7. The method of claim 5, wherein controls are selected, in part, based on input from one or more sensors.