Systems and methods for initializing a robot to autonomously travel a trained route

• US 10,241,514 B2
• Filed: 05/11/2016
• Issued: 03/26/2019
• Est. Priority Date: 05/11/2016
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Passot, Jean-Baptiste, Rombouts, Jaldert, Griffin, Cody, Black, John
• Primary Examiner(s): Troost, Aaron L
• Application Number:
  US15/152,436
• Publication Number:
  US 20170329333A1
• Time in Patent Office: 1,049 Days
• CPC Class Codes: A47L 11/4011 Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
• CPC Class Codes: A47L 11/4061 Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00

Abstract

Systems and methods for initializing a robot to autonomously travel a route are disclosed. In some exemplary implementations, a robot can detect an initialization object and then determine its position relative to that initialization object. The robot can then learn a route by user demonstration, where the robot associates actions along that route with positions relative to the initialization object. The robot can later detect the initialization object again and determine its position relative to that initialization object. The robot can then autonomously navigate the learned route, performing actions associated with positions relative to the initialization object.

Claims

1. An autonomously navigating robot comprising:

a user interface configured to prompt a user to initiate navigation of the robot autonomously through one of a plurality of available learned routes from one of a plurality of starting locations, the plurality of starting locations being associated with respective ones of a plurality of initialization objects;

a camera configured to take a first image of an initialization object of the plurality of initialization objects while the robot is in a first location and a second image of the initialization object of the plurality of initialization objects while the robot is in a second location;

an odometry unit configured to determine positions of the robot with respect to the initialization object; and

a controller configured to;

initialize the odometry unit relative to the initialization object determined at least in part from the first image while the robot is in the first location;

learn a route by user demonstration, beginning from the first location, where the learned route associates actions of the robot with positions of the robot relative to the initialization object determined by the odometry unit, the plurality of available learned routes comprising the learned route;

detect, via the camera, the initialization object from an image, and associate the image with the second image, the association being indicative of the robot being in the second location;

autonomously select, by the controller, based on the initialization object detected from the associated image, the learned route from the plurality of available learned routes associated with the initialization object;

initialize the odometry unit relative to the initialization object determined at least in part from the second image; and

navigate the robot autonomously beginning from the second location through at least a portion of the selected learned route while instructing the robot to perform one or more associated actions based at least in part with positions of the robot relative to the initialization object determined by the odometry unit.

2. The autonomously navigating robot of claim 1, wherein the odometry unit comprises at least one of an accelerometer, inertial measurement unit, lidar, odometer, gyroscope, visual odometer, and speedometer.

3. The autonomously navigating robot of claim 1, wherein the initialization object is a binary image.

4. The autonomously navigating robot of claim 1, wherein at least one of the one or more associated actions comprises a turning action for the robot.

5. The autonomously navigating robot of claim 1, wherein at least one of the one or more associated actions comprises the activation of a switch configured to turn on a brush for the robot.

6. The autonomously navigating robot of claim 1, wherein the robot is a floor cleaner.

7. The autonomously navigating robot of claim 1, wherein the first location and the second location are substantially similar.

8. The autonomously navigating robot of claim 1, wherein the one or more associated actions are further associated with trajectories of the robot.

9. The autonomously navigating robot of claim 1, wherein:

the controller is further configured to learn one or more additional routes corresponding to at least a second initialization object at a third location, the plurality of available learned routes comprising the learned one or more additional routes; and

the user interface is further configured to, when the robot detects the initialization object or the at least second initialization object, present to the user only filtered ones of the plurality of available learned routes associated with the detected one of the initialization object or the at least second initialization object, the filtered ones of the plurality of available learned routes corresponding to the detected one of the initialization object or the at least second initialization object.

10. A method for operating a robot comprising:

taking a first image of an initialization object while the robot is in a first location;

determining a first starting position of the robot relative to the initialization object based at least in part on the first image while the robot is in the first location;

learning a plurality of routes by user demonstration, beginning from the first location, where the plurality of learned routes associate actions of the robot with positions of the robot relative to the initialization object, wherein the positions are determined at least in part from the first starting position;

taking a second image of the initialization object while the robot is in a second location;

detecting by the robot the initialization object in the second image of the initialization object;

filtering by the robot the plurality of learned routes using the detected initialization object;

presenting one or more of the filtered routes on a display device for selection by a user;

when a route is selected, determining a second starting position of the robot relative to the initialization object based at least in part on the second image while the robot is in the second location;

causing the robot to navigate autonomously beginning from the second location through at least a portion of the selected route while instructing the robot to perform the one or more associated actions based at least in part with positions of the robot relative to the initialization object, wherein the positions are determined at least in part from the second starting position.

11. The method of claim 10, wherein the taking of the first image and the taking of the second image each comprise generating data with a sensor and storing the data in a data structure.

12. The method of claim 10, further comprising beginning autonomous navigation based at least in part on at least one of a user input and a characteristic of the initialization object.

13. The method of claim 10, wherein at least one of the one or more associated actions comprises turning the robot.

14. The method of claim 10, wherein at least one of the one or more associated actions comprises switching on a brush.

15. The method of claim 10, wherein navigating the robot autonomously further comprises navigating the robot autonomously until the robot detects the initialization object.

16. The method of claim 10, further comprising selecting the learned route from a plurality of routes stored in memory.

17. The method of claim 10, wherein at least one of the one or more associated actions comprises associating actions of the robot with trajectories for the robot.

18. A non-transitory computer-readable storage medium having at least one computer program comprising a plurality of instructions stored thereon, the plurality of instructions being executable by a processing apparatus to operate a robot, the instructions being configured to, when executed by the processing apparatus, cause the processing apparatus to:

determine a first starting position of the robot relative to a first initialization object based at least in part on a first image;

learn at least one first route associated with the first initialization object by user demonstration, where the at least one first learned route associates actions of the robot with positions of the robot relative to the first initialization object, wherein the positions relative to the first initialization object are determined at least in part from the first starting position;

determine a second starting position of the robot relative to a second initialization object based at least in part on a second image;

learn at least one second route associated with the second initialization object by user demonstration, where the second learned route associates actions of the robot with positions of the robot relative to the second initialization object, wherein the positions relative to the second initialization object are determined at least in part from the second starting position;

store (i) the at least one first route associated with the first initialization object and (ii) the at least one second route associated with the second initialization object;

when the robot captures the first image, algorithmically identify the first initialization object in the captured first image based on a library of images stored in the storage medium, and based on the identified first initialization object, automatically select and navigate one of the at least one first route associated with the identified first initialization object; and

when the robot captures the second image, algorithmically identify the second initialization object in the captured second image based on the library of images, and based on the identified second initialization object, automatically select and navigate one of the at least one second route associated with the identified second initialization object.

19. The non-transitory computer-readable storage medium of claim 18, wherein the processing apparatus is further configured to execute the plurality of instructions to:

determine an action of the robot at a determined location of the robot; and

execute the action;

wherein the action comprises a turning action for the robot.

20. The non-transitory computer-readable storage medium of claim 18, wherein the processing apparatus is further configured to execute the plurality of instructions to:

determine an action of the robot at a determined location of the robot; and

execute the action;

wherein the action comprises the activation of a switch configured to turn on a brush.

21. The non-transitory computer-readable storage medium of claim 18, wherein the processing apparatus is further configured to execute the plurality of instructions to recognize a predetermined sequence of actions and autonomously navigate the at least one first route or the at least one second route based at least in part on the predetermined sequence of actions.

22. The non-transitory computer-readable storage medium of claim 18, wherein (i) the navigation of one of the at least one first route associated with the identified first initialization object and (ii) the navigation of one of the at least one second route associated with the identified second initialization object each comprises an autonomous navigation based on the learning thereof.

23. The non-transitory computer-readable storage medium of claim 18, wherein the processing apparatus is further configured to execute the plurality of instructions to:

cause a display of (i) the at least one first route associated with the identified first initialization object and (ii) the at least one second route associated with the identified second initialization object;

detect (i) a selection of one of the displayed at least one first route associated with the identified first initialization object or (ii) a selection of one the at least one second route associated with the identified second initialization object.