Systems and methods for configurable operation of a robot based on area classification

• US 10,168,709 B2
• Filed: 09/14/2017
• Issued: 01/01/2019
• Est. Priority Date: 09/14/2016
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Kleiner, Alexander D., Munich, Mario E.
• Primary Examiner(s): Edwards, Jerrah
• Application Number:
  US15/704,814
• Publication Number:
  US 20180074508A1
• Time in Patent Office: 474 Days
• CPC Class Codes: A47L 11/4011 Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/0488 Combinations or arrangements of several tools, e.g. edge cleaning tools
• CPC Class Codes: A47L 9/2852 Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
• CPC Class Codes: A47L 9/2857 User input or output elements for control, e.g. buttons, switches or displays
• CPC Class Codes: A47L 9/2894 Details related to signal transmission in suction cleaners
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0219 ensuring the processing of the whole working surface
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/2246 displaying a map of the environment
• CPC Class Codes: G05D 1/2464 using an occupancy grid
• CPC Class Codes: G05D 1/6482 by dividing the whole area or space in sectors to be processed separately
• CPC Class Codes: G05D 2105/10 for cleaning, vacuuming or polishing
• CPC Class Codes: G05D 2107/40 Indoor domestic environment
• CPC Class Codes: G05D 2109/10 Land vehicles
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A method of operating a mobile robot includes generating a segmentation map defining respective regions of a surface based on occupancy data that is collected by a mobile robot responsive to navigation of the surface, identifying sub-regions of at least one of the respective regions as non-clutter and clutter areas, and computing a coverage pattern based on identification of the sub-regions. The coverage pattern indicates a sequence for navigation of the non-clutter and clutter areas, and is provided to the mobile robot. Responsive to the coverage pattern, the mobile robot sequentially navigates the non-clutter and clutter areas of the at least one of the respective regions of the surface in the sequence indicated by the coverage pattern. Related methods, computing devices, and computer program products are also discussed.

Claims

1. A computing device, comprising:

a processor; and

a memory coupled to the processor, the memory comprising a non-transitory computer-readable storage medium storing computer-readable program code therein that is executable by the processor to perform operations comprising;

generating a segmentation map defining respective regions of a surface based on occupancy data that is collected by a mobile robot responsive to navigation of the surface, wherein the segmentation map comprises simplified boundaries relative to actual boundaries indicated by the occupancy data;

identifying sub-regions of at least one of the respective regions as non-clutter and clutter areas;

modifying the segmentation map comprising the simplified boundaries to indicate the clutter areas responsive to identification thereof;

computing a coverage pattern based on identification of the sub-regions, the coverage pattern indicating a sequence for navigation of the non-clutter and clutter areas;

and

providing the coverage pattern to the mobile robot,

wherein, responsive to the coverage pattern, the mobile robot sequentially navigates the non-clutter and clutter areas of the at least one of the respective regions of the surface in the sequence indicated by the coverage pattern.

2. The computing device of claim 1, wherein the sequence further comprises navigation of a perimeter of the surface, and wherein the mobile robot sequentially navigates the non-clutter area, the clutter area, and the perimeter of the surface in the sequence indicated by the coverage pattern.

3. The computing device of claim 1, wherein the coverage pattern further indicates a rank direction for the non-clutter area, the rank direction corresponding to a dimension of the at least one of the respective regions that is longer than another dimension thereof, and wherein the mobile robot traverses the non-clutter area in the rank direction indicated by the coverage pattern.

4. The computing device of claim 3, wherein computing the coverage pattern comprises:

determining a location on the surface for ending the navigation of the non-clutter area based on a proximity to the clutter area,

wherein, responsive to traversing the non-clutter area in the rank direction indicated by the coverage pattern, the mobile robot ends the navigation of the non-clutter area at the location having the proximity to the clutter area and traverses the clutter area in a different pattern while avoiding obstacles in the clutter area.

5. The computing device of claim 1, wherein the respective regions defined by the segmentation map correspond to respective rooms, wherein the coverage pattern further indicates an order of navigation of the respective rooms, and wherein the mobile robot sequentially navigates the non-clutter and clutter areas of one of the respective rooms in the sequence indicated by the coverage pattern before navigation of a next one of the respective rooms in the order indicated by the coverage pattern.

6. The computing device of claim 1, wherein the operations further comprise:

providing the segmentation map to a user device, the user device comprising a user interface that is configured to display the segmentation map; and

receiving, from the user device, a selection input responsive to providing the segmentation map thereto,

wherein computing the coverage pattern is further based on the selection input.

7. The computing device of claim 6, wherein the selection input comprises respective levels of cleaning for the at least one of the respective regions and/or the sub-regions thereof, wherein the coverage pattern further indicates the respective levels of cleaning, and wherein the mobile robot executes a number of cleaning passes in navigating the at least one of the respective regions and/or the sub-regions thereof according to the respective levels of cleaning indicated by the coverage pattern.

8. The computing device of claim 6, wherein the selection input comprises a user-defined boundary or label for one or more of the respective regions and/or sub-regions thereof, wherein the coverage pattern further indicates the user-defined boundary or label, and wherein the mobile robot avoids navigation of the one or more of the respective regions and/or sub-regions thereof according to the user-defined boundary or label indicated by the coverage pattern.

9. The computing device of claim 1, wherein the operations further comprise:

detecting respective flooring types of the sub-regions,

wherein computing the coverage pattern further comprises determining the sequence for navigation of the non-clutter and clutter areas based on the respective flooring types of the sub-regions corresponding thereto.

10. The computing device of claim 1, wherein the segmentation map comprises data previously received from a user device and/or collected by the mobile robot responsive to at least one previous navigation of the surface.

11. The computing device of claim 10, wherein generating the segmentation map comprises:

comparing the occupancy data collected by the mobile robot responsive to the navigation of the surface with the data collected by the mobile robot responsive to the at least one previous navigation of the surface; and

updating the segmentation map to include commonalities indicated by the comparing and to exclude outliers indicated by the comparing.

12. The computing device of claim 1, wherein generating the segmentation map comprises:

computing a binary image based on the occupancy data by applying a thresholding function; and

performing a watershed transformation on the binary image to partition the binary image into the respective regions by applying a distance transform.

13. The computing device of claim 1, wherein the modifying the segmentation map comprising the simplified boundaries to indicate the clutter areas responsive to the identification thereof is prior to providing the segmentation map to a user device.

14. The computing device of claim 1, wherein identifying the sub-regions as non-clutter and clutter areas comprises:

accessing a data store comprising a plurality of patterns and identifications thereof; and

classifying pixel regions within the sub-regions based on similarities to the plurality of patterns stored in the data store.

15. The computing device of claim 14, wherein one or more of the identifications of the plurality of patterns are based on respective labeling inputs received from one or more user devices.

16. The computing device of claim 15, wherein identifying the sub-regions as non-clutter and clutter areas further comprises:

classifying the pixel regions as static structures corresponding to walls or dynamic structures corresponding to clutter responsive to accessing the data store.

17. The computing device of claim 14, wherein identifying the sub-regions as non-clutter and clutter areas comprises:

identifying at least one boundary based on the occupancy data; and

prior to accessing the data store, distinguishing the pixel regions as attached pixel regions that are adjacent to at least one boundary and detached pixel regions that are spaced apart from the at least one boundary.

18. The computing device of claim 1, wherein the computing device comprises a component of the mobile robot, wherein the occupancy data is detected by at least one sensor of the mobile robot, and wherein the operations further comprise:

operating a drive of the mobile robot to sequentially navigate the non-clutter and clutter areas of the at least one of the respective regions of the surface in the sequence indicated by the coverage pattern.

19. The computing device of claim 1, wherein the computing device comprises a component of a server communicatively coupled to the mobile robot, and wherein the operations further comprise:

receiving the occupancy data from the mobile robot responsive to the navigation of the surface; and

storing the segmentation map in a data store.

20. A method of operating a mobile robot, comprising:

executing, by at least one processor, computer readable instructions stored in a non-transitory computer readable storage medium to perform operations comprising;

generating a segmentation map defining respective regions of a surface based on occupancy data that is collected by the mobile robot responsive to navigation of the surface, wherein the segmentation map comprises simplified boundaries relative to actual boundaries indicated by the occupancy data;

identifying sub-regions of at least one of the respective regions as non-clutter and clutter areas;

modifying the segmentation map comprising the simplified boundaries to indicate the clutter areas responsive to identification thereof;

computing a coverage pattern based on identification of the sub-regions, the coverage pattern indicating a sequence for navigation of the non-clutter and clutter areas; and

providing the coverage pattern to the mobile robot,

wherein, responsive to the coverage pattern, the mobile robot sequentially navigates the non-clutter and clutter areas of the at least one of the respective regions of the surface in the sequence indicated by the coverage pattern.

21. A method of operating a user terminal, the method comprising:

executing, by at least one processor of the user terminal, computer readable instructions stored in a memory comprising a non-transitory computer readable storage medium to perform operations comprising;

receiving, via a transceiver of the user terminal, a segmentation map defining respective regions of a surface based on occupancy data that is collected by a mobile robot responsive to navigation of the surface, wherein the segmentation map comprises simplified boundaries relative to actual boundaries indicated by the occupancy data and identifies sub-regions of at least one of the respective regions as clutter and non-clutter areas;

displaying, via a user interface of the user terminal, a graphical representation of the segmentation map, wherein the graphical representation of the segmentation map comprises the simplified boundaries and indicates the clutter areas;

receiving, via the user interface, a selection input corresponding to one or more of the respective regions and/or the sub-regions thereof; and

transmitting, via the transceiver, a modified segmentation map comprising the selection input to the mobile robot,

wherein, responsive to the modified segmentation map, the mobile robot navigates the one or more of the respective regions of the surface and/or the sub-regions thereof corresponding to the selection input.

22. A computing device, comprising:

a processor; and

a memory coupled to the processor, the memory comprising a non-transitory computer-readable storage medium storing computer-readable program code therein that is executable by the processor to perform operations comprising;

generating a segmentation map defining respective regions of a surface based on occupancy data that is collected by a mobile robot responsive to navigation of the surface;

identifying sub-regions of at least one of the respective regions as non-clutter and clutter areas;

computing a coverage pattern based on identification of the sub-regions, the coverage pattern indicating a sequence for navigation of the non-clutter and clutter areas and a rank direction for traversing the non-clutter area, wherein the rank direction corresponds to a dimension of the at least one of the respective regions that is longer than another dimension thereof, and wherein computing the coverage pattern comprises determining a location on the surface for ending the navigation of the non-clutter area based on a proximity to the clutter area; and

providing the coverage pattern to the mobile robot,

wherein, responsive to the coverage pattern, the mobile robot sequentially navigates the non-clutter and clutter areas of the at least one of the respective regions of the surface in the sequence indicated by the coverage pattern, and

wherein, responsive to traversing the non-clutter area in the rank direction indicated by the coverage pattern, the mobile robot ends the navigation of the non-clutter area at the location having the proximity to the clutter area and traverses the clutter area in a different pattern while avoiding obstacles in the clutter area.

23. The computing device of claim 22, wherein the sequence further comprises navigation of a perimeter of the surface, and wherein the mobile robot sequentially navigates the non-clutter area, the clutter area, and the perimeter of the surface in the sequence indicated by the coverage pattern.

24. The computing device of claim 22, wherein the respective regions defined by the segmentation map correspond to respective rooms, wherein the coverage pattern further indicates an order of navigation of the respective rooms, and wherein the mobile robot sequentially navigates the non-clutter and clutter areas of one of the respective rooms in the sequence indicated by the coverage pattern before navigation of a next one of the respective rooms in the order indicated by the coverage pattern.

25. The computing device of claim 22, wherein the operations further comprise:

providing the segmentation map to a user device, the user device comprising a user interface that is configured to display the segmentation map; and

receiving, from the user device, a selection input responsive to providing the segmentation map thereto,

wherein computing the coverage pattern is further based on the selection input.

26. The computing device of claim 25, wherein the selection input comprises

a user-defined boundary or label for one or more of the respective regions and/or sub-regions thereof, wherein the coverage pattern further indicates the user-defined boundary or label, and wherein the mobile robot avoids navigation of the one or more of the respective regions and/or sub-regions thereof according to the user-defined boundary or label indicated by the coverage pattern.

27. The computing device of claim 25, wherein the selection input comprises respective levels of cleaning for the at least one of the respective regions and/or the sub-regions thereof, wherein the coverage pattern further indicates the respective levels of cleaning, and wherein the mobile robot executes a number of cleaning passes in navigating the at least one of the respective regions and/or the sub-regions thereof according to the respective levels of cleaning indicated by the coverage pattern.

28. The computing device of claim 22, wherein responsive to traversing the clutter area, the mobile robot does not traverse the non-clutter area in the rank direction before traversing at least one other of the sub-regions.

29. The computing device of claim 22, wherein the computing device comprises a component of the mobile robot, wherein the occupancy data is detected by at least one sensor of the mobile robot, and wherein the operations further comprise:

operating a drive of the mobile robot to sequentially navigate the non-clutter and clutter areas of the at least one of the respective regions of the surface in the sequence indicated by the coverage pattern.