Trainable autonomous lawn mower

• US 9,137,943 B2
• Filed: 07/26/2013
• Issued: 09/22/2015
• Est. Priority Date: 07/27/2012
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Einecke, Nils, Franzius, Mathias, Wersing, Heiko
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): PECHE, JORGE O
• Application Number:
  US13/951,504
• Publication Number:
  US 20140032033A1
• Time in Patent Office: 788 Days
• Field of Search: 701/24, 701/519, 700/245, 700/259, 901/46, 901/47
• US Class Current: 1/1
• CPC Class Codes: A01D 34/008 for automated or remotely controlled operation
• CPC Class Codes: A01D 75/185 Avoiding collisions with obstacles
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

The present invention proposes a profound user-mower interaction, which enables the user to teach a robotic lawn mower for improving its visual obstacle detection. The main idea of the invention is that the user can show some typical obstacles from his garden to the mower. This will increases the performance of the visual obstacle detection. In the simplest scenario the user just places the mower in front of an obstacle, and activates the learning mechanism of the classification means (module) of the mower. The increasing use of smart devices such as smart phones or tablets enables also more elaborated learning. For example, the mower can send the input image to the smart device, and the user can provide a detailed annotation of the image. This will drastically improve the learning, because the mower is provided with ground truth information.

Claims

1. A robotic lawn mower comprising:

at least one camera for obtaining an input image;

a memory for storing information about obstacles;

a classifier module of a computing unit connected to the memory, for segmenting obstacles within the input image supplied from the camera and identifying segmented obstacles based on the information stored in the memory; and

a control unit for controlling a movement of the mower based on the identification of at least one obstacle;

wherein the classifier module is provided with an interface in order to edit the information stored in the memory upon acceptance of user input,

wherein the robotic lawn mower is configured to send a message containing a snapshot image taken by the at least one camera to a remote smart device preferably using a wireless channel, and

receive information about an annotated snapshot image from the smart device as user input.

2. The robotic lawn mower according to claim 1,

wherein the classifier module is configured to edit the information stored in the memory according to segmented obstacles within the input image.

3. The robotic lawn mower according to claim 1, which is adapted to be selectively in a training mode,

wherein in the training mode the classifier module is configured to automatically start editing the information stored in the memory, if an obstacle is positioned in the field of view of the camera.

4. The robotic lawn mower according to claim 3,

which is provided with mowing blades and which is configured to drive forward with its mowing blades switched off

until the mower hits the obstacle, in order to estimate the position of the obstacle, or

until the obstacle is beneath the mower, in order to estimate the maximal height of the obstacle.

5. The robotic lawn mower according to claim 1,

wherein the classifier module is configured to estimate the position of the obstacle in the field of view of the camera by using a segmentation algorithm that separates foreground pixels from background pixels.

6. The robotic lawn mower according to claim 5,

which is provided with mowing blades and which is configured to drive forward with its mowing blades switched off

until the mower hits the obstacle, in order to estimate the position of the obstacle, or

until the obstacle is beneath the mower, in order to estimate the maximal height of the obstacle.

7. The robotic lawn mower according to claim 1,

which is configured to send the message to the remote smart device,

when the classifier module has segmented an obstacle within the input image, which it cannot identify, or

when a confidence value of the obstacle identification is below a given threshold.

8. The robotic lawn mower according to claim 1,

which is configured to receive a user notification as user input, and

to determine from the user notification, whether the snapshot image contains an obstacle or not.

9. The robotic lawn mower according to claim 8,

which is configured to obtain additional information about an obstacle from at least one tag received together with the user notification.

10. The robotic lawn mower according to claim 9,

wherein the at least one tag is an obstacle class.

11. The robotic lawn mower according to claim 9,

wherein the control unit is configured to control different behaviors of the mower according to different received obstacle tags.

12. The robotic lawn mower according to claim 11,

wherein the different behaviors include evading an obstacle, driving over an obstacle, taking a snapshot image of an obstacle, and signaling an acoustic alarm.

13. The robotic lawn mower according to claim 1,

which is configured to determine from the fact that a user has added a contour around an object in the snapshot image that said object is an obstacle.

14. The robotic lawn mower according to claim 1,

which is configured to communicate with the remote smart device via a data server and a client software.

15. The robotic lawn mower according to claim 14,

wherein the data sever is located in a charging station of the mower.

16. A robotic lawn mower according to claim 14,

wherein the client software is software installed on the remote smart device.

17. A robotic lawn mower according to claim 14,

wherein the data server is adapted to buffer messages in a queue for performing an asynchronous communication between the mower and the remote smart device.

18. A robotic lawn mower according to claim 1,

which is configured to receive a control instruction from the remote smart device.

19. A robotic lawn mower according to claim 1, which is configured to send a live stream of the input image of the at least one camera to the remote smart device.

20. A robotic lawn mower according to claim 1,

wherein the remote smart device is a PDA, a smart phone, a tablet, a PC, an intelligent house control unit or an external control unit for the mower.

21. A method for training a robotic lawn mower comprising a classifier module of a computing unit for segmenting obstacles within an input image supplied from at least one camera and identifying segmented obstacles based on the information stored in a memory,

wherein the method for training comprises:

providing user input via an interface of the classifier module in order to edit the information stored in the memory;

sending a message containing a snapshot image taken by the at least one camera to a remote smart device preferably using a wireless channel; and

receiving information about an annotated snapshot image from the smart device as user input.