Cleaning robot and method for controlling the same

• US 9,658,616 B2
• Filed: 02/03/2016
• Issued: 05/23/2017
• Est. Priority Date: 06/13/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Dong Hun, Park, Heum yong
• Primary Examiner(s): Jen, Ian
• Application Number:
  US15/014,843
• Publication Number:
  US 20160161945A1
• Time in Patent Office: 475 Days
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/0223 involving speed control of the vehicle vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: Y10S 901/02 Arm motion controller

Abstract

A cleaning robot includes a main body, a moving assembly to move the main body, a cleaning tool provided at a bottom part of the main body to collect foreign substances on a floor, an imager to collect images around the main body and a controller to recognize motion of a hand by performing image processing of the collected images, identify a control command corresponding to the motion of the hand, plan a moving direction and a moving distance of the main body as movement information based on the control command, and control operations of the moving assembly and the cleaning tool based on the planned movement information. Since the user directly controls movement of the cleaning robot, it is possible to improve interactivity between human and cleaning robot, reduce the user'"'"'s labor and increase convenience.

Claims

1. A method for controlling a cleaning robot having a moving assembly and a cleaning tool provided at a bottom part of a main body, comprising:

identifying a cleaning mode;

determining whether the identified cleaning mode is an interactive mode;

collecting images around the main body by using an imager upon determining that the cleaning mode is the interactive mode;

recognizing a hand image of a hand of a user in the collected images;

recognizing motion of the hand in the recognized hand image;

planning a moving direction and a moving distance of the main body as movement information based on the motion of the hand;

controlling operations of the moving assembly and the cleaning tool based on the planned movement information;

determining whether an operation command corresponding to the motion of the hand is a command for setting up a virtual wall;

recognizing a start position at a start point and an end position at an end point of recognition of the motion of the hand for setting up the virtual wall upon determining that the operation command is the command for setting up the virtual wall; and

setting up the virtual wall by connecting the start position to the end position.

2. The method according to claim 1, wherein the recognizing the motion of the hand comprises recognizing a position of the hand and a shape of the hand.

3. The method according to claim 2, wherein the planning the moving direction and the moving distance comprises:

selecting any one point in the hand image;

determining a change in position of the selected point; and

determining a moving direction and a moving distance of the hand corresponding to the change in position; and

planning the moving direction and the moving distance of the main body based on the moving direction and the moving distance of the hand, respectively.

4. The method according to claim 3, wherein the determining the change in position comprises:

recognizing a position of the point at predetermined time intervals; and

determining a change in the recognized position of the point.

5. The method according to claim 3, further comprising:

recognizing the shape of the hand;

determining the operation command corresponding to the recognized shape of the hand; and

controlling the operations of the moving assembly and the cleaning tool in response to the determined operation command.

6. The method according to claim 5, wherein the determining the operation command corresponding to the recognized shape of the hand and the controlling the operations comprise:

determining whether the operation command corresponding to the shape of the hand is a position moving command;

identifying the number of unfolded fingers in the shape of the hand upon determining that the operation command is the position moving command;

determining a position corresponding to the identified number of fingers; and

operating the moving assembly to move to the determined position.

7. The method according to claim 3, further comprising:

determining a moving speed of the hand based on the moving distance and a predetermined time; and

controlling a moving speed of the main body based on the determined moving speed of the hand.

8. The method according to claim 1, further comprising:

operating the moving assembly and the cleaning tool such that the cleaning robot performs cleaning while autonomously traveling about a cleaning area based on an obstacle detection signal detected by an obstacle detector upon determining that the cleaning mode is an automatic mode.

9. The method according to claim 1, further comprising:

recognizing the shape of the hand;

determining whether the recognized shape of the hand is a shape for mode change;

changing the cleaning mode from the interactive mode to an automatic mode upon determining that the recognized shape of the hand is a shape for mode change; and

operating the moving assembly and the cleaning tool such that the cleaning robot performs cleaning while autonomously traveling about a cleaning area.

10. The method according to claim 1, further comprising:

determining whether a hand image is recognized in the collected images; and

terminating the interactive mode upon determining that the hand image is not recognized.

11. The method according to claim 1, further comprising:

determining whether a hand image is recognized in the collected images;

changing the cleaning mode from the interactive mode to an automatic mode upon determining that the hand image is not recognized; and

operating the moving assembly and the cleaning tool such that the cleaning robot performs cleaning while autonomously traveling about a cleaning area.

12. The method according to claim 1, wherein the identifying the cleaning mode comprises identifying a cleaning mode input to a user interface.

13. The method according to claim 1, wherein the collecting the images around the main body comprises collecting images in an upward direction of the main body.

14. The method according to claim 1, further comprising:

tracking positions of the hand at predetermined time intervals if a traveling pattern setting command has been input;

setting a traveling pattern by connecting the tracked positions of the hand; and

controlling movement of the moving assembly in the set traveling pattern.

15. The method according to claim 1, wherein the planning the moving direction and the moving distance of the main body comprises:

recognizing a marker image in the collected images;

identifying a change in position of the recognized marker image; and

planning the moving direction and the moving distance of the main body based on the identified change in position,

wherein the marker image is an image of a marker that is one or more of attached to the hand of the user and held in the hand of the user.

16. The method according to claim 1, wherein the collecting the images around the main body by using the imager comprises:

receiving images around the main body transmitted from an imaging device that is an imager mounted separately from the main body.