Auto-cleaning system, cleaning robot and method of controlling the cleaning robot

• US 9,597,804 B2
• Filed: 07/28/2014
• Issued: 03/21/2017
• Est. Priority Date: 07/29/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoon, Sang Sik, Chung, Woo Ram
• Primary Examiner(s): OH, HARRY Y
• Application Number:
  US14/444,189
• Publication Number:
  US 20150032260A1
• Time in Patent Office: 967 Days
• Field of Search: 700/257
• US Class Current: 1/1
• CPC Class Codes: A47L 11/4011 Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
• CPC Class Codes: A47L 2201/00 Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/2857 User input or output elements for control, e.g. buttons, switches or displays
• CPC Class Codes: B25J 11/0085 Cleaning
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39441 Voice command, camera detects object, grasp, move
• CPC Class Codes: G05B 2219/45098 Vacuum cleaning robot
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00

Abstract

A cleaning robot that performs cleaning while travelling a space to be cleaned, the cleaning robot including: a travelling unit that moves the cleaning robot; a cleaning unit that cleans the space to be cleaned; an image capturing unit that captures an image viewed from the cleaning robot; a voice input unit to which a user'"'"'s voice instructions are input; and a controller obtaining the user'"'"'s motion instructions through the image capturing unit and determining a restricted area in which entry of the cleaning robot is prohibited and/or a focused cleaning area to be intensely cleaned by the cleaning robot based on the user'"'"'s motion instructions or the user'"'"'s voice instructions when the user'"'"'s voice instructions are input through the voice input unit. The restricted area and the focused cleaning area may be input to the cleaning robot through the user'"'"'s voice and motion.

Claims

1. A cleaning robot that performs cleaning while travelling a space to be cleaned, the cleaning robot comprising:

a travelling unit that moves the cleaning robot;

a cleaning unit that cleans the space to be cleaned;

an image capturing unit that captures an image viewed from the cleaning robot;

a voice input unit to which a user'"'"'s voice instructions are input;

a communication unit that communicates with a user terminal; and

a controller that obtains a user'"'"'s motion instructions through the image capturing unit when the user'"'"'s voice instructions are input through the voice input unit and determines a restricted area and/or a focused cleaning area based on the user'"'"'s motion instructions,

wherein the controller transmits the image captured by the image capturing unit and information regarding the restricted area and/or the focused cleaning area to the user terminal through the communication unit, and

the transmitted image and the information regarding the restricted area and/or the focused cleaning area are displayed on the user terminal.

2. The cleaning robot of claim 1, wherein the voice input unit comprises at least three microphones to which the user'"'"'s voice instructions are input.

3. The cleaning robot of claim 2, wherein the controller estimates a position of a user based on a difference in times at which the user'"'"'s voice instructions are input to the at least three microphones.

4. The cleaning robot of claim 3, wherein the controller controls the travelling unit to rotate the cleaning robot so that the image capturing unit is directed toward the position of the user.

5. The cleaning robot of claim 1, wherein the image capturing unit comprises a camera that captures a two-dimensional image of the user and an infrared sensor that obtains distance information of the user captured by the camera.

6. The cleaning robot of claim 5, wherein the controller detects a motion of the user based on the two-dimensional image of the user captured by the image capturing unit and the distance information of the user obtained by the image capturing unit.

7. The cleaning robot of claim 6, wherein the controller detects a hand and a shoulder of the user from the two-dimensional image of the user and determines coordinates of the hand and the shoulder of the user based on the distance information of the user.

8. The cleaning robot of claim 7, wherein the controller determines an area instructed by the user based on the coordinates of the hand and the shoulder of the user and determines the area instructed by the user as the restricted area and/or the focused cleaning area.

16. The cleaning robot of claim 4, wherein the controller controls the travelling unit to rotate in a direction where a rotational displacement is lowest.

17. The cleaning robot of claim 6, wherein the controller detects first and second portions of the user from the two-dimensional image of the user and determines coordinates of the first and second portions of the user based on the distance information of the user.

18. The cleaning robot of claim 17, wherein the controller determines an area instructed by the user based on the coordinates of the first and second portions of the user and determines the area instructed by the user as the restricted area and/or the focused cleaning area.

9. A method of controlling a cleaning robot that performs cleaning while travelling a space to be cleaned, the method comprising:

obtaining an image by an image capturing unit when a user'"'"'s voice instructions are input from a user;

obtaining a motion instruction of the user based on the image; and

determining a restricted area and/or a focused cleaning area based on the user'"'"'s voice instructions and the motion instruction of the user; and

transmitting images the image captured by the image capturing unit and information regarding the restricted area and/or the focused cleaning area to a user terminal,

wherein the transmitted image and the information regarding the restricted area and/or the focused cleaning area are displayed on the user terminal.

10. The method of claim 9, the method further comprising:

determining a position of the user based on a difference in times at which the user'"'"'s voice instructions are input to a plurality of microphones;

rotating the cleaning robot toward the position of the user;

radiating infrared rays toward the user; and

obtaining distance information of the user based on the infrared rays reflected from the user.

11. The method of claim 9, the method further comprising:

detecting a hand and a shoulder of the user from the image; and

determining coordinates of the hand and the shoulder of the user based on distance information of the user.

12. The method of claim 11, the method further comprising:

determining an area instructed by the user based on the coordinates of the hand and the shoulder of the user; and

determining the area instructed by the user as the restricted area or the focused cleaning area.

13. The method of claim 9, the method further comprising:

displaying an image of the space to be cleaned; and

receiving the restricted area and/or the focused cleaning area from the user with respect to the image of the space to be cleaned.

14. The method of claim 13, the method further comprising:

determining coordinates of the focused cleaning area and/or the restricted area in the space to be cleaned based on the coordinates of the focused cleaning area and/or the restricted area input with respect to the image of the space to be cleaned.

15. The method of claim 14, the method further comprising:

displaying an image of the restricted area and/or the focused cleaning area over the image of the space to be cleaned.

19. The method of claim 9, the method further comprising:

detecting first and second portions of the user from the image; and

determining coordinates of the first and second portions of the user based on distance information of the user.

20. The method of claim 19, the method further comprising:

determining an area instructed by the user based on the coordinates of the first and second portions of the user; and

determining the area instructed by the user as the restricted area and/or the focused cleaning area.