Cleaning robot, dirt recognition device thereof and cleaning method of robot

• US 8,924,019 B2
• Filed: 06/10/2010
• Issued: 12/30/2014
• Est. Priority Date: 07/03/2009
• Status: Active Grant

• Current Assignee: Ecovacs Robotics (Suzhou) Co. Ltd.
• Original Assignee: Ecovacs Robotics Suzhou Company Limited
• Inventors: Tang, Jinju
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): LI, CE LI
• Application Number:
  US13/381,617
• Publication Number:
  US 20120103367A1
• Time in Patent Office: 1,664 Days
• Field of Search: 700/245, 700/250, 700253-254, 700257-259, 701/23, 701 27- 28, 134/18, 134/21, 15/3, 153/001, 153/19, 153/39, 382/103, 382/108, 382/169, 901/1, 901 46- 47
• US Class Current: 700/259
• CPC Class Codes: A47L 2201/06 Control of the cleaning action for autonomous devices; Automatic detection of the surface condition before, during or after cleaning
• CPC Class Codes: A47L 9/2805 Parameters or conditions being sensed
• CPC Class Codes: A47L 9/2852 Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A cleaning robot a dirt recognition device thereof and a cleaning method of the robot are disclosed. The recognition device includes an image collecting module and an image processing module. The image collecting module may be used for collecting the image information of the surface to be treated by the cleaning robot and sending the image information to the image processing module. The image processing module may divide the collected image information of the surface to be treated into N blocks, extract the image information of each block and process the image information in order to determine the dirtiest surface to be treated that corresponds to one of the N blocks. Through the solution provided by the present invention, the cleaning robot can make an active recognition to the dirt such as dust, so that it can get into the working area accurately and rapidly.

Claims

1. A dirt recognition device of a cleaning robot, comprising;

an image processing module;

an image collecting module for collecting image information of a surface to be treated by the cleaning robot and sending the image information to the image processing module;

wherein the image processing module is configured to divide the collected image information of the surface to be treated into N blocks, wherein N>

1, to extract the image information of each block of image and to process the image information in order to determine, based on the processed image information, a dirtiest surface to be treated that corresponds to one of the N blocks, and consequently the cleaning robot can make an active recognition to dirt.

2. The dirt recognition device of the cleaning robot of claim 1, characterized in that the image processing module comprises:

an image segmentation unit for dividing the collected image information of the surface to be treated by the clean robot into N blocks, wherein N>

1; and

an information extraction unit for extracting the image information of each block of image information and sending the image information to a calculation unit;

wherein the calculation unit is configured to determine the dirtiest surface to be treated that corresponds to one of the N blocks of image information according to the image information of each block of image information.

3. The dirt recognition device of the cleaning robot of claim 2, characterized in that the information extraction unit is a gray-scale value extraction unit, and the calculation unit is a comparison unit and wherein:

the gray-scale value extraction unit is configured to extract the gray-scale value corresponding to each block of the image information; and

the comparison unit is configured to compare the gray-scale value of each block of image, so as to determine the image information with a maximum gray-scale value as the dirtiest surface to be treated.

4. The dirt recognition device of the cleaning robot of claim 2, characterized in that, the information extraction unit is a gray-scale value extraction unit and the calculation unit comprises a characteristic value extraction unit and a comparison unit:

wherein the gray-scale value extraction unit is configured to extract the gray-scale value corresponding to each block of image information;

wherein the characteristic value extraction unit is configured to convert the gray-scale value of each block of image information into a characteristic value corresponding to the gray-scale value, so as to extract a dirt characteristic value from each block of image information; and

wherein the comparison unit is configured to compare the dirt characteristic value of each block of image information, so as to determine the image information with a maximum dirt characteristic value as the dirtiest surface to be treated.

5. The dirt recognition device of the cleaning robot of claim 2, characterized in that, the information extraction unit is a gray-scale value extraction unit and the calculation unit comprises a characteristic value extraction unit, a theoretical confidence calculation unit and a comparison unit;

wherein the gray-scale value extraction unit is configured to extract a corresponding gray-scale value from each block of image information;

wherein the characteristic value extraction unit is configured to convert the gray-scale value of each block into a characteristic value corresponding to the gray-scale value, so as to extract a dirt characteristic from each block of image information;

wherein, using the characteristic value and an instantaneous confidence which is obtained from a database and corresponds to the characteristic value as parameters, the theoretical confidence calculation unit is configured to obtain a theoretical confidence corresponding to each block of image information, according to functional relations among the characteristic value, the instantaneous confidence and the theoretical confidence,

wherein the functional relations among the characteristic value, the instantaneous confidence and the theoretical confidence are described as follows;

P_A=AF_A

wherein, A represents the characteristic value, F_A represents the instantaneous confidence of the characteristic value A, and P_A represents the theoretical confidence of the characteristic value A; and

wherein the comparison unit is configured to compare the theoretical confidence of each block of image information, so as to determine the image information with a maximum theoretical confidence as a dirtiest surface to be treated.

6. The dirt recognition device of the cleaning robot of claim 5, characterized in that, the dirt recognition device also comprises a memory device bearing the database, and the database stores the characteristic value, the instantaneous confidence, the initial confidence and the theoretical confidence which correspond to each other, and the value of the instantaneous confidence corresponding to an initial characteristic value is equivalent to a value of initial confidence.

7. The dirt recognition device of the cleaning robot of claim 2, characterized in that, the information extraction unit is a gray-scale value extraction unit and the calculation unit comprises a characteristic value extraction unit, a theoretical confidence calculation unit and a comparison unit, and the dirt recognition device also comprises:

a dust sensor for sensing a dirt quantity on the surface to be treated and sending information on the sensed dirt quantity to a theoretical confidence correction unit configured to calculate out a difference value between actual dirt quantity and standard dirt quantity, and further configured to calculate out a deviation value of instantaneous confidence according to the functional relation between the difference value and the deviation value of instantaneous confidence;

wherein the gray-scale value extraction unit is configured to extract the corresponding gray-scale value from each block of image information,

wherein the characteristic value extraction unit is configured to convert the gray-scale value of each block into a characteristic value corresponding to the gray-scale value, so as to extract the dirt characteristic from each block of image information,

wherein, according to the functional relations among the characteristic value, an instantaneous confidence, which is obtained from a database and corresponds to the characteristic value, and the deviation value of instantaneous confidence, the theoretical confidence calculation unit is configured to calculate out a theoretical confidence corresponding to each block of image information after correction, and

wherein the comparison unit is configured to compare the theoretical confidence of each block of image information after correction and determine the image information with a maximum theoretical confidence as a dirtiest surface to be treated.

8. The dirt recognition device of the cleaning robot of claim 7, characterized in that, the functional relation between the difference value of actual dirt quantity and standard dirt quantity and the deviation value of instantaneous confidence is described as follows:

F_A(n)=f(x)

wherein X represents the difference value between actual dirt quantity and standard dirt quantity, F_A(n) represents the deviation value of instantaneous confidence when the characteristic value is A after nth correction, and f represents the functional relation expression;

wherein, based on the following formula, the theoretical confidence calculation unit is configured to calculate out the theoretical confidence corresponding to each block of image information after correction;

P_A(n)′

=A(F_A+F_A(n)+F<sub>A(n−

1)</sub>+ . . . _A(1))

wherein, A represents the characteristic value, F_A represents the instantaneous confidence when the characteristic value is A, F_A(n) represents the deviation value of instantaneous confidence that corresponds to the difference value between actual dirt quantity and standard dirt quantity at the time of nth correction, F<sub>A(n−

1)</sub> represents the deviation value of instantaneous confidence that corresponds to the difference value between actual dirt quantity and standard dirt quantity at the time of (n−

1)th correction, F_A(1) represents the deviation value of instantaneous confidence that corresponds to the difference value between actual dirt quantity and standard dirt quantity at the time of the 1st correction, and P_A(n)′

represents the theoretical confidence when the characteristic value is A after the nth correction.

9. The dirt recognition device of the cleaning robot of claim 8,

characterized in that, the functional relation expression of f in F_A(n)=f(x) is represented as follows:

F_A(n)=a_nXⁿ+a<sub>n−

1</sub>X<sup>n−

1</sup>+a<sub>n−

2</sub>X<sup>n−

2</sup>+ . . . +a₁X¹+a₀

wherein, X represents the difference value between actual dirt quantity and standard dirt quantity, F_A(n) represents the deviation value of instantaneous confidence when the characteristic value is A after the nth correction, and a_n through a₀ represent multiple factors.

10. The dirt recognition device of the cleaning robot of claim 7, characterized in that, the dirt recognition device also comprises a memory device bearing the database, and the database stores the characteristic value, the instantaneous confidence, an initial confidence, the deviation value of instantaneous confidence, the theoretical confidence and a threshold value of standard dirt quantity which correspond to each other, and the value of the instantaneous confidence corresponding to an initial characteristic value is equivalent to a value of initial confidence.

11. The dirt recognition device of the cleaning robot of claim 7, characterized in that, the dirt recognition device also comprises a setting unit, which may be used for setting up an initial confidence, a threshold value of standard dirt quantity, or both the initial confidence and the threshold value of standard dirt quantity, and the setting unit is a press key, a knob, a touch or remote control type device.

12. A cleaning robot, comprising:

a robot main body;

a control unit;

a drive unit;

a walking unit;

a cleaning unit, and

a dirt recognition device comprising an image processing module and an image collecting module for collecting image information of a surface to be treated by the cleaning robot and sending the image information to the image processing module, the image processing module being configured to divide the collected image information of the surface to be treated into N blocks, wherein N>

1, to extract the image information of each block of image and being configured to process the image information in order to determine, based on the processed image information, a dirtiest surface to be treated that corresponds to one of the N blocks, and consequently the cleaning robot can make an active recognition to dirt,

wherein the control unit is further configured to control the cleaning unit in operation and control the drive unit, so that the drive unit drives the walking unit to walk, according to the dirtiest surface to be treated, which is determined by the dirt recognition device, and wherein the control unit is further configured to determine a walking route of the walking unit with the dirtiest surface to be treated as a destination, so as to prioritize cleaning on the dirtiest surface to be treated.

13. A cleaning method of a cleaning robot, characterized in that the cleaning method comprises the following steps:

Step 1;

using an image collecting module to collect image information of the surface to be treated by the cleaning robot;

Step 2;

using an image processing module to divide the image information into a plurality of blocks and to convert the image information of each block of image information into a gray-scale value;

Step 3;

the image processing module may determine a dirtiest surface to be treated according to the gray-scale value; and

Step 4;

driving the cleaning robot to walk to the dirtiest surface to be treated and make cleaning on the dirtiest surface to be treated.

14. The cleaning method of the cleaning robot of claim 13, characterized in that step 3 specifically comprises the following steps:

Step 301;

converting the gray-scale value of each block of image information into a characteristic value corresponding to the gray-scale value, thus extracting dirt characteristics from each block of image information; and

Step 302;

comparing the characteristic value of each block of image information, and determining the block with a maximum characteristic value as the dirtiest surface to be treated.

15. The cleaning method of the cleaning robot of claim 13, characterized in that step 3 specifically comprises the hereinafter steps:

Step 311;

converting the gray-scale value of each block into a characteristic value corresponding to the gray-scale value;

Step 312;

according to functional relations among the characteristic value, an instantaneous confidence and a theoretical confidence, obtaining a theoretical confidence that corresponds to each block; and

Step 313;

comparing the theoretical confidence of each block, and determining the block with the maximum theoretical confidence as the dirtiest surface to be treated.

16. The cleaning method of the cleaning robot of claim 15, characterized in that, the functional relation among the characteristic value, the instantaneous confidence and the theoretical confidence as stated in step 312 is as follows:

P_A=AF_A

wherein, A represents the characteristic value, F_A represents the instantaneous confidence when the characteristic value is A, and P_A represents the theoretical confidence when the characteristic value is A.

17. The cleaning method of the cleaning robot of claim 15, characterized in that, the cleaning method also comprises the following steps after step 4:

Step 5;

sensing a dirt quantity on a treating surface through a dust sensor;

Step 6;

calculating a difference value between actual dirt quantity and standard dirt quantity; and

Step 7;

according to the functional relation between the difference value and a deviation value of instantaneous confidence, calculate out the deviation value of instantaneous confidence, and then return to step 312, wherein the functional relation between the difference value (between actual dirt quantity and standard dirt quantity) and the deviation value of instantaneous confidence is expressed as follows;

F_A(n)=f(x)

wherein x represents the difference value between actual dirt quantity and standard dirt quantity and F_A(n) represents the deviation value of instantaneous confidence after the nth correction, and f represents the functional relation expression;

wherein the functional relation among the characteristic value, the instantaneous confidence and the theoretical confidence in step 312 is as follows;

P_A(n)′

=A(F_A+F_A(n)+F<sub>A(n−

1)</sub>+ . . . F_A(1))

wherein, A represents the characteristic value, F_A represents the instantaneous confidence when the characteristic value is A, F_A(n) represents the deviation value of instantaneous confidence that corresponds to the difference value between actual dirt quantity and standard dirt quantity at the time of nth correction, F<sub>A(n−

1)</sub> represents the deviation value of instantaneous confidence that corresponds to the difference value between actual dirt quantity and standard dirt quantity at the time of (n−

1)th correction, F_A(1) represents the deviation value of instantaneous confidence that corresponds to the difference value between actual dirt quantity and standard dirt quantity at the time of the 1st correction, and P_A(n)′

represents the theoretical confidence when the characteristic value is A after the nth correction.

18. The cleaning method of the cleaning robot of claim 17, characterized in that, the functional relation expression of f in F_A(n)=f(x) is:

F_A(n)=a_nXⁿ+a<sub>n−

1</sub>X<sup>n−

1</sup>+a<sub>n−

2</sub>X<sup>n−

2</sup>+ . . . +a₁X¹+a₀

wherein, X represents the difference value between actual dirt quantity and standard dirt quantity, F_A(n) represents the deviation value of instantaneous confidence when the characteristic value is A after the nth correction, and a_n through a₀ represent multiple factors.

19. The cleaning method of the cleaning robot of claim 17, characterized in that, in the step 4, in making cleaning on the dirtiest surface to be treated, if the actual dirt quantity sensed by the dust sensor within a preset time period is less than a set value or the actual dirt quantity sensed by the dust sensor within another preset time period has been greater than a set value, terminating the cleaning on the treated surface and returning to step 1.

20. The cleaning method of the cleaning robot stated in claim 15, characterized in that, the cleaning method also comprises the following step:

detecting whether all instantaneous confidence values have converged to a low limit value within a value range of instantaneous confidence and, if all the instantaneous confidence values have converged to the low limit value within the value range of instantaneous confidence, stopping cleaning work.