Robot cleaner, apparatus and method for recognizing gesture

• US 9,582,711 B2
• Filed: 04/30/2015
• Issued: 02/28/2017
• Est. Priority Date: 06/16/2014
• Status: Active Grant

• Current Assignee: Industry Academic Cooperation Foundation, LG Electronics, Inc. (LG Corporation)
• Original Assignee: Industry Academic Cooperation Foundation, LG Electronics, Inc. (LG Corporation)
• Inventors: Shin, Yongmin, Kim, Euntai, Baek, Jeonghyun
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US14/701,195
• Publication Number:
  US 20150363637A1
• Time in Patent Office: 670 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/46 Sensing device

Abstract

A robot cleaner is provided. The robot cleaner includes a camera obtaining an image including a user; and a control unit extracting, an arm image including an arm, from the image obtained by the camera, calculating an angle of an arm of an user from the arm image, and determining a function intended by the angle of the arm calculated to control execution of the function.

Claims

1. A robot cleaner comprising:

a camera to obtain an image comprising a user of the robot cleaner; and

a control unit to extract from the user image an arm image of an arm of the user, calculate an angle of the arm from the arm image, and determine a function intended by the calculated angle of the arm to control an execution of the determined function,

wherein the control unit comprises;

an image processing unit to divide the arm image into a plurality of cell images;

a histogram calculation unit to locate gradients on the plurality of cell images and calculate histograms in each gradient direction; and

a primary angle calculation unit to calculate primary angles for each cell image based on the calculated histogram of each cell image,

wherein the primary angle calculation unit calculates, as a primary angle of a calculated histogram, an angle corresponding to a bin having a highest value and having a value equal to or larger than a reference value for each histogram.

2. The robot cleaner according to claim 1, further comprising:

a casing on which the camera is disposed; and

a moving device to move the casing,

wherein the control unit controls the moving device based on the calculated angle of the arm.

3. The robot cleaner according to claim 1, wherein the control unit further comprises:

a score calculation unit to calculate a score of each cell image based on the calculated primary angle of each cell image; and

an arm angle calculation unit to calculate the angle of the arm based on the calculated score of each cell image.

4. The robot cleaner according to claim 3, wherein the image processing unit extracts a face image and a shoulder image of the user from the image obtained by the camera and recognizes the arm of the user based on the extracted face and shoulder images.

5. The robot cleaner according to claim 3, wherein the score calculation unit calculates the score of each cell image based on a similarity of the histogram of each cell image to a histogram adjacent in a primary angle direction and a similarity of the histogram of each cell image to a histogram adjacent in a direction perpendicular to the primary angle.

6. The robot cleaner according to claim 3, further comprising a memory to store reference information for calculation of the angle of the arm,

wherein the arm angle calculation unit extracts an angle information and a location information required for calculation of the angle of the arm based on the calculated score of each cell image, extracts determination information by using the angle information and the location information, and calculates the angle of the arm by using the extracted determination information and the reference information stored in the memory.

7. An apparatus for recognizing a gesture, the apparatus comprising:

a camera to obtain an image comprising a user of the apparatus; and

a control unit to extract from the user image an image of an arm of the user, and calculate an angle of the arm,

wherein the control unit comprises;

an image processing unit to divide the arm image into a plurality of cell images;

a histogram calculation unit to locate gradients on the plurality of cell images and calculate histograms in each gradient direction; and

a primary angle calculation unit to calculate primary angles for each cell image based on the calculated histogram of each cell image,

wherein the primary angle calculation unit calculates, as a primary angle of a calculated histogram, an angle corresponding to a bin having a highest value and having a value equal to or larger than a reference value for each histogram.

8. The apparatus according to claim 7, wherein the control unit further comprises:

a score calculation unit to calculate a score of each cell image based on the calculated primary angle of each cell image; and

an arm angle calculation unit to calculate the angle of the arm based on the calculated score of each cell image.

9. The apparatus according to claim 8, further comprising a memory to store reference information for calculation of the angle of the arm,

wherein the arm angle calculation unit extracts an angle information and a location information required for calculation of the angle of the arm based on the calculated score of each cell image, extracts determination information by using the angle information and the location information, and calculates the angle of the arm by using the extracted determination information and the reference information stored in the memory.

10. A method for recognizing a gesture, the method comprising:

extracting, by an image processing unit, an arm image from an image obtained by a camera;

dividing, by the image processing unit, the extracted arm image into a plurality of cell images and calculating a histogram of each cell image;

calculating, by a primary angle calculation unit, a primary angle for each cell image based on the histogram of each cell image;

calculating, by score calculation unit, a score of each cell image based on the primary angle of each cell image; and

calculating, by an arm angle calculation unit, an angle of the arm based on the calculated score of each cell image,

wherein the calculating the primary angle comprises;

calculating, by the primary angle calculation unit, as a primary angle of a calculated histogram, an angle corresponding to a bin having a highest value and having a value equal to or larger than a reference value for each histogram.

11. The method according to claim 10, wherein the calculating of the angle of the arm comprises:

extracting, by the arm angle calculation unit, an angle information and a location information needed for extraction of the angle of the arm based on the calculated score of each cell image;

extracting, by the arm angle calculation unit, determination information by using the angle information and the location information; and

calculating, by the arm angle calculation unit, the angle of the arm by using the extracted determination information and the reference information stored in the memory.

12. The method according to claim 11, further comprising:

controlling a robot cleaner, by a function execution unit, wherein the function execution unit controls the robot cleaner based on the calculated angle of the arm.

13. The method according to claim 10, further comprising:

extracting, by the image processing unit, a face image and a shoulder image of a user from the image obtained by the camera and recognizes the arm of the user based on the extracted face and shoulder images.

14. The method according to claim 10, further comprising:

calculating, by the score calculation unit, the score of each cell image based on a similarity of the histogram of each cell image to a histogram adjacent in a primary angle direction and a similarity of the histogram of each cell image to a histogram adjacent in a direction perpendicular to the primary angle.

15. The method according to claim 10, further comprising:

storing, in a memory, reference information for calculation of the angle of the arm.