Method and apparatus for recognizing gesture in image processing system

• US 8,014,567 B2
• Filed: 07/17/2007
• Issued: 09/06/2011
• Est. Priority Date: 07/19/2006
• Status: Active Grant

• Current Assignee: Intellectual Discovery Co., Ltd.
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Hye Jin, Yoon, Ho Sub, Chi, Su Young, Kim, Do Hyung, Lee, Jae Yeon
• Primary Examiner(s): CARTER, AARON W
• Application Number:
  US11/778,724
• Publication Number:
  US 20080019589A1
• Time in Patent Office: 1,512 Days
• Field of Search: 382/118, 382/100, 382/103, 382/190, 382/165, 382/209, 382218-228, 345/156, 345173-178, 715/863
• US Class Current: 382/103
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A method and apparatus for recognizing a gesture in an image processing system. In the apparatus, an input unit receives an image obtained by capturing a gesture of a user using a camera. A detector detects a face area in the input image, and detects a hand area in gesture search areas. The gesture search areas being set by dividing the image into predetermined areas with reference to a predetermined location of the detected face area. A controller sets the gesture search areas, determines whether a gesture occurs in the detected hand area, and selects a detection area with respect to the gesture to generate a control command for controlling an image device. A calculator calculates skin-color information and differential-area information for checking a gesture in the detected hand area. Accordingly, a hand area can be accurately detected, and a gesture can be separated from peripheral movement information, so that mal-functioning caused by gesture recognition can be reduced.

Claims

1. A method for recognizing a gesture in an image processing system, the method comprising:

detecting a face area in an image captured and input from a camera;

dividing the image to predetermined areas with reference to a predetermined location of the face area, and setting the predetermined areas to gesture search areas;

detecting a hand area in the gesture search areas;

determining whether a gesture occur in the detected hand area; and

selecting a detection area with respect to the gesture and transferring an image of the selected detection area as a control command for controlling an image device

wherein the determining of the gesture occurrence comprises;

determining a skin color of the detected hand area by determining skin-color information of the face area;

calculating a differential image of the hand area between a previous frame and a current frame;

combining differential-area information of the differential image with the determined skin-color information to obtain a final image; and

recognizing gesture occurrence from the final image when an overlapping area between the differential area information and the skin color information is larger than a predetermined area.

2. The method according to claim 1, further comprising switching a resolution of the image of the selected detection area to a high resolution, and switching again the resulting resolution to an initial resolution with reference to a center point to transfer a resulting image as the control command of the image device.

3. The method according to claim 1, wherein the determining of the skin-color information of the face area comprises:

scanning individual red (R), green (G), and blue (B) unit pixels having a red pixel value, a green pixel value, and a blue pixel value in the face area using a preset algorithm;

comparing the scanned R, G and B unit pixels; and

determining a skin color when a result of the comparing meets a predetermined condition to create skin color information of the face area.

4. The method according to claim 3, wherein the preset condition is that the red pixel value is greater than the green pixel value and the blue pixel value in the scanned R, G and B unit pixels of the face area.

5. The method according to claim 1 or 3, wherein the determining of the skin color of the detected hand area comprises calculating a hand color area (T(i, j)) in a predetermined area (i, j) using the skin-color information of the face area through a preset algorithm of the following equation:

6. The method according to claim 5, wherein the determining of the skin color of the detected hand area comprises calculating a final hand color area (T_t(I, j)) by removing an area where a red pixel value greater than a predetermined value is determined as a skin color from the calculated hand color area (T(i, j)) through a preset algorithm of the following equation:

7. The method according to claim 1, wherein the calculating of the differential image comprises obtaining an image where an absolute value (abs( )) of a value obtained by subtracting an intensity value (I_t-1(i, j)) at a previous frame time (t−

1) from an intensity value (I_t(i, j)) at a current frame time (t) is greater than a first threshold value (threshold 1) in a predetermined area (i, j) through a preset algorithm of the following equation;

8. The method according to claim 1, wherein the calculating of the final image comprises calculating a final image with a pixel value of 255 by combining the differential area information (D_t(I, j)) of the differential image with the detected skin-color information (T(i, j)) through a preset algorithm of the following equation:

9. The method according to claim 1, wherein the recognizing of the gesture occurrence comprises recognizing gesture occurrence when an overlapping information count value between the differential area information and the skin-color information is greater than a second threshold value (threshold 2) through a preset algorithm of the following equation:

10. The method according to claim 1, wherein the dividing of the image to the predetermined areas with reference to the predetermined location of the face area and the setting of the predetermined areas to the gesture search areas comprises:

dividing the image into a left-hand area and a right-hand area with reference to a face location of the face area;

setting a reference line at a location randomly spaced apart from the face location; and

setting the predetermined areas to gesture search areas for controlling the image device in the left or right hand areas with reference to the reference line.

11. An apparatus for recognizing a gesture in an image processing system comprising:

an input unit receiving an image obtained by capturing a gesture of a user using a camera;

a detector detecting a face area in the input image, and detecting a hand area in gesture search areas, the gesture search areas being set by dividing the image into predetermined areas with reference to a predetermined location of the detected face area;

a controller setting the gesture search areas, determining whether a gesture occurs in the detected hand area, and selecting a detection area with respect to the gesture to generate a control command for controlling an image device; and

a calculator calculating skin-color information and differential-area information for checking a gesture in the detected hand area

wherein the controller switches a resolution of an image of the selected detection area to a high resolution, switches the high resolution again to an initial resolution with reference to a center point, and transfers a resulting image as a control command of the image device.

12. The apparatus according to claim 11, wherein the controller divides the image into a left-hand area and a right-hand area with reference to a face location of the face area, sets a reference line at a location randomly spaced apart from the face location, and sets predetermined areas to gesture search areas for controlling the image device in the left and right hand areas with reference to the reference line.

13. The apparatus according to claim 11, wherein the calculator scans individual red (R), green (G), and blue (B) unit pixels having a red pixel value, a green pixel value, and a blue pixel value in the face area using a preset algorithm, compares the scanned R, G and B unit pixels, and determines a skin color when a result of the comparing meets a predetermined condition to create the skin color information of the face area.

14. The apparatus according to claim 13, wherein the preset condition is that the red pixel value is greater than the green pixel value and the blue pixel value in the scanned R, G and B unit pixels of the face area.

15. The apparatus according to claim 11, wherein the calculator determines a skin color of the detected hand area by determining skin-color information from the face area, obtains a differential image of the hand area between a previous frame and a current frame, and combines differential area information of the differential image with the determined skin-color information to obtain a final image.

16. The apparatus according to claim 15, wherein the calculator calculates a hand color area (T(i, j)) in a predetermined area (i, j) using the skin-color information of the face area through a preset algorithm of the following equation:

17. The apparatus according to claim 16, wherein calculator calculates a final hand color area (T_t(I, j)) by removing an area where a red pixel value greater than a predetermined value is determined as a skin color from the obtained hand color area (T(i, j)) through a preset algorithm of the following equation:

18. The method according to claim 15, wherein the calculator obtains an image where an absolute value (abs( )) of a value obtained by subtracting an intensity value (I_t-1(i, j)) at a previous frame time (t−

1) from an intensity value (I_t(i, j)) at a current frame time (t) is greater than a first threshold value (threshold 1) in a predetermined area (i, j) through a preset algorithm of the following equation;

19. The method according to claim 18, wherein the calculator calculates the final image having a pixel value of 255 by combining the differential area information (D_t(I, j)) of the differential image with the detected skin-color information (T(i, j)) through a preset algorithm of the following equation:

20. The method according to claim 19, wherein the controller recognizes gesture occurrence when an overlapping information count value between the differential area information and the skin-color information is greater than a second threshold value (threshold 2) through a preset algorithm of the following equation: