Gesture recognition apparatus, robot system including the same and gesture recognition method using the same

• US 9,129,154 B2
• Filed: 12/15/2009
• Issued: 09/08/2015
• Est. Priority Date: 07/03/2009
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Do Hyung, Lee, Jae Yeon, Yun, Woo Han, Chi, Su Young, Yoon, Ho Sub, Kim, Hye Jin, Yoon, Young Woo
• Primary Examiner(s): SERRAO, RANODHI N
• Application Number:
  US12/638,302
• Publication Number:
  US 20110001813A1
• Time in Patent Office: 2,093 Days
• Field of Search: 348/77, 345/158, 382/199, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Provided is a gesture recognition apparatus. The gesture recognition apparatus includes a human detection unit, a gesture region setting region, an arm detection unit and a gesture determination unit. The human detection unit detects a face region of a user from an input image. The gesture region setting unit sets a gesture region, in which a gesture of the user'"'"'s arm occurs, with respect to the detected face region. The arm detection unit detects an arm region of the user in the gesture region. The gesture determination unit analyzes a position, moving directionality and shape information of the arm region in the gesture region to determine a target gesture of the user. Such a gesture recognition apparatus may be used as a useful means for a human-robot interaction in a long distance where a robot has difficulty in recognizing a user'"'"'s voice.

Claims

1. A gesture recognition apparatus, comprising:

a camera detecting an input image;

a human detection unit detecting a face region of a user from the input image;

a gesture region setting unit setting a gesture region, in which a gesture of the user'"'"'s arm occurs, with respect to the detected face region;

an arm detection unit detecting an arm region of the user in the gesture region; and

a gesture determination unit analyzing a position, moving directionality and shape information of the arm region in the gesture region to determine a target gesture of the user,

wherein the gesture region setting unit sets the gesture region which comprises a left upper region which is disposed in an left upper portion of the head region, a right upper region which is disposed in a right upper portion of the head region, a center upper region which is adjacent to the head region and is disposed between the left upper region and the right upper region, a left lower region which is adjacent to the head region and is disposed at a lower portion of the left upper region, and a right lower region which is adjacent to the head region and is disposed at a lower portion of the right upper region, with respect to a position of a head region comprising the face region,

wherein the gesture determination unit determines the target gesture which comprises a waving gesture which repeatedly shakes any one of a right arm and left arm of the user from side to side, a calling gesture which repeatedly shakes any one of the right arm and left arm of the user upward and downward, a raising gesture which maintains a state, where the user is holding up any one of the right and left arms to at least face height, for a certain time, and a stopping gesture which maintains a state, where the user is holding up the right and left arms to face height, for a certain time.

2. The gesture recognition apparatus of claim 1, wherein the gesture determination unit comprises:

a region analyzer determining whether the position of the arm region of the user is in the gesture region to discriminate the target gesture and a noise gesture which corresponds to a daily behavior of the user,

a motion analyzer analyzing the moving directionality of the arm region of the user to discriminate the waving gesture and the calling gesture; and

a shape analyzer analyzing the shape information which comprises a relative length ration and angle of the right and left arms of the user to discriminate the raising gesture and the stopping gesture.

3. The gesture recognition apparatus of claim 2, wherein the region analyzer determines the target gesture and the noise gesture with a lookup table in which the target gesture and the gesture region are configured per item.

4. The gesture recognition apparatus of claim 2, wherein the motion analyzer detects coordinates of end of a hand of the arm region, analyzes a moving direction and speed of the detected coordinates of end of the hand to set a moving direction code value, and determines the waving gesture and the calling gesture on the basis of an accumulated value of the set moving direction code value.

5. The gesture recognition apparatus of claim 2, wherein in a case where the user expresses the raising gesture with the right arm,

the shape analyzer determines the gesture of the user as the raising gesture, when there is no left arm region in the right lower region and a length of a right arm region, which exists across the left lower and upper regions, is greater by two orders or more of magnitude than a length of the left arm region, and

the shape analyzer determines the gesture of the user as the noise gesture, when there is no right arm region in the left lower region and a length of a left arm region in the right upper region is greater than a length of the right arm region.

6. The gesture recognition apparatus of claim 5, wherein the shape analyzer determines the gesture of the user, when a length of a right arm region in the left lower region is greater than approximately 1.3 times a vertical height of the head region and an angle of the right arm is from about 60 degrees to about 135 degrees.

7. The gesture recognition apparatus of claim 2, wherein:

the shape analyzer determines the gesture of the user as the stopping gesture, when a length of a right arm region in the left lower region is the substantially same as a length of a left arm region in the right lower region, and

the shape analyzer determines the gesture of the user as the noise gesture, when the length of the right arm region and the length of left arm region are greater than a predetermined multiple of a height of the head region.

8. The gesture recognition apparatus of claim 7, wherein:

a value of possibility that the gesture of the user is the stopping gesture is the maximum, when an angle of the right arm region is equal to or less than about 90 degrees and an angle of the left arm region is equal to more than about 90 degrees, and

a value of possibility that the gesture of the user is the stopping gesture decreases, when the angle of the right arm region exceeds about 90 degrees and the angle of the left arm region is less than about 90 degrees.

9. The gesture recognition apparatus of claim 7, wherein:

a value of possibility that the gesture of the user is the stopping gesture is the maximum, when a position of the right arm region and a position of the left arm region are separated by a width of the head region with respect to the head region, and

as a displacement in a corresponding position enlarges, a value of possibility that the gesture of the user is the stopping gesture decreases.

10. The gesture recognition apparatus of claim 7, wherein the shape analyzer a value of possibility that the gesture of the user is the stopping gesture on the basis of an angle and position of the right arm region and an angle and position of the left arm region.

11. A gesture recognition method using a robot system which includes a moving means and in which a human-robot interaction technology is implemented, the gesture recognition method comprising:

detecting a face region of a user from an input image which is sequentially inputted;

calculating, executed by a processor, a size of a gesture region, in which a gesture of the user'"'"'s arm occurs, at a certain rate on the basis of a position and size of the detected face region;

acquiring a background subtraction image which comprises an arm region of the user in the calculated gesture region;

detecting the arm region of the user in the gesture region by using the acquired background subtraction image; and

analyzing a position, moving directionality and posture of the arm region in the gesture region to determine a target gesture of the user which comprises a waving gesture, a calling gesture, a raising gesture and a stopping gesture,

wherein the determining of a target gesture comprises;

determining whether an arm region of the user is in the gesture region with a lookup table, in which the target gesture and the gesture region are configured per item, to determine a noise gesture which corresponds to the target gesture and a daily behavior of the user according to a result of the determination;

detecting coordinates of end of a hand of the arm region, in the gesture region;

analyzing a moving direction and speed of the detected coordinates of end of the hand to set a moving direction code value, and determining the waving gesture and the calling gesture on the basis of an accumulated value of the set moving direction code value; and

analyzing the shape information, which comprises a length rate and angle of right and left arms of the user, to determine the raising gesture and the stopping gesture.