APPARATUS AND METHOD FOR RECOGNIZING USER'S GESTURE FOR CARRYING OUT OPERATION OF VEHICLE

• US 20150186707A1
• Filed: 10/15/2014
• Published: 07/02/2015
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Joo, Sihyun
• Granted Patent: US 9,330,308 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

An apparatus for recognizing a user'"'"'s gesture for carrying out an operation of a vehicle may include: an image generator that generates an object image by photographing a user'"'"'s hand; a feature point candidate detector that generates a hand image including a hand region by eliminating a background of the object image; a palm detector that detects a palm region by performing a morphology operation on the hand region; a finger detector that detects a finger region by eliminating the palm region from the hand region; a feature point detector that detects the finger valleys located within the palm region and the fingertips located within the finger region as feature points; and a gesture recognizer that recognizes a number of unfolded fingers based on the hand model, and recognizes the user'"'"'s gesture by selecting a category that corresponds to the number of unfolded fingers among a plurality of categories.

Claims

1. An apparatus for recognizing a user'"'"'s gesture for carrying out an operation of a vehicle, comprising:

an image generator configured to generate an object image by photographing a user'"'"'s hand;

a feature point candidate detector configured to generate a hand image including a hand region by eliminating a background of the object image, and detect a center point of a wrist and feature point candidates including fingertips and finger valleys;

a palm detector configured to detect a palm region by performing a morphology operation on the hand region, and detect a center point of the palm region;

a finger detector configured to detect a finger region by eliminating the palm region from the hand region;

a feature point detector configured to detect the finger valleys located within the palm region and the fingertips located within the finger region as feature points, and generate a hand model by using the center point of the wrist, the center point of the palm region, and the feature points; and

a gesture recognizer configured to recognize a number of unfolded fingers based on the hand model, and recognize the user'"'"'s gesture by selecting a category that corresponds to the number of unfolded fingers among a plurality of categories.

2. The apparatus of claim 1, wherein the feature point candidate detector converts the hand image to a binary image.

3. The apparatus of claim 2, wherein the feature point candidate detector normalizes the hand region after converting the hand image to the binary image or after detecting the feature point candidates.

4. The apparatus of claim 1, wherein the feature point candidate detector detects an outline of the user'"'"'s hand, and detects the fingertips or the finger valleys based on a curvature of the outline of the user'"'"'s hand.

5. The apparatus of claim 4, wherein the feature point candidate detector detects points where the curvature is greater than a predetermined value as the fingertips or the finger valleys.

6. The apparatus of claim 4, wherein the palm detector performs an erosion operation along the outline of the user'"'"'s hand to eliminate the finger region, performs a dilation operation along an outline of the palm region remaining after being partially eliminated by the erosion operation to dilate the palm region, and detects a center point of the dilated palm region by using an ellipse matching method or a least squares method.

7. The apparatus of claim 6, wherein finger detector detects the finger region by eliminating the dilated palm region from the hand region.

8. The apparatus of claim 1, wherein the feature point detector detects the feature point candidates located within the palm region or the finger region by primarily filtering the feature point candidates.

9. The apparatus of claim 8, wherein the feature point detector detects the feature points by secondarily filtering the primarily filtered feature point candidates based on distances and angles between the center point of the wrist, the center point of the palm region, and the primarily filtered feature point candidates.

10. The apparatus of claim 1, wherein the gesture recognizer selects a hand pose that corresponds to the hand model in the selected category.

11. A method for recognizing a user'"'"'s gesture for carrying out an operation of a vehicle, comprising:

generating a hand image by eliminating a background of an object image;

converting the hand image to a binary image including a hand region;

detecting a center point of a wrist and feature point candidates including fingertips and finger valleys within the hand region;

detecting a palm region and a center point of the palm region by performing a morphology operation on the hand region;

detecting a finger region by eliminating the palm region from the hand region;

detecting the feature point candidates located within the palm region or the finger region by primarily filtering the feature point candidates;

detecting feature points by secondarily filtering the primarily filtered feature point candidates based on distances and angles between the center point of the wrist, the center point of the palm region, and the primarily filtered feature point candidates; and

generating a hand model by using the center point of the wrist, the center point of the palm region, and the feature points.

12. The method of claim 11, further comprising normalizing the hand region after the converting the hand image or after the detecting the center point of the wrist and feature point candidates.

13. The method of claim 11, wherein the detecting the center point of the wrist and the feature point candidates comprises:

detecting an outline of a user'"'"'s hand; and

calculating a curvature of the outline of the user'"'"'s hand.

14. The method of claim 13, wherein the detecting the center point of the wrist and the feature point candidates further comprises detecting points where the curvature is greater than a predetermined value as the fingertips or the finger valleys.

15. The method of claim 13, wherein the detecting the palm region and the center point of the palm region by performing the morphology operation on the hand region comprises:

performing an erosion operation along the outline of the user'"'"'s hand to eliminate the finger region;

performing an dilation operation along an outline of the palm region remaining after being partially eliminated by the erosion operation to dilate the palm region; and

detecting the center point of the dilated palm region by using an ellipse matching method or a least squares method.

16. The method of claim 15, wherein the detecting the finger region comprises eliminating the dilated palm region from the hand region.

17. The method of claim 11, further comprising:

recognizing a number of unfolded fingers based on the hand model;

selecting a category that corresponds to the number of unfolded fingers among a plurality of categories; and

recognizing the user'"'"'s gesture by selecting a hand pose that corresponds to the hand model in the selected category.

18. A non-transitory computer readable medium containing program instructions executed by a processor or controller, the computer readable medium comprising:

program instructions that generate a hand image by eliminating a background of an object image;

program instructions that convert the hand image to a binary image including a hand region;

program instructions that detect a center point of a wrist and feature point candidates including fingertips and finger valleys within the hand region;

program instructions that detect a palm region and a center point of the palm region by performing a morphology operation on the hand region;

program instructions that detect a finger region by eliminating the palm region from the hand region;

program instructions that detect the feature point candidates located within the palm region or the finger region by primarily filtering the feature point candidates;

program instructions that detect feature points by secondarily filtering the primarily filtered feature point candidates based on distances and angles between the center point of the wrist, the center point of the palm region, and the primarily filtered feature point candidates; and

program instructions that generate a hand model by using the center point of the wrist, the center point of the palm region, and the feature points.