Vision-based user interface and related method

• US 8,897,490 B2
• Filed: 03/23/2011
• Issued: 11/25/2014
• Est. Priority Date: 03/23/2011
• Status: Active Grant

• Current Assignee: Arcsoft Corporation Limited
• Original Assignee: ArcSoft, Inc.
• Inventors: Li, JianFeng, Mei, Li, Wang, Jin, Zhang, ZhiWei
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US13/069,401
• Publication Number:
  US 20120242566A1
• Time in Patent Office: 1,343 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A vision-based user interface includes an image input unit for capturing frame images, an image processor for recognizing a posture in at least one of the captured frame images, and generating a recognized gesture according to the posture, and a control unit for generating a control command corresponding to the recognized gesture.

Claims

1. A vision-based user interface comprising:

an image input unit for capturing frame images;

an image processor for recognizing a posture in at least one of the captured frame images, and generating a recognized gesture according to the posture;

the image processor comprising;

a hand area localization unit for limiting range for recognizing the posture to a limited range of the at least one captured frame images;

a feature extraction unit for obtaining candidate color blocks within the limited range, fitting the candidate color blocks into rectangular windows, and extracting variant features from each rectangular window; and

a model matching unit for transferring the variant features into parallel classifiers to generate a matching calculation result, and determining each rectangular window to be a posture-containing window or a none window, wherein each parallel classifier has a pre-built model corresponding to at least one predefined posture; and

a control unit for generating a control command corresponding to the recognized gesture.

2. The vision-based user interface of claim 1, wherein the image processor further comprises:

an area merging unit for merging two or more posture-containing windows into a merged block according to distribution of the two or more posture-containing windows.

3. The vision-based user interface of claim 2, wherein the image processor further comprises:

a feature tracking unit for searching possible positions in the current frame image according to features extracted from pre-gestures to generate a position; and

a motion prediction unit for predicting the position according to previous velocity of a hand over the at least one captured frame images.

4. The vision-based user interface of claim 3, wherein the image processor further comprises:

an information fusion unit for combining outputs of the feature tracking unit and the motion prediction unit.

5. The vision-based user interface of claim 4, wherein the image processor further comprises:

a gesture recognition unit electrically connected to the hand posture detection unit and the hand gesture tracking unit for outputting the recognized gesture according to a hand posture type outputted by the hand posture detection unit and a hand movement outputted by the hand gesture tracking unit.

6. A method of controlling a device according to a gesture, the gesture performed by a user, the method comprising:

capturing frame images by an image input unit;

limiting range for recognizing the posture to a limited range of the at least one captured frame images;

obtaining candidate color blocks within the limited range;

fitting the candidate color blocks into rectangular windows;

extracting variant features from each rectangular window;

transferring the variant features into parallel classifiers to generate a matching calculation result, wherein each parallel classifier has a pre-built model corresponding to at least one predefined posture;

determining each rectangular window to be a posture-containing window or a none window;

recognizing a posture in at least one of the captured frame images by an image processor;

generating a recognized gesture according to the posture by the image processor; and

generating a control command corresponding to the recognized gesture by a control unit.

7. The method of claim 6, further comprising:

merging two or more posture-containing windows into a merged block according to distribution of the two or more posture-containing windows.

8. The method of claim 7, further comprising:

(a) searching possible positions in the current frame image according to features extracted from pre-gestures to generate a position; and

(b) predicting the position according to previous velocity of a hand over the at least one captured frame images.

9. The method of claim 8, further comprising:

combining results of steps (a) and (b) to generate a tracking result.

10. The method of claim 9, further comprising:

outputting the recognized gesture according to a hand posture type and a hand movement.