GESTURE BASED USER INTERFACE

• US 20150009124A1
• Filed: 02/07/2014
• Published: 01/08/2015
• Est. Priority Date: 07/08/2013
• Status: Active Grant

• Current Assignee: Augmenta Oy, Augumenta Ltd.
• Original Assignee: Augumenta Ltd.
• Inventors: ANTONIAC, Peter
• Granted Patent: US 9,207,771 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A system and method for recognition of hand gesture in computing devices. The system recognizes a hand of a user by identifying a predefined first gesture and further collects visual information related to the hand identified on the basis of the first predefined gesture. The visual information is used to extract a second gesture (and all other gestures after the second) from the video/image captured by the camera and finally interpreting the second gesture as a user input to the computing device. The system enables gesture recognition in various light conditions and can be operated by various user hands including the ones wearing gloves.

Claims

1. A portable computing device including a camera configured to capture one or more images, a memory and a central processing unit, the central processing unit configured to analyze the image and detect and identify a hand gesture of a user in the image, the central processing unit comprising computer readable code means, which when executed on the central processing unit are configured to:

determine that the captured image includes a predefined activation gesture;

detect a second gesture in the captured image;

identify a command corresponding to the second gesture; and

executing the command to perform a function in the portable computing device corresponding to the command.

2. The portable computing device of claim 1, wherein the central processing unit is configured to separate the first gesture from a background of the image and extract information from the first gesture.

3. The portable computing device of claim 2, wherein the central processing unit is configured to use the extracted information to analyze the second gesture to identify the command.

4. The portable computing device of claim 1, wherein the captured image is a real-time image.

5. The portable computing device of claim 1, wherein the captured image is a sequence of real-time images.

6. The portable computing device of claim 1, wherein the captured image is a video.

7. The portable computing device of claim 1, wherein the central processing unit is configured to determine that the captured image includes the predefined activation gesture by:

detecting a first hand gesture in the captured image;

comparing the first hand gesture in the captured image to a database of stored hand gestures; and

determining that the first hand gesture corresponds to a pre-defined activation gesture.

8. The portable computing device according to claim 7, wherein determining that the first hand gesture corresponds to the pre-defined activation gesture comprises matching the first hand gesture to a registered hand gesture in the database of stored hand gestures, and wherein the central processing unit is configured to authenticate the user of the portable computing device when the first hand gesture is matched to a registered hand gesture.

9. The portable computing device of claim 1, wherein the extracted information comprises a color of the first hand gesture, a location of the first hand gesture in the image, a pose of the first hand gesture, a size of the first hand gesture, a shape of the first hand gesture or a motion of the first hand gesture.

10. The portable computing device of claim 1, wherein the predefined activation gesture is in the form of a palm of a user'"'"'s hand.

11. The portable computing device of claim 1, wherein the captured image includes an image of a hand of the user.

12. The portable computing device of claim 1, wherein the portable computing device comprises a mobile communication device, a computer, a head mounted device or a smart phone.

13. The portable computing device of claim 1, wherein the central processing unit is configured to:

detect a shape in the captured image

detect a background in the captured image;

separate the detected shape from the background;

extracting information from the separated shape, the information comprising at least a color, location and size of the separated shape;

using the extracted information to analyze the detected second gesture to identify the command.

14. The portable computing device of claim 13, wherein the central processing unit is configured to separate the detected shape from the background using a motion vector, edge detection, shape detection and/or color detection.

15. The portable computing device of claim 1, further comprising a feedback device, wherein the central processing unit is configured to cause the feedback device to provide one or more of a tactile, audio or visual feedback to the user prior to executing the command.

16. A method of interpreting a hand gesture in an image captured by an image capturing device communicatively coupled to a computing device, comprising using a processor of the computing device to:

detect a first hand gesture in the captured image;

determine that the first hand gesture corresponds to an activation gesture;

detect a subsequent hand gesture in the captured image;

identify a command corresponding to the subsequent hand gesture, the command being configured to execute an action in the computing device when the command is triggered; and

execute the command in the computing device.

17. The method of claim 16, wherein determining that the first hand gesture is the activation gesture comprises matching the first hand gesture to hand gesture images stored in a database of pre-defined hand gestures.

18. The method of claim 16, wherein after the first hand gesture is determined to be the activation gesture, the processor is configured to separate the first hand gesture from a background of the captured image and extract information from the first hand gesture that is used to analyze the second gesture to identify the command.

19. The method of claim 18, wherein the information comprises a color of the first hand gesture, a location of the first hand gesture in the image, a size of the first hand gesture or a shape of the first hand gesture.

20. The method of claim 18, wherein the processor is configured to:

detect a shape in the captured image;

detect a background in the captured image;

separate the detected shape from the background;

extracting information from the separated shape, the information comprising at least a color, location and size of the separated shape;

using the extracted information to analyze the detected second gesture to identify the command.