GESTURE RECOGNITION SYSTEMS AND RELATED METHODS

• US 20100295783A1
• Filed: 05/20/2010
• Published: 11/25/2010
• Est. Priority Date: 05/21/2009
• Status: Active Grant

• Current Assignee: Golden Edge Holding Corporation
• Original Assignee: Edge Technologies, Inc
• Inventors: King, Joshua T., El Dokor, Tarek A.
• Granted Patent: US 9,417,700 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for performing gesture recognition. In one embodiment of the invention, the method includes the steps of receiving one or more raw frames from one or more cameras, each of the one or more raw frames representing a time sequence of images, determining one or more regions of the one or more received raw frames that comprise highly textured regions, segmenting the one or more determined highly textured regions in accordance textured features thereof to determine one or more segments thereof, determining one or more regions of the one or more received raw frames that comprise other than highly textured regions, and segmenting the one or more determined other than highly textured regions in accordance with color thereof to determine one or more segments thereof. One or more of the segments are then tracked through the one or more raw frames representing the time sequence of images.

Claims

1. A method for performing gesture recognition, comprising the steps of:

receiving one or more raw frames from one or more cameras, each of the one or more raw frames representing a time sequence of images;

segmenting the one or more received raw frames; and

tracking one or more of the segments through the one or more raw frames representing the time sequence of images.

2. The method of claim 1, wherein the segmenting further comprises the steps of:

determining one or more regions of the one or more received raw frames that comprise highly textured regions;

segmenting the one or more determined highly textured regions in accordance textured features thereof to determine one or more segments thereof;

determining one or more regions of the one or more received raw frames that comprise other than highly textured regions; and

segmenting the one or more determined other than highly textured regions in accordance with color thereof to determine one or more segments thereof.

3. The method of claim 1, wherein the segmenting further comprises the steps of:

determining position changes of one or more portions between the received raw frames indicating motion of a portion thereof; and

segmenting the received raw frames in accordance with these determined position changes.

4. The method of claim 1, wherein raw images are received from two cameras, and further comprising the step of tracking one or more of the segments through corresponding ones of the one or more raw frames received from each of the two cameras.

5. The method of claim 4, further comprising the step of determining one or more disparities between the corresponding one of the one or more frames in accordance with the segments tracked therein.

6. The method of claim 5, wherein the one or more disparities are determined in accordance with the application of disparity space decomposition.

7. The method of claim 1, further comprising the steps of:

8. The method of claim 1, further comprising the steps of:

determining a particular segment to be tracked;

determining whether the particular segment is providing a static gesture or a dynamic gesture in accordance with a determination of change of position of the particular segment in the time sequence of images.

9. The method of claim 8, further comprising the step of applying a deep belief network for gesture recognition if it is determined that the particular segment is providing a static gesture.

10. The method of claim 8, further comprising the step of applying a disparity space decomposition procedure for gesture recognition if it is determined that the particular segment is providing a dynamic gesture.

11. The method of claim 8, further comprising the steps of:

associating the gesture with a predetermined action; and

performing the action upon recognition of the gesture.

12. A method for clustering portions of an image in accordance with a segmentation process, comprising the steps of:

defining a cluster map comprising pixels associated with at least a portion of an image;

assigning each pixel in the cluster map a unique numerical identifier corresponding to a spatial coordinate thereof;

defining a two dimensional window;

assigning a lowest cluster number to all of the pixels within the two dimensional window that do not include indications of a changed segment;

comparing a cluster number associated with each pixel with the spatial coordinate thereof;

if it is determined that the cluster number associated with the pixel does not match the spatial coordinate, tracking the cluster number back to a previous pixel whose spatial coordinate actually matches that cluster number;

writing to the cluster map for the pixel being evaluated the matching spatial coordinate;

comparing neighboring pixels and storing the lowest cluster number of the neighboring pixels.

13. The method of claim 12, wherein the indications of a changed segment include changes in at least one of, intensity, color, or other defined pixel characteristic.

14. The method of claim 12, wherein the spatial coordinates are assigned to consecutive pixels.

15. The method of claim 12, wherein the step of tracking the cluster number back to a previous pixel further comprises the steps of:

linking to a prior pixel associated with the cluster number associated with a current pixel;

determining whether the cluster number of that prior pixel matches the spatial coordinate thereof;

if the cluster number of that prior pixel does match the spatial coordinate thereof, writing the cluster number to the cluster number of the current pixel; and

if the cluster number of that prior pixel does not match the spatial coordinate thereof, perform the linking step to another prior pixel.

16. A system for performing gesture recognition, comprising:

an input port for receiving one or more raw frames from one or more cameras, each of the one or more raw frames representing a time sequence of images; and

a processor for segmenting the one or more received raw frames, and tracking one or more of the segments through the one or more raw frames representing the time sequence of images.

17. The system of claim 16, wherein the processor is further adapted to segment the one or more received frames by determining one or more regions of the one or more received raw frames that comprise highly textured regions, segmenting the one or more determined highly textured regions in accordance textured features thereof to determine one or more segments thereof, determining one or more regions of the one or more received raw frames that comprise other than highly textured regions, and segmenting the one or more determined other than highly textured regions in accordance with color thereof to determine one or more segments thereof.

18. The system of claim 16, further comprising two cameras for providing the raw images.