Video hand image computer interface

• US 6,252,598 B1
• Filed: 07/03/1997
• Issued: 06/26/2001
• Est. Priority Date: 07/03/1997
• Status: Expired due to Term

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Segen, Jakub
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): Nguyen, Thomas T.
• Application Number:
  US08/887,765
• Time in Patent Office: 1,454 Days
• Field of Search: 345/157, 345/158, 345/175, 345/358, 382/181, 382/170
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An interface method and apparatus using video images of hand gestures. A video signal having a frame image containing regions is input to a processor. A plurality of regions in the frame are defined and screened to locate an image of a hand in one of the regions. The hand image is processed to locate extreme curvature values, such as peaks and valleys, corresponding to predetermined hand positions and gestures. The number of peaks and valleys are then used to identify and correlate a predetermined hand gesture to the hand image for effectuating a particular computer operation or function.

Claims

1. A method of controlling a computer using images of predetermined hand gestures identified in frames of a video signal, comprising the steps of:

inputting to a processor a current video frame of a video signal containing a frame image;

defining a plurality of regions in the current video frame image with each defined region being represented by a corresponding boundary of pixels;

screening each of the corresponding boundaries of the defined regions to identify one of said defined regions containing an image of a hand;

locating extreme concave and convex curvature values on the corresponding boundary of said identified one region and defining said concave curvature values as valleys and said convex curvature values as peaks;

determining a count of the defined peaks and valleys and correlating the count with prestored data associated with one of a plurality of predetermined hand gestures to identify in the one region of the current video frame a particular one of the plural hand gestures present in the current video frame; and

utilizing the identified hand gesture for operating the computer to perform a computer function associated with the identified hand gesture.

2. The method of claim 1, wherein said current video frame is comprised of a plurality of pixels each having an intensity value, and wherein said defining step comprises calculating one of a difference and an absolute difference between the intensity value of each pixel at a specific location and a reference value at said specific location to provide a calculated value for each pixel, and comparing the calculated value to a threshold constant.

3. The method of claim 2, wherein said video signal comprises a current frame and a previous-in-time frame, wherein said previous-in-time frame is comprised of a plurality of pixels each having an intensity value, and wherein said reference value comprises the intensity value of a pixel in said previous-in-time frame having said specific location.

4. The method of claim 1, wherein said video signal comprises a current frame and a previous-in-time frame, with each of said frames comprised of pixels having color values and wherein said defining step comprises grouping pixels having similar color values into common regions.

5. The method of claim 1, wherein each of the corresponding boundaries has a length, and wherein said screening step comprises comparing the boundary lengths to a preset range and selecting a boundary having a length within said preset range as the boundary of said identified one region.

6. The method of claim 1, wherein said determining step comprises detecting, in said defined region containing an image of a hand, an index finger and using the peak closest to said detected index finger to identify said particular one of the plural hand gestures.

7. A method of controlling a computer using images of predetermined hand gestures identified in frames of a video signal, comprising the steps of:

inputting to a processor a current video frame of a video signal containing a frame image;

defining a plurality of regions in the current video frame image with each defined region being represented by a corresponding boundary of pixels;

screening each of the corresponding boundaries of the defined regions to identify one of said defined regions containing an image of a hand;

locating extreme concave and convex curvature values on the corresponding boundary of said identified one region and defining said concave curvature values as valleys and said convex curvature values as peaks;

determining a count of the defined peaks and valleys and correlating the count with prestored data associated with one of a plurality of predetermined hand gestures to identify in the one region of the current video frame a particular one of the plural hand gestures present in the current video frame, said determining step comprising detecting, in said defined region containing an image of a hand, an index finger and using the peak closest to said detected index finger to identify said particular one of the plural hand gestures; and

utilizing the identified hand gesture for operating the computer to perform a computer function associated with the identified hand gesture.

8. The method of claim 7, wherein said current video frame is comprised of pixels each having an intensity value, and wherein said defining step comprises calculating one of a difference and an absolute difference between the intensity value of each pixel at a specific location and a reference value at said specific location to provide a calculated value for each pixel, and comparing the calculated value to a threshold constant.

9. The method of claim 8, wherein said video signal comprises a current frame and a previous-in-time frame, wherein said previous-in-time frame is comprised of a plurality of pixels each having an intensity value, and wherein said reference value comprises the intensity value of a pixel in said previous-in-time frame having said specific location.

10. The method of claim 7, wherein said video signal comprises a current frame and a previous-in-time frame, with each of said frames comprised of pixels having color values and wherein said defining step comprises grouping pixels having similar color values into common regions.

11. The method of claim 7, wherein each of the corresponding boundaries has a length, and wherein said screening step comprises comparing the boundary lengths to a preset range and selecting a boundary having a length within said preset range as the boundary of said identified one region.

12. A device for controlling a computer using images of objects identified in frames of a video signal, comprising:

a processor;

means for inputting a current video frame of a video signal containing a frame image to said processor;

defining means for defining a plurality of regions in the current video frame image with each defined region being represented by a corresponding boundary of pixels;

screening means for screening each of the corresponding boundaries of the defined regions to identify one of said defined regions containing an image of an object;

locating means for locating extreme concave and convex curvature values on the corresponding boundary of said identified one region and defining said concave curvature values as valleys and said convex curvature values as peaks;

determining means for determining a count of the defined peaks and valleys and comparing the count with prestored data associated with one of a plurality of objects to identify in the one region of the current video frame a particular one of the objects present in the current video frame; and

controlling means for utilizing the identified object for operating the computer to perform a computer function associated with the identified object.