Computer vision gesture based control of a device

• US 8,693,732 B2
• Filed: 07/10/2013
• Issued: 04/08/2014
• Est. Priority Date: 10/13/2009
• Status: Expired due to Fees

• Current Assignee: PointGrab Ltd.
• Original Assignee: PointGrab Ltd.
• Inventors: Eilat, Eran, Kaplan, Amir, Perski, Haim
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US13/939,055
• Publication Number:
  US 20140043234A1
• Time in Patent Office: 272 Days
• Field of Search: 382/103, 382/181, 382/218
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A system and method are provided for controlling a device based on computer vision. Embodiments of the system and method of the invention are based on receiving a sequence of images of a field of view; detecting movement of at least one object in the images; applying a shape recognition algorithm on the at least one moving object; confirming that the object is a user hand by combining information from at least two images of the object; and tracking the object to control the device.

Claims

1. A method for controlling a device based on computer vision, the method comprising

obtaining a sequence of images of a field of view, each of the images comprising at least one hand;

applying a shape recognition algorithm on the images to identify the at least one hand;

detecting, from the images, movement of the hand;

determining a size of the moving hand relative to the image frame which includes the hand;

and moving a displayed object according to movement of the hand while normalizing movement of the displayed object according to the determined size of the hand.

2. The method according to claim 1, comprising:

obtaining an indication of a distance of the hand from a camera, the indication of a distance of the hand from the camera being based on the size of the hand relative to the image which includes the hand; and

wherein controlling movement of the object on the display of the device is based on the indication of a distance of the hand from the camera.

3. The method according to claim 1, wherein the object is a cursor.

4. The method according to claim 2, further comprising normalizing cursor sensitivity to the size of the hand relative to the image which includes the hand.

5. The method according to claim 1, wherein the images of the field of view are 2D images.

6. The method according to claim 1, wherein a change in the sequence of images of the size of the hand is used as a basis for detecting movement of the hand.

7. The method according to claim 1, wherein the camera is a forward-facing camera.