METHOD FOR TOUCHLESS CONTROL OF A DEVICE

• US 20130343607A1
• Filed: 06/02/2013
• Published: 12/26/2013
• Est. Priority Date: 06/20/2012
• Status: Abandoned Application

• Current Assignee: PointGrab Ltd.
• Original Assignee: PointGrab Ltd.
• Inventors: WILF, Saar, Perski, Haim
• Application Number:
  US13/907,925
• Publication Number:
  US 20130343607A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A system and method for computer vision based control of a device may include using a virtual line passing through an area of a user'"'"'s eyes and through a user'"'"'s hand (or any object controlled by the user) to a display of the device, to control the device.

Claims

1. A method for computer vision based control of a device, the device in communication with a display, the method comprising

obtaining an image of a field of view, the field of view comprising a user;

detecting within the image an area of the user'"'"'s face and at least a part of the user'"'"'s hand;

detecting a point on the display, the point being on a virtual line which is dependent on a location of the user'"'"'s face and on a location of the part of the user'"'"'s hand; and

controlling the device based on the point on the display.

2. The method of claim 1 comprising detecting an area of the user'"'"'s eyes and wherein the virtual line is dependent on the location of the area of the user'"'"'s eyes.

3. The method of claim 1 comprising detecting a dominant eye of the user and wherein the virtual line is dependent on the location of the dominant eye of the user.

4. The method of claim 1 wherein detecting at least a part of the user'"'"'s hand comprises detecting a shape of the of the user'"'"'s hand or part of the user'"'"'s hand and further comprising controlling the device based on the point on the display and based on the shape of the user'"'"'s hand or part of the user'"'"'s hand.

5. The method of claim 4 comprising controlling the device based on the point on the display and based on a movement of the user'"'"'s hand or part of the user'"'"'s hand.

6. The method of claim 1 wherein the location of the user'"'"'s hand or part of the user'"'"'s hand comprises a point in between a tip of a thumb and a tip of another finger almost touching the thumb.

7. The method of claim 1 wherein controlling the device comprises controlling content displayed on the display.

8. The method of claim 1 comprising displaying an indication of the point on the display at the location of the point on the display.

9. The method of claim 1 comprising:

determining a distance of the part of the user'"'"'s hand from a camera used to obtain the image of the field of view; and

using the distance of the part of the hand from the camera to calculate the point on the display.

10. The method of claim 1 comprising

determining a distance of the area of the user'"'"'s face from a camera used to obtain the image of the field of view;

and

using the distance of the area of the user'"'"'s face from the camera to calculate the point on the display.

11. A method for computer vision based control of a device, the device in communication with a display and a camera, the method comprising

capturing by the camera an image comprising a user;

detecting within the image an area of the user'"'"'s face and at least a part of the user'"'"'s hand;

calculating a virtual line passing through the area of the user'"'"'s face and through the part of the user'"'"'s hand to the display, the virtual line passing through the display at a point on the display; and

controlling the device based on the point on the display.

12. The method of claim 11 comprising controlling the device based on the point on the display and based on a posture of the part of the user'"'"'s hand.

13. The method of claim 11 comprising applying shape recognition algorithms to detect at least part of the user'"'"'s hand.

14. The method of claim 11 wherein controlling the device comprises manipulating content on the display according to movement of the part of the user'"'"'s hand.

15. The method of claim 11 comprising determining a size of the part of the user'"'"'s hand or of the user'"'"'s face and using the size of the part of the user'"'"'s hand or user'"'"'s face to calculate the point on the display.

16. The method of claim 11 wherein the camera is a 2D camera.

17. A system for computer vision based control of a device, the system comprising

an imager to obtain an image of a user;

a display in communication with the device;

a processor to;

detect within the image an area of the user'"'"'s face and at least a user'"'"'s hand;

calculate a point on the display, the point being on a virtual line which is dependent on a location of the user'"'"'s face and on a location of the part of the user'"'"'s hand; and

control the device based on the point on the display.

18. The system of claim 17 wherein the imager is a 2D camera.

19. The system of claim 17 wherein the imager is positioned at a known position relative to the display.

20. The system of claim 17 wherein the processor to detect within the image an area of the user'"'"'s face and at least a user'"'"'s hand applies shape recognition algorithms on the image.