System and method for inputing user commands to a processor

• US 8,199,115 B2
• Filed: 03/22/2005
• Issued: 06/12/2012
• Est. Priority Date: 03/22/2004
• Status: Active Grant

• Current Assignee: eyeSight Mobile Technologies Ltd.
• Original Assignee: eyeSight Mobile Technologies Ltd.
• Inventors: Katz, Itay
• Primary Examiner(s): SHERMAN, STEPHEN G
• Application Number:
  US10/593,628
• Publication Number:
  US 20080042981A1
• Time in Patent Office: 2,639 Days
• Field of Search: 345/156, 345/168, 345/169, 345/173, 178 1801- 1809, 178/18.11, 178 1901- 1907, 178/20.01
• US Class Current: 345/173
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A system for inputting operation system (OS) commands to a data processing device. The system comprises a video camera that captures images of a viewing space. A processor detects a predetermined object in the images using an object recognition algorithm not involving background information in an image. One or more image analysis parameters of the object are extracted from the images and one or more motion detection tests are applied. Each motion detection test has an associated OS command, and when a test succeeds, the OS command associated with the test is executed. By not relying on background information in an image, the system of the invention may be used in devices that are moved in use, such as a palm plot, personal digital assistant (PDA), a mobile telephone, a digital camera, and a mobile game machine.

Claims

1. A system for inputting operation system (OS) commands to a data processing device comprising:

(a) a video camera capturing images of a viewing space; and

(b) a processor configured to;

i) detect a predetermined object in one or more images obtained by the camera using a segmentation algorithm;

ii) extract one or more image analysis parameters of the object in the one or more images obtained by the camera; and

iii) for each of one or more motion detection tests;

(I) applying the motion detection test to image analysis parameters extracted during a recent time window; and

(II) executing an operating system command associated with the motion detection test if the motion detection test succeeds,

wherein the segmentation algorithm comprises steps of;

(A) for k=1 to N, where N is a predetermined integer;

binarizing pixels in the image into a set A_k of pixels having an intensity below a kth predetermined intensity and a set B_k of pixels having an intensity above the kth predetermined intensity, the kth predetermined intensity being greater than the (k−

1)th intensity, for k=2 to N;

identifying contiguous sets of pixels in the set A_k and identifying contiguous sets of pixels in the set B_k;

(B) identifying one or more stable sets of pixels in the image, a stable set of pixels being a contiguous set of pixels present in each of the sets A_k for k=M to N, where M is a predetermined constant, or a contiguous set of pixels present in each of the sets B_k for k=M to N, and

(C) applying an object recognition procedure on the stable sets to identify the predetermined object among the stable sets.

2. The system according to claim 1, wherein the predetermined object is a finger or a stylus.

3. The system according to claim 1, wherein one or more of the image analysis parameters is history independent.

4. The system according to claim 1, wherein one or more of the image analysis parameters is history dependent.

5. The system according to claim 1, wherein one or more of the image analysis parameters is selected from the group consisting of:

(a) a location of a tip of the object in an image;

(b) a width of the object in an image;

(c) a length of the object in an image;

(d) an orientation of the object in an image;

(e) a speed of the object at a time the image was obtained by the camera;

(f) a change in the width of the object at a time the image was obtained by the camera;

(g) a rate of rotation of the object at a time the image was obtained by the camera; and

(h) an image analysis parameter having a first value if the object is detected in the image and a second value if the object is not detected in the image.

6. The system according to claim 1, wherein one or more of the motion detection tests is a motion detection test detecting a motion selected from the group consisting of:

(a) during the time window the object approached the camera;

(b) during the time window the object moved away from the camera;

(c) during the time window the object first approached the camera and then moved away from the camera;

(d) during the time window the object disappeared from the viewing space of the camera;

(e) during the time window the object moved in a predetermined path;

(f) during the time window the object rotated;

(g) during the time window the object was stationary;

(h) during the time window the object moved;

(i) during the time window the object performed a flicking motion;

(j) during the time window the object accelerated;

(k) during the time window the object decelerated; and

(l) during the time window the object moved and then stopped.

7. The system according to claim 6, wherein one or more of the motion detection tests is a motion detection test detecting that the object moved in a predetermined path during the time window.

8. The system according to claim 1, wherein one or more of the OS commands is selected from the group consisting of:

(a) depressing a virtual key displayed on a screen;

(b) moving a cursor appearing on a screen;

(c) running on the processor a software application;

(d) turning a light on or off;

(e) turning off the system;

(f) zooming in or out of a picture on a screen;

(g) adjusting a radio or other entertainment device;

(h) adjusting a medical device; and

(i) sending a command to an application.

9. A data processing device comprising the system for inputting operation system (OS) commands according to claim 1.

10. The data processing device according to claim 9, wherein the device is selected from the group consisting of a personal computer (PC), a portable computer, a PDA, a laptop, a mobile telephone, a radio, a digital camera a vehicle, a medical device, a smart home appliance, and a mobile game machine.

11. A method for inputting operation system (OS) commands to a data processing device having a video camera capturing images of a viewing space, comprising:

(a) detecting a predetermined object in one or more images obtained by the camera using a segmentation algorithm;

(b) extracting one or more image analysis parameters of the object in the one or more images obtained by the camera; and

(c) for each of one or more motion detection tests;

i) applying the motion detection test to image analysis parameters extracted during a recent time window; and

ii) executing an operating system command associated with the motion detection test if the motion detection test succeeds,

wherein the segmentation algorithm comprises steps of;

(A) for k=1 to N, where N is a predetermined integer;

binarizing pixels in the image into a set A_k of pixels having an intensity below a kth predetermined intensity and a set B_k of pixels having an intensity above the kth predetermined intensity, the kth predetermined intensity being greater than the (k−

1)th intensity, for k=2 to N;

identifying contiguous sets of pixels in the set A_k and identifying contiguous sets of pixels in the set B_k;

(B) identifying one or more stable sets of pixels in the image, a stable set of pixels being a contiguous set of pixels present in each of the sets A_k for k=M to N, where M is a predetermined constant, or a contiguous set of pixels present in each of the sets B_k for k=M to N, and

(C) applying an object recognition procedure on the stable sets to identify the predetermined object among the stable sets.

12. The method according to claim 11, wherein the predetermined object is one or more fingers or a stylus.

13. The method according to claim 11, wherein one or more of the image analysis parameters is history independent.

14. The method according to claim 11, wherein one or more of the image analysis parameters is history dependent.

15. The method according to claim 11, wherein one or more of the image analysis parameters is selected from the group consisting of:

(a) a location of a tip of the object in an image;

(b) a width of the object in an image;

(c) a length of the object in an image;

(d) an orientation of the object in an image;

(e) a speed of the object at a time the image was obtained by the camera;

(f) a change in the width of the object at a time the image was obtained by the camera;

(g) a rate of rotation of the object at a time the image was obtained by the camera; and

(h) an image analysis parameter having a first value if the object is detected in the image and a second value if the object is not detected in the image.

16. The method according to claim 11, wherein one or more of the motion detection tests is a motion detection test detecting a motion selected from the group consisting of:

(a) during the time window the object approached the camera;

(b) during the time window the object moved away from the camera;

(c) during the time window the object first approached the camera and then moved away from the camera;

(d) during the time window the object disappeared from the viewing space of the camera;

(e) during the time window the object moved in a predetermined path;

(f) during the time window the object rotated;

(g) during the time window the object was stationary;

(h) during the time window the object moved;

(i) during the time window the object performed a flicking motion;

(j) during the time window the object accelerated;

(k) during the time window the object decelerated; and

(l) during the time window the object moved and then stopped.

17. The method according to claim 16, wherein one or more of the motion detection tests is a motion detection test detecting that the object moved in a predetermined path during the time window, wherein the predetermined path traces an alphanumeric character.

18. The method according to claim 11, wherein one or more of the OS commands is selected from the group consisting of:

(a) depressing a virtual key displayed on a screen;

(b) moving a cursor appearing on a screen;

(c) running on the processor a software application;

(d) turning a light on or off;

(e) turning off the system;

(f) zooming in or out of a picture on a screen;

(g) adjusting a radio or other entertainment device;

(h) adjusting a medical device; and

(i) sending a command to an application.

19. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for inputting operation system (OS) commands to a data processing device having a video camera capturing images of a viewing space, the non-transitory method comprising:

(a) detecting a predetermined object in one or more images obtained by the camera using a segmentation algorithm;

(b) extracting one or more image analysis parameters of the object in the one or more images obtained by the camera; and

(c) for each of one or more motion detection tests;

i) applying the motion detection test to image analysis parameters extracted during a recent time window; and

ii) executing an operating system command associated with the motion detection test if the motion detection test succeeds,

wherein the segmentation algorithm comprises steps of;

(A) for k=1 to N, where N is a predetermined integer;

binarizing pixels in the image into a set A_k of pixels having an intensity below a kth predetermined intensity and a set B_k of pixels having an intensity above the kth predetermined intensity, the kth predetermined intensity being greater than the (k−

1)th intensity, for k=2 to N;

identifying contiguous sets of pixels in the set A_k and identifying contiguous sets of pixels in the set B_k;

(B) identifying one or more stable sets of pixels in the image, a stable set of pixels being a contiguous set of pixels present in each of the sets A_k for k=M to N, where M is a predetermined constant or a contiguous set of pixels present in each of the sets B_k for k=M to N, and

(C) applying an object recognition procedure on the stable sets to identify the predetermined object among the stable sets.

20. A computer program product comprising a computer useable medium having computer readable program code embodied therein for inputting operation system (OS) commands to a data processing device having a video camera capturing images of a viewing space, the non-transitory computer program product comprising:

computer readable program code for causing the computer to detect a predetermined object in one or more images obtained by the camera using a segmentation algorithm;

computer readable program code for causing the computer to extract one or more image analysis parameters of the object in the one or more images obtained by the camera; and

computer readable program code for causing the computer, for each of one or more motion detection tests;

to apply the motion detection test to image analysis parameters extracted during a recent time window; and

to execute an operating system command associated with the motion detection test if the motion detection test succeeds,

wherein the segmentation algorithm comprises steps of;

(A) for k=1 to N, where N is a predetermined integer;

binarizing pixels in the image into a set A_k of pixels having an intensity below a kth predetermined intensity and a set B_k of pixels having an intensity above the kth predetermined intensity, the kth predetermined intensity being greater than the (k−

1)th intensity, for k=2 to N;

identifying contiguous sets of pixels in the set A_k and identifying contiguous sets of pixels in the set B_k;

(B) identifying one or more stable sets of pixels in the image, a stable set of pixels being a contiguous set of pixels present in each of the sets A_k for k=M to N, where M is a predetermined constant or a contiguous set of pixels present in each of the sets B_k for k=M to N, and

(C) applying an object recognition procedure on the stable sets to identify the predetermined object among the stable sets.