Gesture keyboard method and apparatus

• US 8,928,590 B1
• Filed: 05/15/2012
• Issued: 01/06/2015
• Est. Priority Date: 04/03/2012
• Status: Active Grant

• Current Assignee: Edge Technologies, Inc
• Original Assignee: Edge Technologies, Inc
• Inventors: El Dokor, Tarek
• Primary Examiner(s): Johnson, Allison
• Assistant Examiner(s): LAM, VINH TANG
• Application Number:
  US13/471,494
• Time in Patent Office: 966 Days
• Field of Search: 345/168, 340/407.1, 341/22, 708/142
• US Class Current: 345/168
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0219 Special purpose keyboards
• CPC Class Codes: G06F 3/0233 Character input methods
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A gesture-enabled keyboard and method are defined. The gesture-enabled keyboard includes a keyboard housing including one or more keyboard keys for typing and a pair of stereo camera sensors mounted within the keyboard housing, a field of view of the pair of stereo camera sensors projecting substantially perpendicularly to the plane of the keyboard housing. A background of the field of view is updated when one or more alternative input devices are in use. A gesture region including a plurality of interaction zones and a virtual membrane defining a region of transition from one of the plurality of interaction zones to another of the plurality of interaction zones is defined within the field of view of the pair of stereo camera sensors. Gesture interaction is enabled when one or more gesture objects are positioned within the gesture region, and when one or more alternative input devices are not in use.

Claims

1. A gesture-enabled keyboard comprising:

a keyboard housing comprising one or more keyboard keys for typing; and

a pair of stereo camera sensors mounted within the keyboard housing, a field of view of the pair of stereo camera sensors projecting substantially perpendicularly to the plane of the keyboard housing, each of the pair of stereo cameras being selected from the group of a fisheye lens camera and a wide angle lens camera, each of the stereo camera sensors providing a distortion to one or more objects located within a field of view of each of the stereo camera sensors, and wherein the distortion of each of the one or more objects is employed to determine at least a location of each of the one or more objects.

2. The gesture-enabled keyboard of claim 1, wherein a gesture region is defined within the field of view of the pair of stereo camera sensors.

3. The gesture-enabled keyboard of claim 2, wherein the gesture region further comprises a plurality of interaction zones.

4. The gesture-enabled keyboard of claim 3, further comprising a virtual membrane defining a region of transition from one of the plurality of interaction zones to another of the plurality of interaction zones.

5. The gesture-enabled keyboard of claim 4, wherein movement of a gesture object from one of the plurality of interaction zones to another of the plurality of interaction zones through the virtual membrane defining the region of transition therebetween defines a touch interaction.

6. The gesture-enabled keyboard of claim 5, wherein the one or more gesture objects comprise the hands of a user, and wherein the hands of the user are not within the gesture region when typing on the one or more keyboard keys.

7. The gesture-enabled keyboard of claim 2, wherein the gesture region is defined as beginning a predetermined distance above the one or more keyboard keys.

8. The gesture-enabled keyboard of claim 7, wherein gesture interaction is enabled when one or more gesture objects are positioned within the gesture region, and wherein gesture interaction is disabled when no gesture objects are within the gesture region.

9. The gesture-enabled keyboard of claim 2, wherein gesture interaction is disabled when one or more alternative input devices are in use, and where gesture interaction is enabled when the one or more alternative input devices are not in use.

10. The gesture-enabled keyboard of claim 9, wherein gesture interaction is further enabled only when one or more gesture objects are positioned within the gesture region, and the one or more alternative input devices is not in use.

11. The gesture-enabled keyboard of claim 10, wherein a background image in the field of view of the pair of stereo cameras is updated when the one or more alternative input devices is in use, and the background image in the field of view of the pair of stereo cameras is not updated when the one or more alternative input devices is not in use.

12. The gesture-enabled keyboard of claim 9, wherein the one or more alternative input devices comprise one or more of a computer mouse, trackball, pointer and input pad.

13. The gesture-enabled keyboard of claim 1, further comprising a pair of micro-electronic machines, each of the micro-electronic machines being coupled with, and adapted to adjust a position of a corresponding one of the pair of stereo cameras.

14. The gesture-enabled keyboard of claim 1, further comprising a pair of mirror systems, each of the pair of mirror systems being coupled with, and adapted to direct light received by a corresponding one of the pair of stereo cameras to a sensor.

15. The gesture-enabled keyboard of claim 14, further comprising one or more micro-electric machines for adjusting a position of one or more corresponding mirrors in the pair of mirror systems.

16. The gesture-enabled keyboard of claim 14, wherein the sensor comprises a single sensor element.

17. A method for inputting a command to a computing system, comprising the steps of:

providing a keyboard further comprising a keyboard housing comprising one or more keyboard keys for typing and a pair of stereo camera sensors mounted within the keyboard housing, a field of view of the pair of stereo camera sensors projecting substantially perpendicularly to the plane of the keyboard housing, each of the pair of stereo camera sensors being selected from the group of a fisheye lens camera and a wide angle lens camera, each of the stereo camera sensors providing a distortion to one or more objects located within a field of view of each of the stereo camera sensors, and wherein the distortion of each of the one or more objects is employed to determine at least a location of each of the one or more objects;

defining a gesture region within the field of view of the pair of stereo camera sensors;

determining whether one or more gesture objects are positioned within the gesture region

determining whether one or more alternative input devices are in use; and

enabling gesture interaction when one or more gesture objects are positioned within the gesture region, and the one or more alternative input devices are not in use.

18. The method of claim 17, further comprising the steps of:

updating a background image in the field of view of the pair of stereo cameras when the one or more alternative input devices are in use; and

fixing the background image in the field of view of the pair of stereo cameras when the one or more alternative input devices are not in use.

19. The method of claim 18, further comprising the step of generating a depth map of the one or more gesture objects in the gesture region when gesture recognition is enabled.

20. A gesture-enabled keyboard, comprising:

a keyboard housing comprising one or more keyboard keys for typing; and

a pair of stereo camera sensors mounted within the keyboard housing, a field of view of the pair of stereo camera sensors projecting substantially perpendicularly to the plane of the keyboard housing, each of the pair of stereo camera sensors being selected from the group of a fisheye lens camera and a wide angle lens camera, each of the stereo camera sensors providing a distortion to one or more objects located within a field of view of each of the stereo camera sensors, and wherein the distortion of each of the one or more objects is employed to determine at least a location of each of the one or more objects, a background of the field of view being updated when one or more alternative input devices are in use;

wherein a gesture region comprising one or more interaction zones and a virtual membrane defining a region of transition from one of the one or more interaction zones to another of the one or more interaction zones is defined within the field of view of the pair of stereo camera sensors;

wherein gesture interaction is enabled when one or more gesture objects are positioned within the gesture region, and when the one or more alternative input devices are not in use.