Image Processing for Camera Based Motion Tracking

• US 20110006991A1
• Filed: 07/08/2009
• Published: 01/13/2011
• Est. Priority Date: 07/08/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: ELIAS, John Greer
• Granted Patent: US 8,860,693 B2
• Time in Patent Office: Days
• US Class Current: 345/168
• CPC Class Codes: G06F 1/1662 Details related to the integrated keyboard
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

Image processing techniques that can improve the user interface experience associated with key-based input devices. In one embodiment, a motion sensitive mechanical keyboard can utilize orthogonally-oriented cameras to sense hand/finger motion over the surface of the keys. This arrangement can enable a standard look and feel mechanical keyboard to receive command and cursor input (e.g., pointing and gestures) from the user without requiring the user to move the user'"'"'s hand off the keyboard. The image processing techniques can be utilized to minimize or remove undesirable cursor movement that can occur based on certain detected hand/finger motion from such cameras, including looming and/or vertical motions for example.

Claims

1. An input device comprising:

an input surface aligned with a plane; and

an optical sensor oriented toward the input surface, the optical sensor having an optical axis parallel to the plane and configured to

detect along the input surface a vertical aspect of a motion relative to the input surface or a looming aspect of the motion relative to the optical axis, and

suppress the motion if the detected aspect exceeds a threshold amount.

2. The input device of claim 1, wherein the optical sensor includes a camera configured to capture an image along the input surface.

3. The input device of claim 2, wherein the optical sensor is configured to detect the vertical aspect or the looming aspect by analyzing difference images based on images captured by the camera.

4. The input device of claim 1, wherein the input surface comprises a mechanical keyboard.

5. The input device of claim 1, wherein the input device is part of a personal computer.

6. A method, comprising:

detecting by an optical sensor a motion along an input surface;

detecting a looming aspect of the detected motion;

reporting the detected motion if the looming aspect does not exceed a threshold amount; and

suppressing the detected motion if the looming aspect exceeds the threshold amount.

7. The method of claim 6, wherein the detecting of the motion includes capturing a first image, a second image and a third image along the input surface in successive frames, generating a first difference image between the first image and the second image, and generating a second difference image between the second image and the third image.

8. The method of claim 7, wherein the detecting of the motion further includes comparing a single vector in a row of the second difference image against a single vector in a corresponding row of the first difference image.

9. The method of claim 7, wherein the detecting of the looming aspect of the detected motion includes comparing multiple vectors in a row of the second difference image against multiple vectors in a corresponding row of the first difference image.

10. The method of claim 6, wherein the detecting of the motion and the looming aspect of the detected motion includes analyzing only a subset or rows associated with the first difference image and the second difference image.

11. A method, comprising:

detecting by an optical sensor a motion along an input surface;

detecting a vertical aspect of the detected motion relative to the input surface;

reporting the detected motion if the vertical aspect does not exceed a threshold amount; and

suppressing the detected motion if the vertical aspect exceeds the threshold amount.

12. The method of claim 11, wherein the detecting of the motion includes capturing a first image, a second image and a third image along the input surface in successive frames, generating a first difference image between the first image and the second image, and generating a second difference image between the second image and the third image.

13. The method of claim 12, wherein the detecting of the motion further includes comparing a single vector in a row of the second difference image against a single vector in a corresponding row of the first difference image.

14. The method of claim 12, wherein the detecting of the vertical aspect of the detected motion includes comparing multiple vectors in a row of the second difference image against multiple vectors in a corresponding row of the first difference image.

15. The method of claim 11, wherein the detecting of the motion and the vertical aspect of the detected motion includes analyzing only a subset or rows associated with the first difference image and the second difference image.

16. A keyboard comprising:

multiple mechanical keys aligned in a plane; and

multiple optical sensors oriented toward the keys, each optical sensor having an optical axis parallel to the plane and configured to

detect a first aspect and a second aspect of a motion along the input surface, wherein

the first aspect of the motion corresponds to a distance in a first direction parallel to the plane and orthogonal to the optical axis, and

the second aspect of the motion corresponds to distance in a second direction orthogonal to the plane, and

suppress the first aspect of the motion if the second aspect of the motion exceeds a threshold amount.

17. The keyboard of claim 16, wherein at least two of the multiple optical sensors are oriented orthogonally to one another.

18. The keyboard of claim 16, wherein a first set of the multiple optical sensors are oriented orthogonally to one another and cover a first side of the keyboard, and a second set of the multiple optical sensors are oriented orthogonally to one another and cover a second side of the keyboard.

19. The keyboard of claim 18, wherein the keyboard is configured to enable an input mode in which the motion is tracked to provide cursor input, and to disable either the first set or the second set of the multiple optical sensors when the input mode is enabled.

20. The keyboard of claim 16, wherein the motion is associated with multiple objects in contact with or in proximity to the keyboard.