Image processing for camera based motion tracking

• US 8,860,693 B2
• Filed: 07/08/2009
• Issued: 10/14/2014
• Est. Priority Date: 07/08/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Elias, John Greer
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): XAVIER, ANTONIO J
• Application Number:
  US12/499,351
• Publication Number:
  US 20110006991A1
• Time in Patent Office: 1,924 Days
• Field of Search: None
• US Class Current: 345/175
• CPC Class Codes: G06F 1/1662 Details related to the integrated keyboard
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

Image processing techniques that can improve the user interface experience associated with key-based input devices. In one embodiment, a motion sensitive mechanical keyboard can utilize orthogonally-oriented cameras to sense hand/finger motion over the surface of the keys. This arrangement can enable a standard look and feel mechanical keyboard to receive command and cursor input (e.g., pointing and gestures) from the user without requiring the user to move the user'"'"'s hand off the keyboard. The image processing techniques can be utilized to minimize or remove undesirable cursor movement that can occur based on certain detected hand/finger motion from such cameras, including looming and/or vertical motions for example.

Claims

1. An input device comprising:

an input surface aligned with a plane;

an optical sensor oriented toward the input surface, the optical sensor having an optical axis parallel to the plane and configured to detect an inplane aspect of a motion along the input surface and orthogonal to the optical axis, a vertical aspect of the motion towards or away from the input surface, and a looming aspect of the motion along the input surface and parallel to the optical axis; and

one or more processors configured to process the vertical aspect of the motion and the looming aspect of the motion without processing the inplane aspect of the motion if the detected vertical or looming aspect of the motion exceeds a threshold amount.

2. The input device of claim 1, wherein the optical sensor includes a camera configured to capture an image along the input surface.

3. The input device of claim 2, wherein the one or more processors are configured to analyze difference images based on images captured by the camera to determine whether the detected vertical aspect or looming aspect exceeds the threshold amount.

4. The input device of claim 1, wherein the input surface comprises a mechanical keyboard.

5. The input device of claim 1, wherein the input device is part of a personal computer.

6. The input device of claim 1, wherein the one or more processors are configured to detect a vertical aspect of motion relative to the input surface.

12. The input device of claim 6, wherein the optical sensor includes a camera configured to capture an image along the input surface.

13. The input device of claim 12, wherein the one or more processors are configured to analyze difference images based on images captured by the camera to determine whether the detected second aspect or third aspect exceeds the threshold amount.

7. A computing system comprising:

a keyboard comprising

multiple mechanical keys aligned in a plane;

multiple optical sensors oriented toward the keys, each optical sensor having an optical axis parallel to the plane and configured to detect a first aspect, a second aspect, and a third aspect of a motion along the input surface, wherein

the first aspect of the motion corresponds to a distance in a first direction parallel to the plane and orthogonal to one of the optical axes,

the second aspect of the motion corresponds to distance in a second direction orthogonal to the plane, and

the third aspect of the motion corresponds to a distance in a third direction parallel to the plane and parallel to the one of the optical axes; and

one or more processors configured to process the second aspect of the motion and the third aspect of the motion without processing the first aspect of the motion if the second aspect of the motion or the third aspect of the motion exceeds a threshold amount.

8. The computing system of claim 7, wherein at least two of the multiple optical sensors are oriented orthogonally to one another.

9. The computing system of claim 7, wherein a first set of the multiple optical sensors are oriented orthogonally to one another and cover a first side of the keyboard, and a second set of the multiple optical sensors are oriented orthogonally to one another and cover a second side of the keyboard.

10. The computing system of claim 9, wherein the keyboard is configured to enable an input mode in which the motion is tracked to provide cursor input, and to disable either the first set or the second set of the multiple optical sensors when the input mode is enabled.

11. The computing system of claim 7, wherein the motion is associated with multiple objects in contact with or in proximity to the keyboard.

14. A computing system, comprising:

a receiving module configured to receive data from an input device, the data received from the input device comprising an inplane aspect of a motion along an input surface of the input device and orthogonal to an optical axes of the input device, a vertical aspect of the motion towards or away from the input surface, and a looming aspect of the motion along the input surface and parallel to the optical axis, wherein the optical axes of the input device being parallel to the input surface; and

a processor configured to process the vertical aspect of the motion and the looming aspect of the motion without processing the inplane aspect of the motion if the detected vertical or looming aspect of the motion exceeds a threshold amount.

15. The computing system of claim 14, wherein the one processor is configured to receive images captured along the input surface of the input device.

16. The computing system of claim 15, wherein the processor is configured to analyze difference images based on the images received to determine whether the detected vertical aspect or looming aspect exceeds the threshold amount.