Optical pseudo trackball controls the operation of an appliance or machine

• US 20020135565A1
• Filed: 03/21/2001
• Published: 09/26/2002
• Est. Priority Date: 03/21/2001
• Status: Active Grant

• Current Assignee: Elan Microelectronics Corporation
• Original Assignee: Agilent Technologies, Inc.
• Inventors: Miller, Edward L, Gordon, Gary B.
• Granted Patent: US 6,677,929 B2
• Time in Patent Office: Days
• US Class Current: 345/169
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen

Abstract

An optical fingertip tracker, which may be a pseudo trackball, responds to the motion of the texture on one of the user'"'"'s digits pressed against an image input aperture to produce motion signals processed within an appliance to be controlled. The appliance may have a minimal GUI, such as an LCD display having only a modest number of low resolution pixel locations and/or preestablished characters or icons. An appliance control parameter is set or adjusted in response to translation ofthe fingertip along a particular axis. E.g., to enter a parameter value (volume) a changing bar graph can be displayed. The display might even be omitted; to set the tone control of an audio appliance it can be sufficient to simply adjust the tone until the result is as desired. A numeric display of control parameter digits may be desirable, as when moving the fingertip tunes a receiver. There can be several particular axes along which fingertip movement changes an associated appliance control parameter. Other appliances can include character recognition of input characters traced by the fingertip. A pressure sensitive switch coupled to the fingertip tracker can serve as a delimiting mechanism useful in ratification of interpreted results and in editing strings that have been input. Excursions is a loss-of-correlation signal can be detected and interpreted as tapping the image input aperture with the fingertip. Such “taps” and “double taps” can be defined, and have selected meanings associated therewith that may be used in conjunction with, or in place of, the pressure sensitive switch.

Claims

1. Input apparatus for controlling the operation of an appliance, the input apparatus comprising:

a housing containing the appliance;

absent from the appliance a keyboard for entry of separate keystrokes representing individual letters of the alphabet;

an image input aperture in the housing and against which a portion of the tip of one of the operator'"'"'s digits may be placed;

a light source disposed proximate the image input aperture and that emits light which illuminates a portion of the tip of the digit placed against the image input aperture;

an image exit surface optically coupled to the image input aperture and from which reflected light that is an image of the tip of the operator'"'"'s digit light may exit;

a motion transducer disposed proximate the image exit surface and that receives the image, from which the motion transducer creates pixelized representations comprising a newly sampled array, a previously sampled array and shifted versions of one of those arrays that are its nearest neighbor arrays, which one array and its nearest neighbor arrays are correlated with the non-shifted array to produce motion signals that correspond to motions in the directions of coordinate axes by the tip of the digit moving against the image input surface and that represent operator input to affect the operation of the appliance;

an appliance control circuit coupled to the motion signals, disposed internal to the appliance and that controls the operation of the appliance by altering the value of at least one appliance control parameter in accordance with motion of the tip of the digit against the image input aperture.

2. Input apparatus as in claim 1 further comprising a lens assembly located within the image input aperture.

3. Input apparatus as in claim 2 wherein the lens assembly comprises a rod lens.

4. Input apparatus as in claim 2 wherein the lens assembly has a dome shape and a portion of which extends beyond an outer surface of the housing.

5. Input apparatus as in claim 4 wherein the dome shaped lens assembly has a diameter of about two to three millimeters.

6. Input apparatus as in claim 1 wherein the image input aperture holds a transparent surface that is generally flat and in about the same plane as an outer surface of the housing at the location thereof whereat the image input aperture is located.

7. Input apparatus as in claim 1 wherein the image exit surface comprises a mirror.

8. Input apparatus as in claim 1 wherein the image exit surface comprises a lens.

9. Input apparatus as in claim 1 further comprising indicia coupled to the appliance control circuit and indicating which appliance control parameter among a plurality thereof is changed in accordance with the motion of the fingertip against the image input surface.

10. Input apparatus as in claim 9 further comprising a display coupled to the appliance control circuit and wherein the indicia comprises a menu element in a menu of choices, the menu element containing a legend associated with an appliance control parameter.

11. Input apparatus as in claim 9 wherein the indica comprises a visible annunciator disposed next to a permanently visible legend associated with an appliance control parameter.

12. Input apparatus as in claim 1 further comprising indicia coupled to the appliance control circuit and indicating the value of an appliance control parameter as it is altered in accordance with the motion of the tip of the digit against the image input surface.

13. Input apparatus as in claim 12 wherein the indica comprises a bar graph display.

14. Input apparatus as in claim 12 wherein the indicia comprises a digital display of a numeric value.

15. Input apparatus as in claim 12 wherein the alteration to the appliance control parameter is a trial alteration that does not affect the operation of the appliance until ratified by the operator.

16. Input apparatus as in claim 15 further comprising an electrical switching device actuated by pressure exerted against the image input aperture and electrically coupled to the appliance control circuit, and wherein ratification of a trial alteration in accomplished by actuating the electrical switching device.

17. Input apparatus as in claim 1 wherein the correlation of the arrays produces interpolated motion signals representing motion less that the size of spatial features in the image.

18. Input apparatus as in claim 1 further comprising an electrical switching device actuated by pressure exerted against the image input aperture and electrically coupled to the appliance control circuit.

19. Input apparatus as in claim 18 wherein there are a plurality of appliance control parameters and the appliance control circuit construes actuation of the electrical switching device as a delimiter that separates the alteration of the value of one appliance control parameter from the alteration of the value of another appliance control parameter.

20. Input apparatus as in claim 1 further comprising a loss-of-correlation signal produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input aperture and coupled to the appliance control circuit, wherein there are a plurality of appliance control parameters and the appliance control circuit construes the presence of the loss-of correlation signal as a delimiter that separates the alteration of the value of one appliance control parameter from the alteration of the value of another appliance control parameter.

21. Input apparatus as in claim 1 further comprising a loss-of-correlation signal produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input aperture and coupled to the appliance control circuit, and wherein the appliance control circuit responds in a selected manner to excursions in the loss-of-control-signal produced by the operator removing the digit from the image input aperture, briefly replacing it without substantial motions in the directions of the coordinate axes, followed by again removing the digit.

22. Input apparatus as in claim 1 wherein the coordinate axes comprise orthogonal first and second axes, the motion signals comprise respectively corresponding first and second motion signals, the at least one appliance control parameter comprises respectively corresponding first and second appliance control parameters, and further wherein the first motion signal alters the value of the first appliance control parameter and the second motion signal alters the value of the second appliance control parameter.

23. Input apparatus as in claim 1 wherein the coordinate axes comprise orthogonal first and second axes, the motion signals comprise respectively corresponding first and second motion signals, the at least one appliance control parameter comprises respectively corresponding first and second appliance control parameters, and further wherein the appliance control circuit leaves the second appliance control parameter unchanged while altering the value of the first appliance control parameter when the motion of the tip of the digit along the first axis exceeds that along the second axes, and leaves the first appliance control parameter unchanged while altering the value of the second appliance control parameter when the motion of the tip of the digit along the second axes exceeds that along the first axis.

24. Input apparatus as in claim 23 wherein once one of the first and second appliance control parameters is left unchanged and the other is altered, the unchanged appliance control parameter remains unchanged and only the other is altered by motion of the tip of the digit, regardless of which motion signal indicates the greater amount of motion.

25. Input apparatus as in claim 1 wherein the coordinate axes comprise orthogonal first and second axes, the motion signals comprise respectively corresponding first and second motion signals, the at least one appliance control parameter comprises respectively corresponding first and second appliance control parameters and also a third appliance control parameter, and further wherein motion of the tip of the digit along a path within a region inclined by a selected amount to one of the coordinate axes and of a selected width, is construed by the appliance control circuit as input from the operator to alter the value of a third appliance control parameter according to the tip of the digit'"'"'s motion along that path and to leave the values of the first and second appliance control parameters unchanged.

26. Input apparatus as in claim 1 wherein the coordinate axes comprise orthogonal first and second axes, the motion signals comprise respectively corresponding first and second motion signals having signs that indicate the direction of motion along their respective axes, the at least one appliance control parameter comprises respectively corresponding first and second appliance control parameters and also a third appliance control parameter, and further wherein when the first and second motion signals have the same signs and substantially the same absolute amount of motion, then the appliance control circuit construes the combination of the first and second motion signals as motion along the third axes and as operator input to alter the value of the third appliance control parameter in accordance with that motion.

27. Input apparatus as in claim 1 wherein the coordinate axes comprise orthogonal first and second axes, the motion signals comprise respectively corresponding first and second motion signals having signs that indicate the direction of motion along their respective axes, the at least one appliance control parameter comprises respectively corresponding first and second appliance control parameters and also a third appliance control parameter, and further wherein when the first and second motion signals have differing signs and substantially the same absolute amount of motion, then the appliance control circuit construes the combination of the first and second motion signals as motion along the third axes and as operator input to alter the value of the third appliance control parameter in accordance with that motion.

28. Input apparatus as in claim 1 wherein the appliance control circuit comprises a micro-processor and a read-only memory that are an embedded control system including executable instructions permanently stored in the memory and fetched therefrom by the microprocessor.

29. Input apparatus as in claim 1 wherein the appliance control circuit comprises a state machine whose transitions are responsive to the motion signals and whose changes in state control the appliance.

30. Input apparatus as in claim 1 wherein the appliance control circuit comprises logic circuitry implemented by a gate array and also comprises registers that are associated with appliance control parameters, that are loaded with values derived from the motion of the tip of the digit, and that control the operation of the appliance.

31. Input apparatus as in claim 1 wherein the appliance operates upon audio program material and the at least one appliance control parameter determines the setting of a volume control for an audible signal.

32. Input apparatus as in claim 1 wherein the appliance operates upon audio program material and the at least one appliance control parameter determines the setting of a tone control for an audible signal.

33. Input apparatus as in claim 1 wherein the appliance comprises a receiver of radio frequency signals and the at least one appliance control parameter adjusts the frequency to which the receiver is tuned.

34. Input apparatus as in claim 1 wherein the appliance is a wireless telephone.

35. Input apparatus as in claim 1 wherein the appliance is a global position receiver.

36. Input apparatus as in claim 1 wherein the appliance is a personal data organizer.

37. Input apparatus as in claim 1 wherein the appliance is a video recorder.

38. Input apparatus as in claim 1 wherein the appliance is a video player.

39. Input apparatus as in claim 1 wherein the appliance is a digital camera.

40. Apparatus as in claim 1 wherein the appliance operates upon images and the motions of the tip of the digit delimit a region of interest within a work image operated on by the appliance.

41. Input apparatus for controlling the operation of an appliance according to motion of the tip of an operator'"'"'s digit along a plurality of coordinate axes by (a) selecting an entry from within a displayed menu of choices and (b) adjusting the value of an appliance control parameter, the input apparatus comprising:

a housing containing the appliance;

an image input aperture in the housing and against which a portion of the tip of one of the operator'"'"'s digits may be placed;

a light source disposed proximate the image input aperture and that emits light which illuminates a portion of the tip of the digit placed against the image input aperture;

an image exit surface optically coupled to the image input aperture and from which reflected light that is an image of the tip of the operator'"'"'s digit may exit;

a motion transducer disposed proximate the image exit surface and that receives the image, from which the motion transducer creates pixelized representations comprising a newly sampled array, a previously sampled array and shifted versions of one of those arrays that are its nearest neighbor arrays, which one array and its nearest neighbor arrays are correlated with the non-shifted array to produce motion signals that correspond to motions in the directions of coordinate axes by the tip of the digit moving against the image input aperture and that represent operator input to control the operation of the appliance;

an embedded system coupled to the motion signals and that directs the operation of the appliance in response to inputs made by an operator of the appliance;

the embedded system including a motion signal selector that examines corresponding instances of the first and second motion signals to determine if one thereofhas a magnitude larger than the other by a selected amount, and if so, construes the motion signal with the larger magnitude as input made by the operator while also construing the other motion signal as having a magnitude of zero;

a display coupled to the embedded system and that displays indicia corresponding to a menu of choices available to the operator for affecting the operation of the appliance;

the embedded system treating one of the construed motion signals as input made by the operator to select an entry in the menu by altering corresponding displayed indicia in accordance with that input; and

the embedded system treating the other construed motion signal as input made by the operator adjust the value of an appliance control parameter associated with the selected entry in the menu.

42. Input apparatus as in claim 41 wherein the display includes indicia corresponding to the value of an appliance control parameter associated with the selected menu entry.

43. Input apparatus as in claim 41 further comprising an electrical switching device actuated by pressure exerted against the image input aperture and electrically coupled to the embedded system, and further wherein the embedded system construes actuation of the switching device as a delimiter of input made by the operator.

44. Input apparatus as in claim 41 further comprising a loss-of-correlation signal coupled to the embedded system and produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input aperture, and further wherein the embedded system construes the loss-of-correlation signal as a delimiter of input made by the operator.

45. Input apparatus as in claim 41 further comprising a loss-of-correlation signal coupled to the embedded system and produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input aperture, and further wherein the embedded system responds in a selected manner to excursions in the loss-of-control-signal produced by the operator removing the digit from the image input aperture, briefly replacing it without substantial motions in the directions of the coordinate axes, followed by again removing the digit.

47. Input apparatus as in claim 46 wherein the display includes indicia corresponding to the value of an appliance control parameter associated with the selected menu entry.

49. Input apparatus as in claim 48 further comprising a switching device coupled for actuation to the image input aperture and actuated by pressure exerted against the image input aperture.

50. Input apparatus as in claim 49 wherein the switching device is electrically coupled to the embedded system, and further wherein the embedded system construes actuation of the switching device as a delimiter the motions in the coordinate axes by the tip of the digit and that represent a character input by the operator.

51. Input apparatus as in claim 49 wherein the switching device is electrically coupled to the embedded system, and further wherein the embedded system construes actuation of the switching device as a ratification of a character input by the operator and displayed by the display.

52. Input apparatus as in claim 49 wherein the switching device is electrically coupled to the embedded system, and further wherein the embedded system construes actuation of the switching device as a BACKSPACE instruction that allows the overwriting of a previously input character.

53. Input apparatus as in claim 49 wherein the switching device is electrically coupled to the embedded system, and further wherein the embedded system construes steady actuation of the switching device for longer than a selected period of time as a CLEAR instruction that restores the input apparatus to a state existing prior to the input of recent characters.

54. Input apparatus as in claim 49 wherein the switching device is electrically coupled to the embedded system, and further wherein the embedded system construes two actuations of the switching device within a selected period of time as a BACKSPACE instruction that allows the overwriting of a previously input character.

55. Input apparatus as in claim 54 wherein the embedded system construes further single actuations of the switching device subsequent to a BACKSPACE instruction as implied BACKSPACE instructions, each allowing the overwriting of a respectively earlier character previously input by the operator.

56. Input apparatus as in claim 54 further comprising a loss-of-correlation signal coupled to the embedded system and produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input surface, and further wherein the embedded system construes the loss of correlation signal as a nullification of any BACKSPACE instructions still in effect, so that a newly input character will be appended to end of a string of previously input characters.

57. Input apparatus as in claim 48 further comprising a loss-of-correlation signal electrically coupled to the embedded system and produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input surface, and further wherein the embedded system construes the loss of correlation signal as a delimiter of a character input by the operator.

58. Input apparatus as in claim 48 further comprising a loss-of-correlation signal electrically coupled to the embedded system and produced by the motion transducer in response to an absence of the operator'"'"'s digit from the image input surface, and further wherein the embedded system responds in a selected manner to excursions in the loss-of-control-signal produced by the operator removing the digit from the image input aperture, briefly replacing it without substantial motions in the directions of the coordinate axes, followed by again removing the digit.

59. Input apparatus as in claim 48 wherein the appliance comprises a telephone.

60. Input apparatus as in claim 59 wherein the characters input by the operator comprise a telephone number.

61. Input apparatus as in claim 48 wherein the appliance comprises a global position receiver.

62. Input apparatus as in claim 48 wherein the appliance comprises a personal data organizer.

46. Input apparatus for controlling the operation of an appliance according to motion of the tip of and operator'"'"'s digit along a plurality of coordinate axes by (a) selecting an entry from within a displayed menu of choices and (b) adjusting the value of an appliance control parameter, the input apparatus comprising:

a housing containing the appliance;

an image input aperture in the housing and against which a portion of the tip of one of the operator'"'"'s digits may be placed;

a light source disposed proximate the image input aperture and that emits light which illuminates a portion of the tip of the digit placed against the image input aperture;

an image exit surface optically coupled to the image input aperture and from which reflected light that is an image of the tip of the operator'"'"'s digit may exit;

a motion transducer disposed proximate the image exit surface and that receives the image, from which the motion transducer creates pixelized representations comprising a newly sampled array, a previously sampled array and shifted versions of one of those arrays that are its nearest neighbor arrays, which one array and its nearest neighbor arrays are correlated with the non-shifted array to produce motion signals that correspond to motions in the directions of coordinate axes by the fingertip moving against the image input surface and that represent operator input to control the operation of the appliance;

an embedded system coupled to the motion signals and that directs the operation of the appliance in response to inputs made by an operator of the appliance;

the embedded system including a motion signal selector that examines corresponding instances of the first and second motion signals to determine if one thereof has a magnitude larger than the other by a selected amount, and if so, construes the motion signal with the larger magnitude as input made by the operator while also construing the other motion signal as having a magnitude of zero, and also examines the corresponding instances to determine if they have magnitudes that are of generally equal absolute value, and if so, construes the first and second motion signals as having magnitudes of zero while producing a third motion signal corresponding to motion of the tip of the digit along an axis inclined to the coordinate axes;

a display coupled to the embedded system and that displays indicia corresponding to a menu of choices available to the operator for affecting the operation of the appliance;

the embedded system treating one of the first and second construed motion signals as input made by the operator to select in the direction of the corresponding coordinate axis an entry in the menu by altering corresponding displayed indicia in accordance with that input, treating the other of the first and second construed motion signals as input made by the operator to select in the direction of the other coordinate axis another entry in the menu by altering corresponding displayed indicia in accordance with that input; and

the embedded system treating the third motion signal as input made by the operator adjust the value of an appliance control parameter associated with the selected entry in the menu.

48. Character input apparatus for controlling the operation of an appliance, the character input apparatus comprising:

a housing containing the appliance;

absent from the appliance a keyboard for entry of separate keystrokes representing individual letters of the alphabet and the numeric digits;

an image input aperture in the housing and against which a portion of the tip of one of the operator'"'"'s digits may be placed;

a light source disposed proximate the image input aperture and that emits light which illuminates a portion of the tip of the digit placed against the image input aperture;

an image exit surface optically coupled to the image input aperture and from which reflected light that is an image of the operator'"'"'s digit light may exit;

a motion transducer disposed proximate the image exit surface and that receives the image, from which the motion transducer creates pixelized representations comprising a newly sampled array, a previously sampled array and shifted versions of one of those arrays that are its nearest neighbor arrays, which one array and its nearest neighbor arrays are correlated with the non-shifted array to produce motion signals that correspond to motions in the directions of coordinate axes by the tip of the digit moving against the image input surface and that represent a character in the font and whose shape the operator has traced with the motion of his digit upon the imaging orifice;

the motion signals coupled to the embedded system;

the embedded system construing the motion signals as the character traced by the operator'"'"'s digit and altering the displayed indicia in accordance therewith; and

a display coupled to the embedded system and that displays characters input by the operator of the appliance.

63. A method of setting the value of an appliance control parameter affecting the operation of an appliance, the method comprising the steps of:

placing the tip of an operator'"'"'s digit against an imaging surface;

moving the tip of the digit across the imaging surface in patterns corresponding to user input;

illuminating that portion of the digit that is against the imaging surface;

focusing light reflected from the illuminated portion of the digit as an image onto an array of photo detectors that are part of a motion transducer;

digitizing the output values of the photo detectors and storing the digital results as a reference array in a memory;

subsequent to storing the reference array, digitizing the output values of the photo detectors and storing the digital results as a sample array in a memory;

correlating the sample array with the reference array and also with nearest neighbor shifted versions of one of those arrays array to produce a motion signal having a value indicative of motion along an axis by the tip of the digit as it moves across the imaging surface; and

setting the value of an appliance control parameter in accordance with the value of the motion signal.

64. A method as in claim 63 further comprising the step of displaying an indication of the value to which the appliance control parameter is set by the setting step.

65. A method as in claim 63 further comprising the steps of recognizing the removal, briefand substantially motion-free replacement and subsequent removal of the digit against the imaging surface, and then in response that recognition, performing a selected appliance operation.

66. A method as in claim 63 wherein the correlating step produces two motion signals having respective values indicative of motion along a plurality of axes and the setting step sets the value of a first appliance control parameter in accordance with the value of one of the motion signals and sets the value of a second appliance control parameter in accordance with the value of the other motion signal.

68. A method as in claim 67 further comprising the step of displaying indicia representative of the patterned variations detected in the motion signals.

69. A method as in claim 67 further comprising the steps of recognizing the removal, brief and substantially motion-free replacement and subsequent removal of the digit against the imaging surface, and then in response that recognition, performing a selected machine operation.

70. A method as in claim 67 further comprising the steps of recognizing a first instance of the removal, brief and substantially motion-free replacement and subsequent removal of the digit against the imaging surface, recognizing a second instance of the removal, brief and substantially motion-free replacement and subsequent removal of the digit against the imaging surface, the first and second instances occurring within a selected duration of time, and then in response to such first and second instances of recognition, performing a selected machine operation.

67. A method of controlling the operation of a machine having a control system responsive to input characters, the method comprising the steps of:

placing a tip of an operator'"'"'s digit against an imaging surface;

moving the tip of the digit across the imaging surface in patterns corresponding to input characters;

illuminating that portion of the digit that is against the imaging surface;

focusing light reflected from the illuminated portion of the digit as an image onto an array of photo detectors that are part of a motion transducer;

producing motion signals with the motion transducer that are indicative of the motion of the tip of the digit against the imaging surface;

detecting with the control system patterned variations in the motion signals corresponding to input characters traced by the by the operator'"'"'s digit and that are to control the operation of the machine; and

operating the machine in accordance with the detected patterned variations in the motion signals.