System and method for a miniature user input device

• US 20050012714A1
• Filed: 06/21/2004
• Published: 01/20/2005
• Est. Priority Date: 06/25/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Atrua LLC (Apple Inc.)
• Inventors: Russo, Anthony P., Howell, Mark J.
• Granted Patent: US 7,474,772 B2
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06V 40/1335 Combining adjacent partial images (e.g. slices) to create a composite input or reference pattern; Tracking a sweeping finger movement

Abstract

A system and method of the present invention emulates electronic input devices such as a steering wheel, a mouse, a joystick, a navigation bar, a scroll wheel, and a pressure-sensitive button. The method comprises swiping a finger on a swipe sensor, calculating a difference between a first position of the finger on the swipe sensor and a second position on the swipe sensor, and translating the difference into a plurality of sets of signals. Each set of signals is used to emulate the signals generated by one of the electronic input devices. The method thus uses a single swipe sensor to emulate multiple electronic input devices. In a preferred embodiment, the system is used to control a display on a portable electronic device is coupled to or forms part of the portable electronic device.

Claims

1. A method comprising:

a. calculating a difference between first patterned image data and second patterned image data; and

b. translating the difference into a set of signals, the set of signals corresponding to one or more signals used to emulate any one of a plurality of electronic input devices.

2. The method of claim 1, wherein the first patterned image data corresponds to an image of a biometric object at a first position and the second patterned image data corresponds to the image of the biometric object at a second position.

3. The method of claim 2, further comprising reading the image of the biometric object at the first position and reading the image of the biometric object at the second position.

4. The method of claim 3, wherein biometric image data comprises finger image data related to a fingerprint image.

5. The method of claim 4, wherein reading the image of the biometric object at the first and second positions both comprise swiping a finger across a finger swipe sensor.

6. The method of claim 4, wherein reading the image of the biometric object at the first and second positions comprises placing a finger onto a finger placement sensor.

7. The method of claim 4, wherein calculating a difference comprises:

a. sensing a presence of a finger on a finger sensor; and

b. calculating a linear difference between the image of the biometric object at the first position and the image of the biometric object at second position using a row-based correlation.

8. The method of claim 7, wherein one set of signals corresponds to output signals of a computer mouse for controlling an onscreen pointer.

9. The method of claim 8, further comprising non-linearly scaling the linear difference with a scaling factor.

10. The method of claim 9, wherein non-linearly scaling comprises smoothing the linear difference.

11. The method of claim 10, wherein the scaling factor is determined dynamically, based on a pre-determined number of positions of the finger on the finger sensor.

12. The method of claim 11, wherein non-linearly scaling further comprises exponentially filtering the linear difference.

13. The method of claim 12, wherein the non-linear scaling further comprises median filtering.

14. The method of claim 7, wherein one set of signals corresponds to the output signals of a scroll wheel for controlling an onscreen object.

15. The method of claim 14, wherein translating the difference into a set of signals comprises:

a. translating the linear difference into a direction of movement of the finger on the finger sensor using one or more non-linear scaling functions; and

b. generating the set of signals when the finger is detected on the finger sensor corresponds to selecting an item indicated by a scroll wheel.

16. The method of claim 7, wherein one set of signals corresponds to the output signals of a navigation bar for controlling an onscreen object.

17. The method of claim 16, wherein translating the difference into the one set of signals comprises:

a. comparing the linear difference in a direction to a pre-determined threshold, the direction being one of a plurality of directions controllable by a navigation bar;

b. generating signals corresponding to the movement of the onscreen object by a unit of motion in a direction controllable by the navigation bar when the linear difference exceeds the threshold; and

c. repeating steps a and b for the remaining plurality of directions controllable by the navigation bar.

18. The method of claim 7, wherein one set of signals corresponds to output signals of a joystick for controlling an onscreen object.

19. The method of claim 18, wherein translating the difference into the one set of signals comprises:

a. translating the linear difference into a direction of movement of the finger on the finger sensor using non-linear scaling; and

b. generating a signal corresponding to moving the onscreen object to a home position on a display when a finger is not detected on the finger sensor.

20. The method of claim 19, wherein generating a signal corresponding to moving the onscreen object to a home position on a display comprises generating a signal that simulates a spring-mass damper decreasing a velocity of the onscreen object.

21. The method of claim 20, wherein sensing the presence of the finger comprises measuring a pressure applied by the finger on the finger sensor.

22. The method of claim 21, wherein measuring a pressure applied by the finger comprises computing the weighted average of regional finger presence data, smoothed average of finger image data, and smoothed variance of the finger image data.

23. The method of claim 22, wherein generating a signal corresponding to moving the object to a home position comprises generating a signal corresponding to a damping signal at a rate related to a reduced pressure on the finger sensor.

24. The method of claim 7, wherein one set of signals corresponds to the output signals of a steering wheel for controlling an object on a display.

25. The method of claim 24, wherein calculating the linear difference comprises estimating a rotational movement of the finger on the finger sensor.

26. The method of claim 25, wherein translating the difference into the one set of signals comprises:

a. converting the rotational movement to an angle of rotation for the onscreen object; and

b. generating a signal corresponding to returning the onscreen object to a home position on the display in the event that the finger is not present on the finger sensor.

27. The method of claim 26, wherein generating a signal corresponding to returning the onscreen object to the home position on the display comprises generating a signal corresponding to a spring-mass damper that decreases in velocity.

28. The method of claim 27, wherein sensing the presence of the finger further comprises measuring a pressure applied by the finger on the sensor.

29. The method of claim 28, wherein generating a signal corresponding to returning the onscreen object to the home position on the display comprises generating a damping signal corresponding to a reduction on the amount of pressure applied by the finger on the finger sensor.

30. The method of claim 7, wherein sensing the presence of the finger on the finger sensor comprises tracking fluctuations of smoothed variance of the finger image data.

31. The method of claim 1, further comprising transmitting a set of signals from the plurality of sets of signals to a display, thereby controlling an onscreen object on the display.

32. The method of claim 31, wherein the display forms part of a host device, the host device selected from the group consisting of a portable computer, a personal digital assistant, a portable telephone, a portable gaming device, a biometric/health monitor, and a digital camera.

33. A method comprising:

a. determining a difference between first patterned image data and second patterned image data; and

b. using the difference to concurrently control a plurality of onscreen objects.

34. The method of claim 33, wherein using the difference to concurrently control a plurality of onscreen objects comprises concurrently generating a plurality of sets of signals, each set of signals corresponding to the outputs of an input device used to control one of the plurality of onscreen objects.

35. The method of claim 34, wherein the input device is selected from the group consisting of a steering wheel, a mouse, a joystick, a navigation bar, a scroll wheel, and a pressure sensitive button.

36. The method of claim 33, wherein patterned image data correspond to biometric image, data.

37. The method of claim 36, further comprising:

a. sensing a biometric image; and

b. generating corresponding biometric image data.

38. The method of claim 37, wherein sensing a biometric image comprises sensing a finger swiped across a finger swipe sensor.

39. The method of claim 37, wherein sensing a biometric image comprises sensing a finger placed upon a finger placement sensor.

40. The method of claim 34, wherein the sets of signals correspond to outputs of a mouse, outputs of a joystick, outputs of a navigation bar, outputs of a scroll wheel, or outputs of a pressure sensitive button by using row-based correlation to determine a difference between the first patterned image data and the second patterned image data.

41. The method of claim 40, wherein generating a set of signals corresponding to the outputs of a steering wheel comprises generating rotational movement between the first patterned image data and the second patterned image data.

42. The method of claim 37, further comprising storing one or more patterned image data.

43. The method of claim 42, further comprising timestamping stored biometric image data.

44. A system for controlling a movement of an onscreen object, the system comprising:

a. a means for reading patterned images and generating raw patterned image data;

b. a means for converting raw patterned image data into movement information, the means for converting raw patterned image data coupled to the means for reading patterned images; and

c. a means for converting movement information into sets of computer input device signals for controlling movement of the onscreen object, the means for converting movement information coupled to the means for converting raw patterned image data.

45. The system of claim 44, wherein the means for reading comprises a finger sensor and the patterned images correspond to fingerprint images.

46. The system of claim 45, wherein the finger sensor comprises a finger swipe sensor.

47. The system of claim 45, wherein the finger sensor comprises a finger placement sensor.

48. The system of claim 46, wherein the finger swipe sensor comprises a finger sensor selected from the group consisting of a capacitive sensor, an optical sensor, and a thermal sensor.

49. The system of claim 44, wherein the sets of computer input device signals comprise signals for emulating at least two devices selected from the group consisting of a steering wheel, a mouse, a joystick, a navigation bar, and a scroll wheel.

50. The system of claim 49, wherein the means for converting raw patterned image data comprises a linear movement correlator configured to calculate linear movements of a patterned image using a row-based correlation algorithm.

51. The system of claim 50, wherein the means for converting the movement information is further configured to perform non-linear scaling.

52. The system of claim 51, wherein the means for converting raw patterned image data further comprises a pressure detector coupled to the means for reading patterned images.

53. The system of claim 52, wherein the means for converting raw patterned image data further comprises a patterned image presence detector coupled to the means for reading patterned images.

54. The system of claim 53, wherein the patterned image presence detector comprises a finger presence detector.

55. The system of claim 53, wherein the means for converting raw patterned image data further comprises a rotational movement correlator coupled to the means for reading patterned images.

56. The system of claim 55, wherein the means for converting the movement information comprises a steering wheel emulator unit coupled to both the rotational movement correlator and the linear movement correlator, wherein the rotational movement correlator is configured to translate a change in position of a read patterned image into an angle of rotation.

57. The system of claim 55, wherein the means for converting the movement information comprises a mouse emulator unit coupled to the linear movement correlator and the patterned image presence detector.

58. The system of claim 55, wherein the means for converting the movement information comprises a joystick emulator unit coupled to the linear movement correlator and the patterned image presence detector.

59. The system of claim 55, wherein the means for converting movement information comprises a navigation bar emulator unit coupled to the linear movement correlator and the patterned image presence detector.

60. The system of claim 55, wherein the means for converting the movement information comprises a scroll wheel emulator unit coupled to the linear movement correlator and the patterned image presence detector.

61. The system of claim 55, wherein the means for converting movement information comprises a pressure sensitive button emulator unit coupled to the pressure detector unit and the patterned image presence detector.

62. The system of claim 53, further comprising a time interval accumulator coupling the means for reading patterned images to any two of the rotational movement correlator, the linear movement correlator, the pressure detector, and the patterned image presence detector.

63. The system of claim 62, wherein the patterned image presence detector is configured to compare the variance of raw patterned image data with a first threshold value and a second threshold value to determine a presence of a patterned image on the means for reading a patterned image.

64. The system of claim 63, wherein the pressure detector is configured to determine a pressure by computing a weighted average of regional patterned image presence data, of smoothed average, and of smoothed variance between patterned image movement information.

65. The system of claim 64, wherein the mouse emulator unit is configured to perform at least one of smoothing, scaling, and filtering patterned image movement information.

66. The system of claim 46, wherein the finger swipe sensor has a resolution of less than 250 dots per inch.

67. The system of claim 44, further comprising a display for presenting the onscreen object controllable by the means for converting movement information.

68. The system of claim 67, wherein the means for reading patterned images, the means for converting raw patterned image data and the means for converting the movement information form part of an integral host device.

69. The system of claim 68, wherein the host device is a device selected from the group consisting of a portable computer, a personal digital assistant, a portable telephone, a portable gaming device, a biometric/health monitor, and a digital camera.