System and method for a miniature user input device

• US 7,474,772 B2
• Filed: 06/21/2004
• Issued: 01/06/2009
• Est. Priority Date: 06/25/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Atrua Technologies Incorporated (Apple Inc.)
• Inventors: Howell, Mark J., Russo, Anthony P.
• Primary Examiner(s): KIM, CHONG R
• Application Number:
  US10/873,393
• Publication Number:
  US 20050012714A1
• Time in Patent Office: 1,660 Days
• Field of Search: None
• US Class Current: 382/124
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06V 40/1335 Combining adjacent partial images (e.g. slices) to create a composite input or reference pattern; Tracking a sweeping finger movement

Abstract

A system and method of the present invention emulates electronic input devices such as a steering wheel, a mouse, a joystick, a navigation bar, a scroll wheel, and a pressure-sensitive button. The method comprises swiping a finger on a swipe sensor, calculating a difference between a first position of the finger on the swipe sensor and a second position on the swipe sensor, and translating the difference into a plurality of sets of signals. Each set of signals is used to emulate the signals generated by one of the electronic input devices. The method thus uses a single swipe sensor to emulate multiple electronic input devices. In a preferred embodiment, the system is used to control a display on a portable electronic device is coupled to or forms part of the portable electronic device.

Claims

1. A method of emulating multiple electronic input devices comprising:

a. calculating a difference between first patterned image data and second patterned image data using row-based correlation, wherein the row-based correlation comprises obtaining a correlation measure between a single row of the first patterned image data and a single row of the second patterned image data;

b. translating the difference into linear movement and multiple signals corresponding to output signals of multiple electronic input devices, thereby emulating the multiple electronic input devices, wherein the multiple electronic input devices comprise any two or more of a mouse, a joystick, a navigation bar, and a scroll wheel; and

c. converting the difference into rotational movement and signals of at least one electronic input device that utilizes rotational movement and is different from the multiple electronic input devices.

2. The method of claim 1, wherein the first patterned image data corresponds to an image of a patterned object at a first position and the second patterned image data corresponds to an image of the patterned object at a second position.

3. The method of claim 2, further comprising reading the image of the patterned object at the first position and reading the image of the patterned object at the second position.

4. The method of claim 3, wherein the image of the patterned object comprises a fingerprint image.

5. The method of claim 4, wherein reading the image of the patterned object at the first and second positions comprises swiping a finger across a finger swipe sensor.

6. The method of claim 4, wherein reading the image of the patterned object at the first and second positions comprises placing a finger onto a finger placement sensor.

7. The method of claim 4, further comprising sensing a presence of the patterned object on a finger sensor.

8. The method of claim 7, wherein the presence is determined from a history of variance of patterned image data.

9. The method of claim 7, wherein the difference between the first patterned image data and the second patterned image data comprises a linear difference.

10. The method of claim 9, wherein translating the difference comprises mapping the linear difference into movement for controlling an onscreen object on a display.

11. The method of claim 10, wherein mapping the linear difference comprises smoothing the linear difference, filtering the linear difference, non-linearly scaling the linear difference, or a combination of these.

12. The method of claim 11, wherein a scaling factor is based on a speed with which the patterned object is moved across a surface of the finger sensor.

13. The method of claim 11, wherein the scaling factor is determined dynamically, based on a pre-determined number of positions of the patterned object on the finger sensor.

14. The method of claim 11, wherein the multiple signals and the signals of at least one electronic input device that utilizes rotational movement automatically returns the onscreen object to a home position on the display.

15. The method of claim 10, wherein the movement for controlling on onscreen object corresponds to rotational movement.

16. The method of claim 2, further comprising storing and timestamping data corresponding to rotational movement of the patterned object, linear movement of the patterned object, clicks, double-clicks, pressure changes of the patterned object on a finger sensor, presence of the patterned object on a finger sensor, and any combination of these before transmitting the multiple sets of output signals to a host system.

17. The method of claim 1, wherein translating the difference into the multiple signals comprises:

c. comparing the linear movement in a direction to a pre-determined threshold, the direction being one of a plurality of directions controllable by a navigation bar;

d. generating signals corresponding to a movement of an onscreen object by a unit of motion in a direction controllable by the navigation bar when the linear movement exceeds the threshold; and

e. repeating steps c and d for remaining plurality of directions controllable by the navigation bar.

18. The method of claim 1, further comprising switching to a host system one or more output signals of only one of the multiple electronic input devices and the at least one electronic input device that utilizes rotational movement.

19. The method of claim 18, wherein the display forms part of a host device, the host device selected from the group consisting of a portable computer, a personal digital assistant, a portable telephone, a portable gaming device, a biometric/health monitor, and a digital camera.

20. The method of claim 1, further comprising storing and timestamping the first and second patterned image data.

21. The method of claim 1, further comprising translating the difference into a change in pressure of a patterned object on a surface of a finger sensor and output signals of a pressure-sensitive button.

22. The method of claim 21, wherein the change in pressure is determined by a presence of the patterned object in discrete regions of the finger sensor.

23. The method of claim 21, wherein the change in pressure is determined by a weighted average of regional patterned object presence data, smoothed average of patterned image data, and smoothed variance of patterned image data.

24. The method of claim 1, wherein the at least one electronic input device that utilizes rotational movement comprises a steering wheel.

25. A system for generating signals of multiple electronic input devices, the system comprising:

a. a biometric image sensor configured to read first and second patterned images;

b. a first module configured to determine differences between the patterned images using row-based correlation, wherein the row-based correlation comprises obtaining a correlation measure between a single row of the first patterned image and a single row of the second patterned image; and

c. a second module configured to convert the differences into linear movement and output signals of multiple electronic input devices and into rotational movement and output signals of at least one electronic input device that utilizes rotational movement and is different from the multiple electronic input devices, wherein the multiple electronic input devices comprise any two or more of a mouse, a joystick, a navigation bar, and a scroll wheel.

26. The system of claim 25, wherein the biometric image sensor is a finger sensor.

27. The system of claim 26, wherein the finger sensor comprises a finger swipe sensor.

28. The system of claim 27, wherein the finger swipe sensor comprises a finger sensor selected from the group consisting of a capacitive sensor, an optical sensor, and a thermal sensor.

29. The system of claim 27, wherein the finger swipe sensor has a resolution of less than 250 dots per inch.

30. The system of claim 26, wherein the finger sensor comprises a finger placement sensor.

31. The system of claim 26, wherein the first module is further configured to detect a pressure of an object on a surface of the biometric image sensor from the first and second patterned images, and the second module is further configured to convert the pressure into a pressure metric.

32. The system of claim 26, wherein the first module comprises a pressure detector coupled to the biometric image sensor.

33. The system of claim 32, wherein the second module comprises a pressure sensitive button emulator unit coupled to the pressure detector unit and the patterned image presence detector.

34. The system of claim 32, wherein the pressure detector is configured to determine a pressure by computing a weighted average of regional patterned image presence data, smoothed average, and smoothed variance between patterned image movement information.

35. The system of claim 26, wherein the first module comprises a patterned image presence detector coupled to the biometric image sensor.

36. The system of claim 35, wherein the patterned image presence detector comprises a finger presence detector.

37. The system of claim 35, wherein the first module further comprises a rotational movement correlator coupled to the biometric image sensor.

38. The system of claim 37, wherein the second module comprises a steering wheel emulator unit coupled to the rotational movement correlator, the linear movement correlator, and the patterned image presence detector, wherein the rotational movement correlator is configured to translate a change in position of a read patterned image into an angle of rotation.

39. The system of claim 35, wherein the second module comprises a mouse emulator unit coupled to the linear movement correlator and the patterned image presence detector.

40. The system of claim 35, wherein the second module comprises a joystick emulator unit coupled to the linear movement correlator and the patterned image presence detector.

41. The system of claim 35, wherein the second module comprises a navigation bar emulator unit coupled to the linear movement correlator and the patterned image presence detector.

42. The system of claim 35, wherein the second module comprises a scroll wheel emulator unit coupled to the linear movement correlator and the patterned image presence detector.

43. The system of claim 35, wherein the patterned image presence detector is configured to compare the variance of raw patterned image data with a first threshold value and a second threshold value to determine a presence of a patterned image on the biometric image sensor.

44. The system of claim 25, wherein the first module comprises a linear movement correlator configured to calculate a linear difference between the first and second patterned images using a row-based correlation algorithm.

45. The system of claim 44, wherein the second module is further configured to smooth linear differences, filter linear differences, non-linearly scale linear differences, or any combination of these, thereby generating signals for controlling an onscreen object.

46. The system of claim 25, further comprising a time interval accumulator coupling the biometric image sensor to the first module.

47. The system of claim 25, further comprising a display for presenting an onscreen object controllable by one or more of the output signals of the multiple electronic input devices and the at least one electronic input device that utilizes rotational movement.

48. The system of claim 47, wherein the biometric image sensor, the first module, and the second module form part of an integral host device.

49. The system of claim 48, wherein the host device is a device selected from the group consisting of a portable computer, a personal digital assistant, a portable telephone, a portable gaming device, a biometric/health monitor, and a digital camera.

50. A system for generating signals of multiple electronic input devices, the system comprising:

a finger sensor configured to read first and second patterned images;

a first module configured to determine a presence of an object on the finger sensor and to determine linear differences between the first and second patterned images using row-based correlation, wherein the row-based correlation comprises obtaining a correlation measure between a single row of the first patterned image and a single row of the second patterned image; and

a second module configured to convert the linear differences into linear movement and output signals of multiple electronic input devices for controlling a computer object, the second module further configured to convert the linear differences into rotational movement and output signals of at least one electronic input device that utilizes rotational movement data and is different from the multiple electronic input devices, wherein the multiple electronic input devices comprise any two or more of a mouse, a joystick, a navigation bar, and a scroll wheel.