Heterogeneous tactile sensing via multiple sensor types using spatial information processing

• US 10,042,479 B2
• Filed: 11/06/2017
• Issued: 08/07/2018
• Est. Priority Date: 12/06/2011
• Status: Active Grant

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Inventors: Ludwig, Lester F., Zaliva, Vadim
• Primary Examiner(s): Sadio, Insa
• Application Number:
  US15/804,778
• Publication Number:
  US 20180059873A1
• Time in Patent Office: 274 Days
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H03K 17/96 Touch switches specially adapted for electronic time-pieces with no moving parts G04G21/08

Abstract

This invention relates to sensor arrangements and signal processing architectures for touch-based user interfaces comprising multiple sensor types and other arrangements so as to create user interface output signals responsive to the touch of at least one human finger. Sensor types and other arrangements can include capacitive tactile sensor arrays, optical tactile sensor arrays, proximity sensor arrays, pressure sensor arrays, and video cameras. At least one software algorithm comprises at least one of a functional partition (wherein some user interface output signals are derived only from a pre-specified sensor), decision-based selection (wherein some user interface output signals are selectively derived from a selected sensor), or spatial information processing (wherein user interface output signals are obtained from threshold testing, conditional testing, vector quantization, algorithms employing parameterized calculations, algorithms employing compensation calculations and operations, artificial neural networks, etc.).

Claims

1. A system for implementing a user interface, the system comprising:

a processor for executing instructions;

a first tactile sensing arrangement for providing a plurality of tactile sensing measurements to the processor, wherein a first tactile sensing measurements is in response to receiving tactile input provided by an object; and

a secondary information sensing arrangement for providing secondary information to the processor, wherein the secondary information is in response;

a second tactile sensing arrangement providing another tactile sensing measurement, wherein the second tactile sensing arrangement is separate from the first tactile sensing arrangement;

wherein the processor is configured to execute instructions to;

in response to a combination of the first tactile sensing measurements when the object is in physical contact with the first tactile sensing arrangement and the secondary information from the second tactile sensing arrangement when the object is to be measured by the secondary information sensing arrangement, generate user interface output signals, wherein the user interface output signals are produced from spatial information processing acting on one or more values of the first tactile sensing measurements and the secondary information from the combination when the object is in physical contact with the first tactile sensing arrangement.

2. The system of claim 1, wherein the first tactile sensing arrangement is a capacitive tactile sensor and the secondary information sensing arrangement comprises second tactile input provided from an optical tactile sensor.

3. The system of claim 2 wherein the secondary information sensing arrangement further comprises at least one video camera.

4. The system of claim 1, wherein the first tactile sensing arrangement is an optical tactile sensor and the secondary information sensing arrangement comprises second tactile input provided from a capacitive tactile sensor.

5. The system of claim 4 wherein the secondary information sensing arrangement further comprises at least one video camera.

6. The system of claim 1, wherein the first tactile sensing arrangement is a proximity sensor and the secondary information sensing arrangement comprises second tactile input provided from a pressure sensor.

7. The system of claim 1, wherein the first tactile sensing arrangement is a pressure sensor the secondary information sensing arrangement comprises second tactile input provided from a proximity sensor.

8. The system of claim 1 wherein the secondary information sensing arrangement comprises at least one video camera.

9. The system of claim 1 wherein the secondary information sensing arrangement comprises at least two video cameras.

10. The system of claim 1 further comprising a functional partition to generate first user interface output signals based on a combination of tactile sensing measurements and secondary information from a first set of tactile sensing arrangements and secondary information sensing arrangements, and to generate second user interface output signals based on a combination of tactile sensing measurements and secondary information from a second set of tactile sensing arrangements and secondary information sensing arrangements.

11. The system of claim 10 further comprising an internal decision arrangement to select one set from the first and second sets of tactile sensing arrangements and secondary information sensing arrangements, the selection made responsive to corresponding measurements by at least a subset of the first and second sets of tactile sensing arrangements.

12. The system of claim 10 further comprising an external decision arrangement to select one set from the first and second sets of tactile sensing arrangements and secondary information sensing arrangements, the selection made responsive to measurements by a third set of tactile sensing arrangements and secondary information sensing arrangements.

13. The system of claim 1 wherein the instructions comprise sensor-fusing wherein the user interface output signals are obtained from threshold testing.

14. The system of claim 1 wherein the instructions comprise sensor-fusing wherein the user interface output signals are obtained from conditional testing.

15. The system of claim 1 wherein the instructions comprise sensor-fusing wherein the user interface output signals are obtained from vector quantization.

16. The system of claim 1 wherein the instructions comprise sensor-fusing wherein the user interface output signals are obtained from parameterized calculations.

17. The system of claim 1 wherein the instructions comprise sensor-fusing wherein the user interface output signals are obtained from compensation calculations and operations.

18. The system of claim 1 wherein the instructions comprise sensor-fusing wherein the user interface output signals are obtained from an artificial neural network.