METHOD AND APPARATUS FOR CLASSIFYING A TOUCH EVENT ON A TOUCHSCREEN AS RELATED TO ONE OF MULTIPLE FUNCTION GENERATING INTERACTION LAYERS AND ACTIVATING A FUNCTION IN THE SELECTED INTERACTION LAYER

• US 20160224145A1
• Filed: 06/26/2015
• Published: 08/04/2016
• Est. Priority Date: 02/02/2015
• Status: Active Grant

• Current Assignee: Qeexo Co. (TDK Corporation)
• Original Assignee: Qeexo Co. (TDK Corporation)
• Inventors: ZUNGRI, LEANDRO DAMIAN, HARRISON, CHRIS, SCHWARZ, JULIA
• Granted Patent: US 9,612,689 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/043 using propagating acoustic waves
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A system for classifying touch events of different interaction layers includes a touch screen configured to display an interactive element, one or more vibro-acoustic sensors coupled to the touch screen, a touch event detector configured to monitor the one or more vibro-acoustic sensors and to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors, wherein the touch event detector is further configured to detect touch events in which the interactive element is touched by a first or a second finger part of a user, and wherein the touch events result in generating the vibro-acoustic signals, and a vibro-acoustic classifier is configured to classify the vibro-acoustic signals and activate corresponding functions in the different layers dependent upon which finger part is used.

Claims

1. A system for classifying finger touch events and generating an event corresponding to one of a plurality of interaction layers, the system comprising:

a touch screen configured to display interactive elements;

one or more vibro-acoustic sensors operating inside the device;

a touch event detector configured to monitor the onset of user touches;

a buffer to save touch and vibro-acoustic signals sensed by the one or more vibro acoustic sensors and the touchscreen, wherein the touch event detector is further configured to detect touch events in which the interactive element is touched by a first or a second finger part of a user, and wherein the touch events result in generating the touch and vibro-acoustic signals;

a touch and vibro-acoustic classifier configured to classify the touch and vibro-acoustic signals; and

a plurality of interaction layers of function generation responsive to whether the touch event is by said first finger part or by said second finger part.

2. The system of claim 1, wherein a first function or a second function related to the interactive element is selectively activated based on classification of the touch and vibro-acoustic signals by the touch and vibro-acoustic classifier and wherein said first and second functions are related to different interaction layers of said system.

3. The system of claim 1, wherein said first and second finger parts are detected as distinctly different touch screen events.

4. The system of claim 2, wherein the touch event detector is sensitive to distinguishing signal characteristics resulting from touching by said first finger part and from said second finger part.

5. The system of claim 2, wherein a vibro-acoustic sensor is an impact sensor, a vibration sensor, an accelerometer, a gyroscope, a strain gauge, a Piezo electric element, or an acoustic sensor.

6. The system of claim 5, wherein the acoustic sensor is a condenser microphone, a piezoelectric microphone or a MEMS microphone.

7. The system of claim 2, wherein the vibro-acoustic signals saved by the touch event detector is transmitted to the classifier to enable the classifier to classify the touch and vibro-acoustic signals.

8. The system of claim 7, wherein the vibro-acoustic classifier identifies whether the first finger part or the second finger part of the user is used to touch the interactive element based on classifying the touch and vibro-acoustic signals.

9. The system of claim 8, wherein the first finger part is a finger tip and the second finger part is a finger knuckle.

10. The system of claim 8, wherein the first finger part is a finger pad and the second finger part is a finger knuckle.

11. The system of claim 8, wherein the first finger part is a finger pad and the second finger part is a finger nail.

12. The system of claim 1, wherein each of said first and second finger parts is a different one of a finger pad, a fingertip, a finger nail, and a finger knuckle.

13. The system of claim 7, wherein the classifier comprises:

a segmentation unit configured to convert and segment the vibro-acoustic signals into a digital representation of the vibro-acoustic signals;

another segmentation unit to generate a digital representation of touchscreen signals;

a conversion unit coupled to the segmentation units and configured to convert the digital representations of the vibro-acoustic and touchscreen signals into electrical signals;

a feature extraction unit coupled to the conversion unit and configured to derive a series of features from the electrical signals; and

a classification unit coupled to the feature extraction unit and configured to use the series of features to distinguish whether a finger part used for the touch event is the first finger part or the second finger part.

14. The system of claim 13, wherein the segmentation units are configured to sample the vibro-acoustic and touchscreen signals using a sliding window of samples of the respective vibro-acoustic and touchscreen signals.

15. The system of claim 14, wherein the conversion unit is configured to perform a Fourier Transform on the sampled vibro-acoustic signals having time domain representation to generate the electrical signals having frequency domain representation, and wherein the feature extraction unit is configured to calculate a series of features from the frequency domain representation of the electrical signals.

16. A method for classifying touch events, the method comprising:

displaying an interactive element of a touch screen;

monitoring one or more vibro-acoustic sensors operating inside the device to save vibro-acoustic signals sensed by the one or more vibro-acoustic sensors;

detecting touch events in which the interactive element is touched by a first or a second finger part of a user, wherein the touch events result in generating touch event signals;

classifying the vibro-acoustic and touch event signals;

determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen based on classified vibro-acoustic and touch event signals; and

selectively generating a corresponding function from a plurality of interaction layers depending upon whether the first finger part or the second finger part is used to touch the interactive element.

17. The method of claim 16, further comprising activating a first function or a second function related to the interactive element based on determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen, said first and second functions operating in different interaction layers.

18. The method of claim 17, wherein classifying the vibro-acoustic and touch event signals comprises:

segmenting the vibro-acoustic signals and touch event signals into digital representations;

converting the digital representation of the vibro-acoustic and touch event signals into electrical signals; and

deriving a series of features from the electrical signals.

19. The method of claim 18, wherein segmenting the vibro-acoustic signals comprise sampling the vibro-acoustic signals using a sliding window of samples of the vibro-acoustic signals.

20. The method of claim 19, wherein converting the digital representation of the vibro-acoustic signals comprises performing a Fourier Transform on sampled vibro-acoustic signals having time domain representation to generate the electrical signals having frequency domain representation.

21. The method of claim 20, wherein deriving the series of features comprise calculating a series of features from the frequency domain representation of the electrical signals, and distinguishing whether a finger part used for the touch event is the first finger part or the second finger part based on the series of features.

22. The method of claim 16, wherein each of said first and second finger parts is a different one of a finger pad, a fingertip, a finger nail and a finger knuckle.

23. A computer readable medium containing instructions for classifying touch events, wherein execution of the program instructions by a processor causes the processor to carry out the steps of:

displaying an interactive element of a touch screen;

monitoring one or more vibro acoustic sensors coupled to the touch screen to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors;

detecting touch events in which the interactive element is touched by a first or a second finger part of a user, wherein the touch events result in generating the vibro-acoustic signals;

classifying at least one of the vibro-acoustic signals and the touch events;

determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen based on said classifying; and

initiating a function selection from different interaction layers depending whether the first finger part or the second finger part was used to touch the interactive element.

24. The computer readable medium recited in claim 23 further comprising processor steps to treat a first interaction layer as a running program function and to treat a second interaction layer as a gesture function.