Method and apparatus for classifying a touch event on a touchscreen as related to one of multiple function generating interaction layers and activating a function in the selected interaction layer

• US 9,612,689 B2
• Filed: 06/26/2015
• Issued: 04/04/2017
• Est. Priority Date: 02/02/2015
• Status: Active Grant

• Current Assignee: Qeexo Co. (TDK Corporation)
• Original Assignee: Qeexo Co. (TDK Corporation)
• Inventors: Zungri, Leandro Damian, Harrison, Chris, Schwarz, Julia
• Primary Examiner(s): SHENG, TOM V
• Application Number:
  US14/751,589
• Publication Number:
  US 20160224145A1
• Time in Patent Office: 648 Days
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/043 using propagating acoustic waves
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A system for classifying touch events of different interaction layers includes a touch screen configured to display an interactive element, one or more vibro-acoustic sensors coupled to the touch screen, a touch event detector configured to monitor the one or more vibro-acoustic sensors and to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors, wherein the touch event detector is further configured to detect touch events in which the interactive element is touched by a first or a second finger part of a user, and wherein the touch events result in generating the vibro-acoustic signals, and a vibro-acoustic classifier is configured to classify the vibro-acoustic signals and activate corresponding functions in the different layers dependent upon which finger part is used.

Claims

1. A system for classifying finger touch events and generating an event corresponding to one of a plurality of interaction layers, the system comprising:

a touch screen configured to display interactive elements;

one or more vibro-acoustic sensors coupled to the touch screen and configured to sense vibro-acoustic signals;

a touch event detector configured to monitor the onset of user touches;

a buffer to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors, wherein the touch event detector is further configured to detect touch events in which the touch screen is touched by a first finger part or a second finger part of a user, and wherein the touch events result in generating the vibro-acoustic signals;

a classifier configured to classify the vibro-acoustic signals;

a first interaction layer configured to activate a first function responsive to the touch event is by said first finger part touching an interactive element; and

a second interaction layer configured to activate a second function responsive to the touch event is by said second finger part performing a gesture on the touch screen.

2. The system of claim 1, wherein the first function and the second function are selectively activated based on classification of the vibro-acoustic signals by the classifier and wherein said first and second functions are related to different interaction layers.

3. The system of claim 2, wherein the touch event detector is sensitive to distinguishing signal resulting from touching by said first finger part and from touching by said second finger part.

4. The system of claim 2, wherein a vibro-acoustic sensor is an impact sensor, a vibration sensor, an accelerometer, a gyroscope, a strain gauge, a Piezo electric element, or an acoustic sensor.

5. The system of claim 4, wherein the acoustic sensor is a condenser microphone, a piezoelectric microphone or a MEMS microphone.

6. The system of claim 2, wherein the vibro-acoustic signals saved by the buffer is transmitted to the classifier to enable the classifier to classify the vibro-acoustic signals.

7. The system of claim 6, wherein the classifier is configured to identify whether the first finger part or the second finger part of the user is used to touch the interactive element based on classifying the vibro-acoustic signals.

8. The system of claim 7, wherein the first finger part is a finger tip and the second finger part is a finger knuckle.

9. The system of claim 7, wherein the first finger part is a finger pad and the second finger part is a finger knuckle.

10. The system of claim 7, wherein the first finger part is a finger pad and the second finger part is a finger nail.

11. The system of claim 6, wherein the classifier comprises:

a segmentation unit configured to convert and segment the vibro-acoustic signals into a digital representation of the vibro-acoustic signals;

a conversion unit coupled to the segmentation unit and configured to convert the digital representations of the vibro-acoustic signals into electrical signals;

a feature extraction unit coupled to the conversion unit and configured to derive a series of features from the electrical signals; and

a classification unit coupled to the feature extraction unit and configured to use the series of features to distinguish whether a finger part used for the touch event is the first finger part or the second finger part.

12. The system of claim 11, wherein the segmentation unit is configured to sample the vibro-acoustic signals using a sliding window of samples of the vibro-acoustic signals.

13. The system of claim 12, wherein the conversion unit is configured to perform a Fourier Transform on the sampled vibro-acoustic signals having time domain representation to generate the electrical signals having frequency domain representation, and wherein the feature extraction unit is configured to calculate a series of features from the frequency domain representation of the electrical signals.

14. The system of claim 1, wherein said first and second finger parts are detected as distinctly different touch events.

15. The system of claim 1, wherein each of said first and second finger parts is a different one of a finger pad, a fingertip, a finger nail, and a finger knuckle.

16. A method for classifying touch events, the method comprising:

displaying an interactive element of a touch screen;

monitoring one or more vibro-acoustic sensors coupled with the touch screen to save vibro-acoustic signals sensed by the one or more vibro-acoustic sensors;

detecting touch events in which the interactive element is touched by a first or a second finger part of a user, wherein the touch events result in generating touch signals;

classifying the vibro-acoustic signals;

determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen based on the classified vibro-acoustic signals; and

selectively activating a first function from a first interaction layer if the interactive element is touched by the first finger part or a second function from a second interaction layer if a gesture is performed on the touch screen by the second finger part.

17. The method of claim 16, wherein said first and second functions operating in different interaction layers.

18. The method of claim 17, wherein classifying the vibro-acoustic signals comprises:

segmenting the vibro-acoustic signals into digital representations;

converting the digital representation of the vibro-acoustic signals into electrical signals; and

deriving a series of features from the electrical signals.

19. The method of claim 18, wherein said segmenting the vibro-acoustic signals comprises sampling the vibro-acoustic signals using a sliding window of samples of the vibro-acoustic signals.

20. The method of claim 19, wherein converting the digital representation of the vibro-acoustic signals comprises performing a Fourier Transform on sampled vibro-acoustic signals having time domain representation to generate the electrical signals having frequency domain representation.

21. The method of claim 20, wherein said deriving the series of features comprises calculating a series of features from the frequency domain representation of the electrical signals, and distinguishing whether a finger part used for the touch event is the first finger part or the second finger part based on the series of features.

22. The method of claim 16, wherein each of said first and second finger parts is a different one of a finger pad, a fingertip, a finger nail and a finger knuckle.

23. A non-transitory computer readable medium containing instructions for classifying touch events, wherein execution of the program instructions by a processor causes the processor to carry out the steps of:

displaying an interactive element on a touch screen;

monitoring one or more vibro-acoustic sensors coupled to the touch screen to save vibro-acoustic signals sensed by the one or more vibro-acoustic sensors;

detecting touch events in which the touch screen is touched by a first or a second finger part of a user, wherein the touch events result in generating the vibro-acoustic signals;

classifying the vibro-acoustic signals;

determining whether the first finger part or the second finger part is used to touch the touch screen based on said classifying; and

activating a first function from a first interaction layer based on the first finger part touching the interactive element, and activating a second function from a second interaction layer based on the second finger part performing a gesture on the touch screen.

24. The computer readable medium of claim 23 wherein the first interaction layer is associated with a running program function, and wherein the second interaction layer a gesture function.