Method and apparatus for providing user interface using acoustic signal, and device including user interface

• US 9,857,920 B2
• Filed: 02/02/2011
• Issued: 01/02/2018
• Est. Priority Date: 02/02/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Chae, Seung Chul, Lee, Sun Jae, Chang, Kyung Ah, Lee, Ho Sub, Choi, Hyun Jin
• Primary Examiner(s): Lee, Nicholas
• Assistant Examiner(s): STONE, ROBERT M
• Application Number:
  US13/019,566
• Publication Number:
  US 20110191680A1
• Time in Patent Office: 2,526 Days
• Field of Search: 345177
• CPC Class Codes: G06F 3/0436 in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

A user interface using an acoustic signal is provided. A device with the user interface includes a sensor to acquire a surface acoustic signal generated outside of the device, and at least one processor. The processor classifies patterns of the surface acoustic signal by analyzing features of the surface acoustic signal, and identifies a user'"'"'s input signal corresponding to the surface acoustic signal based on the pattern of the surface acoustic signal.

Claims

1. An apparatus to provide a user interface of a device, comprising:

a surface acoustic signal acquisition unit configured to acquire a surface acoustic signal propagated through a medium disposed outside of the device;

a pattern recognition unit configured to (1) estimate a type of a medium using a test signal, (2) estimate a tool generating the surface acoustic signal based on frequency-domain features of the surface acoustic signal and (3) recognize a pattern of the surface acoustic signal based on the type of the medium and the tool by analyzing features of the surface acoustic signal, the pattern recognition unit comprising

a feature extractor configured to extract at least one time-domain feature of the surface acoustic signal, and

a pattern analyzer configured to analyze the pattern of the surface acoustic signal based on the at least one time-domain feature, the type of the medium and the tool; and

a user input identification unit configured to identify a user'"'"'s input signal by classifying minimal meaningful units corresponding to the pattern of the surface acoustic signal and combining the minimal meaningful units to form one or more input characters.

2. The apparatus of claim 1, wherein the medium is provided to the user through an object contacting the device.

3. The apparatus of claim 2, wherein the surface acoustic signal acquisition unit comprises:

a medium contactor to receive the surface acoustic signal by contacting the object; and

an acoustic sensor to convert the surface acoustic signal to a digital signal.

4. The apparatus of claim 1, wherein the medium is provided to the user through an external casing of the device.

5. The apparatus of claim 1, wherein:

the feature extractor is configured to extract at least one frequency-domain feature; and

the pattern analyzer is configured to analyze the pattern of the surface acoustic signal based on the at least one frequency-domain feature.

6. The apparatus of claim 1, wherein the at least one time-domain feature comprises an amplitude of the surface acoustic signal, a distribution of a zero-crossing, an amount of energy, a length of the surface acoustic signal, and/or a shape of the surface acoustic signal.

7. The apparatus of claim 1, wherein the user input identification unit identifies the user'"'"'s input signal by receiving a plurality of patterns sequentially from the pattern recognition unit and by combining minimal meaningful units corresponding to the plurality of patterns.

8. The apparatus of claim 1, wherein the pattern comprises information corresponding to a direction of the surface acoustic signal, and

the user input identification unit identifies the user'"'"'s input signal based on the information corresponding to the surface acoustic signal direction.

9. The apparatus of claim 1, further comprising a pattern model storage unit to store pattern models of the surface acoustic signal, classified according to a type of the medium and/or an input tool.

10. The apparatus of claim 1, wherein the user'"'"'s input signal comprises a character, a figure, a dragging, a touch event, or any combination thereof, which are input through the medium.

11. The apparatus of claim 1, wherein the at least one time-domain feature comprises a total sum of energy based on a summation of discrete energy measurements distributed over a length of time.

12. The apparatus of claim 1, wherein the amplitude of the signals are used to identify a straight line, a curve line, a bent line, being input onto the medium.

13. The apparatus of claim 1, wherein the feature extractor is configured to extract specific features of the corresponding signal using a zero-crossing distribution and a variation of the zero-crossing distribution such that the distribution of the zero-crossing denotes a number of spots at which a sign of the signal changes.

14. The apparatus of claim 1, wherein the tool is a stylus.

15. A method of providing a user interface of a device, the method comprising:

acquiring a surface acoustic signal;

extracting at least one time-domain feature of the surface acoustic signal;

determining a type of a medium using a test signal;

estimating a tool generating the surface acoustic signal based on frequency-domain features of the surface acoustic signal;

analyzing a pattern of the surface acoustic signal based on the at least one time-domain feature, the type of the medium and the tool; and

identifying a user'"'"'s input signal by classifying minimal meaningful units corresponding to the pattern of the surface acoustic signal and combining the minimal meaningful units to form one or more input characters.

16. The method of claim 15, wherein the identifying comprises:

receiving a plurality of patterns sequentially; and

identifying the user'"'"'s input signal by combining minimal meaningful units corresponding to the patterns.

17. The method of claim 15, wherein the surface acoustic signal is generated by contact between the medium and the user'"'"'s hand.

18. The method of claim 15, wherein the surface acoustic signal is generated by contact between the medium and a tool being used by the user.

19. The method of claim 15, wherein the surface acoustic signal is acquired in a state in which the device is in contact with the medium.

20. The method of claim 15, comprising:

extracting at least one frequency domain feature of the surface acoustic signal; and

analyzing the pattern of the surface acoustic signal based on the at least one frequency domain feature.

21. A method of providing a user interface of a device, the method comprising:

acquiring an acoustic signal generated by a user interacting with a medium of an object outside of the device;

analyzing the acquired acoustic signal through a process that comprises separating the surface acoustic signal into a plurality of frames based on at least one time-domain feature of the acquired acoustic signal; and

identifying the user'"'"'s input signal by classifying minimal meaningful units corresponding to a pattern of the acoustic signal and combining the minimal meaningful units to form one or more input characters.