Method for locating an impact on a surface and device therefor

• US 7,345,677 B2
• Filed: 06/12/2003
• Issued: 03/18/2008
• Est. Priority Date: 06/12/2002
• Status: Expired due to Term

• Current Assignee: Elo Touch Solutions, Inc., Universite Paris - Sud
• Original Assignee: Sensitive Object
• Inventors: Ing, Ros, Fink, Mathias, Quieffin, Nicolas, Catheline, Stefan
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Shapiro; Leonid
• Application Number:
  US10/518,152
• Publication Number:
  US 20050174338A1
• Time in Patent Office: 1,741 Days
• Field of Search: 345/173, 345/177, 345/179, 345/182
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0433 in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member

Abstract

Apparatus and methods for locating an impact point on a surface of an object. The object is provided with an acoustic sensor that senses the impact from the acoustic waves that are generated when the object is subjected to the impact. The location of the impact on the surface of the object is determined by a recognition step that includes a comparison of the sensed signal to one or more predetermined signals. Each predetermined signal represents an active zone on the surface of the object. A predetermined signal corresponds to the sensed signal when the impact causing the sensed signal is generated in the active zone represented by the predetermined signal. The impact is associated with the active zone represented by a predetermined signal when the sensed signal and the predetermined signal sufficiently correspond to each other.

Claims

1. A method for locating a position of an impact on a surface of an object forming an acoustic interface, the surface provided with at least one acoustic sensor, the method comprising:

measuring at least one sensed signal from acoustic waves generated in the object forming an acoustic interface by said impact;

localizing said position of said impact on said surface by processing of said at least one sensed signal, the processing characterized by a comparison of a sensed signal in the at least one sensed signal with at least one predetermined signal in a plurality of predetermined signals, wherein

each respective predetermined signal in said plurality of predetermined signals corresponds to an active zone in a plurality of active zones on said surface, and

each respective predetermined signal in said plurality of predetermined signals represents a signal that is sensed when a reference impact is generated on the active zone in said plurality of active zones that corresponds to the respective predetermined signal, and wherein,

the position of the impact is associated with an active zone in said plurality of active zones by said localizing when the sensed signal is sufficiently similar to said predetermined signal corresponding to the active zone.

2. The method of claim 1, wherein said localizing comprises comparing the sensed signal with said plurality of predetermined signals, each respective predetermined signal in said plurality of predetermined signals corresponding to a signal sensed when an impact is generated on a corresponding one of said active zones in said plurality of active zones.

3. The method of claim 1 wherein

said at least one acoustic sensor comprises a plurality of acoustic sensors;

said at least one sensed signal comprises a plurality of sensed signals, wherein each sensed signal in said plurality of sensed signals is detected by a different acoustic sensor in said plurality of acoustic sensors; and

said localizing comprises comparing, using a first comparison technique, a first sensed signal in the plurality of sensed signals with one or more predetermined signals in said plurality of predetermined signals that were measured using the same acoustic sensor that sensed said first sensed signal, wherein said comparing of said first sensed signal is performed independent of all other comparisons of sensed signals.

4. The method of claim 3, wherein said localizing further comprises comparing, using a second comparison technique, a second sensed signal in the at least one sensed signals with one or more predetermined signals in said plurality of predetermined signals that were measured using the same acoustic sensor that sensed said second sensed signal, wherein said first comparison technique and said second comparison technique are different from one another.

5. The method of claim 1, wherein said at least one acoustic sensor comprises a plurality of acoustic sensors that sense said at least one sensed signal at several different magnitudes.

6. The method of claim 1 wherein said at least one acoustic sensor consists of one acoustic sensor or two acoustic sensors.

7. The method of claim 1 wherein said at least one acoustic sensor consists of a single acoustic sensor.

8. The method of claim 1, the method further comprising:

experimentally determining a predetermined signal in said plurality of predetermined signals, said experimentally determining comprising;

(i) generating at least one impact in an active zone on the surface of said object, said active zone corresponding to the predetermined signal; and

(ii) measuring a signal caused by the at least one impact using one or more acoustic sensors in said at least one acoustic sensor.

9. The method of claim 1, the method further comprising:

(i) generating at least one impact in an active zone on a surface of a reference object that is identical to or very similar to said object, said active zone corresponding to the predetermined signal; and

(ii) measuring a signal caused by the at least one impact using one or more acoustic sensors in said at least one acoustic sensor.

10. The method of claim 1, wherein said comparison of said at least one sensed signal with said at least one predetermined signal comprises comparison by intercorrelation.

11. The method of claim 1, wherein said comparison of said sensed signal with said at least one predetermined signal comprises comparison by voice recognition, signal recognition, shape recognition or recognition by a neural network.

12. The method of claim 1 wherein said position of said impact is associated with either (i) a unique active zone in said plurality of active zones or (ii) no active zone in said plurality of active zones.

13. The method of claim 1, wherein each active zone in said plurality of active zones corresponds to a predetermined information element and, when the position of the impact is associated with an active zone, an electronic device is made to use the predetermined information element corresponding to the active zone.

14. The method of claim 1, wherein the surface of the object forming an acoustic interface comprises a plurality of active zones and the localizing comprises:

performing a respective intercorrelation between the sensed signal S(t) and each said predetermined signal R_i(t) in said at least one predetermined signals, wherein i is a natural integer between 1 and n and wherein i designates an active zone in said plurality of active zones, thereby obtaining one or more respective intercorrelation functions Ci(t);

determining a potentially activated active zone j corresponding to an intercorrelation function Cj(t), in the one or more respective intercorrelation functions Ci(t), wherein the intercorrelation function Cj(t) has a maximum amplitude greater than those of the other intercorrelation functions Ci(t);

determining the distribution D(i) of the amplitude maxima of the intercorrelation functions wherein;

D(i)=Max (Ci(t));

determining the distribution D′

(i) of the amplitude maxima of the intercorrelation function C′

i(t) between (i) the predetermined signal corresponding to the potentially activated zone, Rj(t), and (ii) each respective predetermined signal Ri(t) wherein;

D′

(i)=Max(C′

i(t));

computing an intercorrelation between D′

(i) and D(i); and

determining whether the impact was generated on the active zone j as a function of a level of correlation between the distributions D(i) and D′

(i) computed by said intercorrelation between D′

(i) and D(i).

15. The method of claim 1, the method further comprising:

extracting data from a sensed signal in said at least one sensed signal representative of a sensed characteristic of the sensed signal;

extracting data from a predetermined signal in said plurality of predetermined signals representative of a reference characteristic of the predetermined signal; and

wherein

said comparison of the sensed signal with the least at one predetermined signals comprises comparing the sensed characteristic to the reference characteristic.

16. The method of claim 15 wherein said sensed characteristic is formulated as a first code and wherein said comparison of the sensed signal with the at least one predetermined signals comprises comparing the first code with a table of codes, wherein each code in said table of codes represents data from a predetermined signal corresponding to an active zone in said plurality of active zones.

17. The method of claim 1, wherein the object forming an acoustic interface comprises a plurality of active zones and wherein said localizing comprises

determining a plurality of resemblance values, each resemblance value representative of a resemblance between the sensed signal and a predetermined signal in said plurality of predetermined signals;

associating the position of the impact (I) with a plurality of adjacent active zones as a function of said plurality of resemblance values; and

identifying the position of the impact (I) on the surface based on a function of the resemblance values attributed to the plurality of adjacent active zones associated with said impact.

18. The method of claim 17, wherein said identifying the position of the impact (I) on the surface based on said function comprises correlating the resemblance values corresponding to the plurality of adjacent active zones to one or more theoretical resemblance values computed for said plurality of adjacent active zones for an impact generated in said position on the surface.

19. The method of claim 18, wherein a theoretical resemblance value in said one or more theoretical resemblance values is a function of the position of the impact on the surface wherein said function is determined prior to said localizing for said plurality of adjacent active zones.

20. The method of claim 1 wherein said comparison of the sensed signal with at least one predetermined signal in the plurality of predetermined signals comprises comparing a phase of a predetermined signal in the plurality of predetermined signals with a phase of the sensed signal.

21. The method of claim 20, the method further comprising:

computing a Fourier transform R_i(ω

)=|R_i(ω

)|·

e<sup>jφ

i(ω

)</sup> of a predetermined signal in the plurality of predetermined signals that corresponds to an active zone i in said plurality of active zones;

computing a Fourier transform S(ω

)=S(ω

)|·

e<sub>j φ

i(ω

)</sub> of a sensed signal in the at least one sensed signals;

wherein

said comparison of the sensed signal in the at least one sensed signal with said at least one predetermined signal in the plurality of predetermined signals comprises comparing;

S′

(ω

)to R′

_i(ω

)

wherein,

S′

(ω

) is the phase component of the Fourier transform of the sensed signal for those frequency bands ω

in which the amplitude |S(ω

) is greater than a predetermined threshold; and

R′

_i(ω

) is the phase component of the Fourier transform of the predetermined signal for those frequency bands ω

in which the amplitude | R_i (ω

)|is greater than a predetermined threshold.

22. The method of claim 21, wherein the predetermined threshold is equal to the maximum of MAX/D and |B(ω

)|, where;

MAX is the maximal value of the amplitude |R_i(ω

)|, the maximal value of amplitude |R_i(ω

)| normalized in energy, or the maximal value of an envelope of an average of amplitude |R_i (ω

)| normalized in energy,

D is a constant, and

|B(ω

)| is an average of a plurality of noise spectra in the object forming an acoustic interface, acquired at different times.

23. The method of claim 21, the method further comprising:

computing, for each active zone i in the plurality of active zones,

Pi(ω

)=SN(ω

) multiplied by the conjugate of RN(ω

);

normalizing, for each active zone i in the plurality of active zones, Pi(ω

);

obtaining a plurality of temporal functions, each temporal function Xi(t) in said plurality of temporal functions corresponding to an active zone i in the plurality of active zones, wherein Xi(t) for a respective active zone i is an inverse Fourier transform of the product Pi(ω

) for the respective active zone; and

attributing the signal S(t) to an active zone in the plurality of active zones as a function of said temporal functions.

24. The method of claim 23, wherein the signal S(t) is attributed to the active zone corresponding to a temporal function having a maximum value that is greater than the maximum value of any other temporal function in said plurality of temporal functions.

26. The method of claim 3, the method further comprising repeating said localizing for each respective sensed signal in the plurality of sensed signals.

27. The method of claim 1, wherein said object is a pane, a door, a window, a portable tray, a computer screen, a display panel, an interactive terminal, a toy, a vehicle dashboard, a rear of a front seat back of an automobile vehicle, a rear of an airplane seat, a wall, a floor, or a vehicle fender.

28. The method of claim 1 wherein a sensor in said at least one sensor is a piezoelectric sensor, a capacitive sensor, a magnetostrictive sensor, an electromagenetic sensor, an acoustic velocimeter, or an optical sensor.

29. The method of claim 1 wherein an active zone in said plurality of active zones is delimited on said surface by a physical marking.

30. The method of claim 1 wherein an active zone in said plurality of active zones is delimited by projecting an image onto said surface.

31. The method of claim 13 wherein the predetermined information element is a command, a digit, a letter, or a position on said surface.

32. The method of claim 1 wherein said plurality of active zones comprise a virtual keyboard and wherein said impact is caused by hitting an active zone in said plurality of active zones with an objected selected from the group consisting of a fingernail, a fingertip, a pen, or a style.

25. A device for locating a position of an impact on a surface forming part of an object forming an acoustic interface, provided with at least one acoustic sensor the device comprising:

means for measuring at least one sensed signal from acoustic waves generated in the object forming said acoustic interface by said impact, and

means for localizing the position of the impact on said surface by processing a sensed signal in said at least one sensed signals, characterized in that said means for localizing comprises;

recognition means suitable for comparing the sensed signal with at least one predetermined signal, each respective predetermined signal in said at least one predetermined signal corresponding to a signal that is sensed when an impact is generated on an active zone forming part of the surface of the object that corresponds to the respective predetermined signal, and

means for associating the location of the impact with said active zone when the sensed signal is sufficiently similar to said predetermined signal.

33. A method of identifying a location of an impact on a surface of an object, wherein said surface is delineated into a plurality of active zones, the method comprising:

measuring a sensed signal caused by said impact;

comparing said sensed signal with a library of predetermined signals, each predetermined signal in said library of predetermined signals corresponding to an active zone in said plurality of active zones, wherein each predetermined signal in said library of predetermined signals is determined experimentally by generating at least one impact on each active zone in the plurality of active zones;

wherein

when a correspondence between said sensed signal and a respective predetermined signal in said plurality of sensed signals is sufficiently similar, said location of said impact is deemed to be in the active zone corresponding to said respective predetermined signal.

34. The method of claim 33 wherein said comparing comprises intercorrelating said sensed signal with a predetermined signal in said library of predetermined signals.

35. The method of claim 33 wherein said sensed signal is normalized prior to said comparing.

36. The method of claim 33, the method further comprising:

converting said sensed signal to a sensed code representative of said sensed signal and wherein

said library of signals comprises a plurality of predetermined codes, each predetermined code representing a different signal in said plurality of predetermined signals; and

wherein

said comparing comprises comparing said sensed code with a predetermined code in said plurality of predetermined codes.

37. The method of claim 36, wherein said sensed code is a 16-bit code wherein

(a) the first eight bits of said 16-bit code are determined by a frequency spectrum of the sensed signal that is subdivided into eight predetermined frequency tranches [f_k, f_k], wherein k=1 . . . 8 and wherein the bit of rank k is equal to

1 when a final energy value given by the spectrum at frequency f_k+1 is greater than an average energy value of an acoustic wave in the frequency tranches [f_k, f_k+1] and

0 when a final energy value given by the spectrum at frequency f_k+1 is not greater than the average energy value of the acoustic wave in the frequency tranches [f_k, f_k+1];

and wherein

(b) the last eight bits of the code are determined from the sensed signal when it is subdivided into nine predetermined temporal tranches [t_k, t_k], wherein k=1 . . . 9 and wherein the bit of rank k+8 is equal to

1 when an average value of a signal power during the period [t_k, t_k+2] is greater than an average value of the signal power during the period [t_k+1, t_k+2], k=1 . . . 8,

2 when an average value of a signal power during the period [t_k, t_k+1] is not greater than an average value of the signal power during the period [t_k+1, t_k+2], k=1 . . . 8.

38. A computer for locating a position of an impact on a surface of an object forming an acoustic interface, the surface provided with at least one acoustic sensor, the computer comprising:

instructions for receiving measurements of at least one sensed signal from acoustic waves generated in the object forming an acoustic interface by said impact;

instructions for localizing said position of said impact on said surface by processing of said at least one sensed signal, the processing characterized by a comparison of a sensed signal in the at least one sensed signal with at least one predetermined signal in a plurality of predetermined signals, wherein

each respective predetermined signal in said plurality of predetermined signals corresponds to an active zone in a plurality of active zones on said surface, and

each respective predetermined signal in said plurality of predetermined signals represents a signal that is sensed when a reference impact is generated on the active zone in said plurality of active zones that corresponds to the respective predetermined signal, and wherein,

the position of the impact is associated with an active zone in said plurality of active zones by said instructions for localizing when the sensed signal is sufficiently similar to said predetermined signal corresponding to the active zone.

39. The computer of claim 38 wherein said acoustic waves generated in the object forming an acoustic interface by said impact are measured by at least one acoustic sensor that is in electrical communication with said computer.

40. A computer for identifying a location of an impact on a surface of an object, wherein said surface is delineated into a plurality of active zones, the computer comprising:

instructions for receiving measurements of a sensed signal from acoustic waves generated in the object by said impact;

instructions for comparing said sensed signal with a library of predetermined signals, each predetermined signal in said library of predetermined signals corresponding to an active zone in said plurality of active zones;

wherein

when a correspondence between said sensed signal and a respective predetermined signal in said library of sensed signals is sufficiently similar, said location of said impact is deemed to originate from the active zone corresponding to said respective predetermined signal.

41. The computer of claim 40 wherein said instructions for comparing comprise instructions for intercorrelating said sensed signal with a predetermined signal in said library of predetermined signals.

42. The computer of claim 40, further comprising instructions for normalizing said sensed signal prior to execution of said instructions for comparing.

43. The computer of claim 40, further comprising:

instructions for converting said sensed signal to a sensed code representative of said sensed signal and wherein

said library of signals comprises a plurality of predetermined codes, each predetermined code in said plurality of predetermined codes representing a different signal in said plurality of predetermined signals; and

wherein

said instructions for comparing comprise comparing said sensed code with a predetermined code in said plurality of predetermined codes.

44. The computer of claim 43, wherein said sensed code is a 16-bit code wherein

(a) the first eight bits of said 16-bit code are determined by a frequency spectrum of the sensed signal that is subdivided into eight predetermined frequency tranches [f_k, f_k+], wherein k=1 . . . 8 and wherein the bit of rank k is equal to

1 when a final energy value given by the spectrum at frequency f_k+1 is greater than an average energy value of an acoustic wave in the frequency tranches [f_k, f_k] and

0 when a final energy value given by the spectrum at frequency f_k+1 is not greater than the average energy value of the acoustic wave in the frequency tranches [f_k, f_k+1];

and wherein

(b) the last eight bits of the code are determined from the sensed signal when it is subdivided into nine predetermined temporal tranches [t_k, t_k], wherein k=1.9 and wherein the bit of rank k+8 is equal to

1 when an average value of a signal power during the period [t_k, t_k+1] is greater than an average value of the signal power during the period [t_k+1, t_k+2], k=1 . . . 8,

0 when an average value of a signal power during the period [t_k, t_k+1] is not greater than an average value of the signal power during the period [t_k+1, t_k+2], k=1 . . . 8.