Method and apparatus for approximating a source position of a sound-causing event for determining an input used in operating an electronic device

• US 6,690,618 B2
• Filed: 04/02/2002
• Issued: 02/10/2004
• Est. Priority Date: 04/03/2001
• Status: Expired due to Fees

• Current Assignee: Heilongjiang Golden Jumping Group Co., Ltd.
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Sürücü, Fahri, Tomasi, Carlo
• Primary Examiner(s): Lobo, Ian J.
• Application Number:
  US10/115,357
• Publication Number:
  US 20020167862A1
• Time in Patent Office: 679 Days
• Field of Search: 367/124, 367/127, 367/129, 367/907
• US Class Current: 367/127
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 5/22 Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements G01S5/28 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/043 using propagating acoustic waves
• CPC Class Codes: G06F 3/0433 in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member
• CPC Class Codes: G06F 3/0436 in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
• CPC Class Codes: G08B 13/19695 Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves
• CPC Class Codes: Y10S 367/907 Coordinate determination

Abstract

A plurality of sound-detecting devices are configured to detect sound from an event of interest. Information about the sound is recorded that is dependent on a distance between the source position and the location of the sound-detection devices. The source position is approximated using the recorded information and the relative position of individual sound-detection devices to one another. The approximated source position is used as input for operating a electronic device.

Claims

1. A method for approximating a source position of a sound-causing event for determining an input used in operating an electronic device, the method comprising:

monitoring an area using a plurality of detection points for an occurrence of the event at any location within the area;

detecting a sound caused by the event occurring at a location corresponding to the source position using the plurality of detection points, the source position being within the area that is being monitored;

independently of knowing a time in which the sound of the event is emitted, recording information about the sound at individual detection points in the plurality of detections points, the information being dependent on a distance between the source position and each of the individual detection points;

determining, approximately, the source position based on the recorded information and on a relative position of individual detection points in the plurality of detection points to other detection points in the plurality of detection points;

determining the input from the source position; and

causing an operation to be performed by the electronic device based on the input.

2. The method of claim 1 wherein detecting a sound includes detecting the sound emitted by a user contacting the area that is being monitored at the location corresponding to the source position, wherein the source position is part of a first designated region wherein the area is divided into a plurality of designated regions that include the first designated region, each of the plurality of designated regions being associated with a corresponding input, and wherein determining the input includes determining the corresponding input for the first designated region.

3. The method of claim 1, wherein detecting a sound includes detecting the sound emitted by a user contacting the monitored region at the location corresponding to the source position, wherein the source position is part of a first designated region and wherein the area is divided into a plurality of regions, each of the plurality of designated regions being associated by the electronic device as a corresponding key in a set of keys, and wherein determining the input includes determining the corresponding key for the first designated region.

4. The method of claim 1, wherein detecting a mound includes detecting the sound emitted by a voice of a person at the location corresponding to the source position, and wherein determining the input includes determining at least a first range value for focusing an image capturing device on the source position.

5. The method of claim 4, wherein determining at least a first range value includes determining range values in at least two dimensions.

6. The method of claim 1, wherein recording information about the sound at individual detection points includes:

identifying an amplitude of the sound at each of the plurality of detection points; and

wherein determining the source position includes;

comparing the amplitude identified at each of the plurality of detection points to the amplitude of the sound identified at one or more other detection point.

7. The method of claim 1, wherein recording information about the sound at individual detection points includes:

determining a time value corresponding to a determination of an arrival of the sound at each of the plurality of detection points; and

wherein determining the source position includes;

comparing the time value determined at each of the plurality of detection points to the time value determined at one or more other detection points in the plurality of detection points.

8. The method of claim 6, wherein comparing the amplitude identified at each of the plurality of detection points includes:

determining a first comparison value representing a difference between the amplitude at each of a first pair of detection points in the plurality of detection points; and

using the first value to determine a first range value of the source position from a designated position along an axis defining a first dimension between the source position and the designated position.

9. The method of claim 8, wherein comparing the amplitude identified at each of the plurality of detection points includes:

determining a second comparison value representing a difference between the amplitude at each of a second pair of detection points in the plurality of detection points; and

using the second comparison value to determine a second range value of the source position from the designated position along an axis defining a second dimension between the source position and the designated position.

10. The method of claim 9, wherein comparing the amplitude identified at each of the plurality of detection points includes:

determining a third comparison value representing a difference between the amplitude of each of a third pair of detection points in the plurality of detection points; and

using the third comparison value to approximate a third range value of the source position from the designated position along an axis defining a third dimension between the source position and the designated position.

11. The method of claim 10, wherein comparing the amplitude identified at each of the plurality of detection points includes determining the amplitude of the sound detected at each detection point in a set of three detection points.

12. The method of claim 1, wherein recording information about the sound at individual detection points in the plurality of detections points includes determining a first time value corresponding to a determination of an arrival of the sound at a first detection point in the plurality of detection point, determining a second time value corresponding to a determination of an arrival of the sound at a second detection point in the plurality of detection points, and determining a third time value corresponding to a determination of an arrival of the sound at a third detection point in the plurality of detection points.

13. The method of claim 12, wherein determining the source position includes determining a first comparison value corresponding to a difference of the first time value and the second time value, and a second comparison value corresponding to a difference of the first time value and the third time value.

14. The method of claim 13, wherein determining the source position includes using the first comparison value to determine a first range value of the source position from a designated position along an axis defining a first dimension between the source position and the designated position, and using the second comparison value to determine a second range value of the source position from the designated position along an axis defining a second dimension between the source position and the designated position.

15. The method of claim 14, wherein determining the source position includes determining a third comparison value corresponding to a difference of the first time value and the third time value, and using the third comparison value to determine a third range value of the source position from the designated position along an axis defining a third dimension between the source position and the designated position.

16. The method of claim 1, wherein detecting a sound caused by the event includes marking an approximate pea amplitude of the sound at each detection point in the plurality of detection points as the arrival of the sound at that detection point.

17. The method of claim 1, wherein detecting a sound caused by the event includes detecting an approximate waveform shape of the sound at each detection point in the plurality of detection points, and comparing the approximate waveform shape to a known waveform shape to determine that the sound from the event arrived at that detection point.

18. The method of claim 1, wherein recording information about the sound at individual detection points in the plurality of detections points includes:

identifying an amplitude of the sound at each of the plurality of detection points;

determining a time value corresponding to detection of the sound at each of the plurality of detection points; and

wherein determining the source position includes;

comparing the amplitude identified at each of the plurality of detection points to the amplitude of the sound identified at one or more other detection point to determine at least a first amplitude-determined range value and a second amplitude-determined range value; and

comparing the time value determined at each of the plurality of detection points to the time value determined at one or more other detection points in the plurality of detection points to determine at least a first time-determined range value and a second time-determined range value.

19. The method of claim 18, wherein determines the source position includes determining a first range value by determining an average of the first amplitude-determined range value and the first time-determined range value, the first range being along an axis defining a first dimension between the source position and the designated position.

20. The method of claim 19, wherein determining the source position includes weighting the average of the first amplitude-determined range value and the first time-determined range value.

21. The meted of claim 19, wherein determining the source position includes determining a second range value by determining an average of the amplitude-determined second range value with the time-determined second range value, the second range value being along an axis defining a second dimension between the source position and the designated position.

22. The method of claim 21, wherein determining the source position includes weighting the average of the amplitude-determined range value and the time-determined range value.

23. The method of claim 21, wherein determining the source position includes;

comparing the amplitude identified at each of the plurality of detection points to the amplitude of the sound identified at one or more other detection point to determine a third amplitude-determined range value; and

comparing the time value determined at each of the plurality of detection points to the time value determined at one or more other detection points in the plurality of detection points to determine a third time-determined range value; and

determining an average of the third amplitude-determined range value and the third time-determined range value.

24. The method of claim 23, further comprising weighting the average of the third amplitude-determined range value and the third time-determined range value.

25. An apparatus for approximating a source position of a sound-causing event for determining an input used in operating an electronic device, the system comprising:

a plurality of sound-detection devices arranged to receive a sound caused by the event when the event is caused by an object that is external to the apparatus; and

a processor operatively coupled to the plurality of sound-detection devices, the processor being configured to;

detect that the sound-detection devices receive the sound;

independent of knowing a time in which the sound of the event is emitted, record information about the sound detected at individual sound-detection devices, the information being dependent on a distance between the source position and the individual sound-detection devices;

approximate the source position based on the recorded information and on a relative position of individual sound-detection devices in the plurality of sound-detection devices to other sound-detection devices in the plurality of sound-detection devices;

determine the input from the source position; and

execute an operation based on the input.

26. The apparatus of claim 25, wherein the processor is configured to detect the sound emitted by a user contacting a region selected from a plurality of regions, each of the plurality of regions being associated with a corresponding input, and to determine the corresponding input for the contacted region.

27. The apparatus of claim 26, wherein the processor is configured to detect the sound emitted by a user contacting a region selected from a plurality of regions, each of the plurality of regions being associated assigned a corresponding key in a set of keys by the processor, and wherein the processor is further configured to determine the corresponding key for the contacted region.

28. The apparatus of claim 27, wherein the processor is configured to detect the sound emitted by a voice at the source position, and to determine at least a first range value fur focusing an image capturing device on the source position.

29. The apparatus of claim 25, wherein the processor is configured to detect that the sound-detection devices receive the sound caused by the event by detecting an amplitude of a waveform corresponding to the sound.

30. The apparatus of claim 25, wherein the processor is configured to detect that the sound-detection devices receive the sound caused by the event by matching a waveform corresponding to the sound to a waveform of the sound-causing event.

31. The apparatus of claim 25, wherein the sound-detection devices are arranged non-linearly relative to a reference plane.

32. The apparatus of claim 31, wherein the processor is configured to approximate the source position based on a relative position of individual sound-detection devices by:

determining a time value corresponding to each of the sound-detection devices being determined to receive the sound from the source position;

comparing the time value determined by each of the plurality of sound-detection devices to the time values determined by one or more other sound-detection devices in the plurality of sound-detection devices.

33. The apparatus of claim 25, wherein the processor is configured to approximate the source position based on a relative position of individual sound-detection devices by:

identifying an amplitude of the sound at each of the plurality of sound-detecting devices;

comparing the amplitude identified at each of the plurality of sound-detecting devices to the amplitude of the sound identified at one or more other sound-detecting device.

53. The apparatus of claim 25, wherein the event corresponds to the object making contact with a surface, and wherein the apparatus does not include either the object or the surface.

34. A method for detecting a user-input, the method comprising:

monitoring an area on a surface for an occurrence of a sound caused by a user directing an object to make contact with the surface at any location within the area;

displaying an image of a set of keys on the area of the surface;

detecting a sound caused by the user directing the object to make contact with the surface at a location within the area;

independently of knowing a time in which the source is emitted, determining a source position of the sound; and

identifying which key in the set of keys correlates to the source position of the sound.

35. The method of claim 34, wherein identifying which key in the set of keys correlates to the source position includes identifying which key is displayed at or nearest to the source position.

36. The method of claim 34, wherein displaying an image of a set of keys includes displaying a QWERTY keyboard.

37. An apparatus for detecting a user-input, the apparatus comprising:

a projector to display an image corresponding to a set of keys on a surface;

a plurality of sound-detection devices arranged to monitor an area and detect a sound corresponding to a user directing an object to make contact with the surface at any location within the area; and

a processor configured to;

record information about the sound at individual detection points in the plurality of detection points, wherein the information is dependent on a distance between a source position of the source and each of the individual detection points;

independent of knowing a time an which the source of the event is emitted, approximate the source position of the sound using the recorded information and the relative position of the individual detection points in the plurality of detection points; and

determine which key in the set of keys corresponds to the approximated source position.

38. The apparatus of claim 37, wherein the processor is configured to determine which key in the set of keys corresponds to the approximated source position by identifying which key has an image that occupies the source position of the sound on the surface.

39. A computer-readable medium for approximating a source position of a sound-causing event for determining an input used to operate an electronic device, the computer-readable carrying instructions fix performing the steps of:

monitoring aware using a plurality of detection points for an occurrence of the event at any location within the area;

detecting a sound caused by to event occurring at a location corresponding to the source position using the plurality of detection points, the source position being within the area that is being monitored;

independently of knowing a time in which the source of the event is emitted, recording information about the sound at individual detection points in the plurality of detections points, the information being dependent on a distance between the source position and each of the individual detection points;

determining, approximately, the source position based on the recorded information and on a relative position of individual detection points in the plurality of detection points to other detection points in the plurality of detection points;

determining the input from the source position; and

causing an operation to be performed by the electronic device based on the input.

40. The computer-readable medium of claim 39, wherein instructions for detecting a sound include instructions for detecting the sound emitted by a user contacting the area that is being monitored at the location corresponding to the source position, wherein the source position is part of a first designated region and wherein the area is divided into a plurality of designated regions that include the first designated region, each of the plurality of designated regions being associated with a corresponding input, and wherein instructions for determining the input include determining the corresponding input for the first designated region.

41. The computer-readable medium of claim 39, wherein instructions for detecting a sound include instructions for detecting the sound emitted by a user contacting the monitored region at the location corresponding to the source position, wherein the source position is part of a first designated region and wherein the area is divided into a plurality of regions, each of the plurality of designated regions being associated by the electronic device as a corresponding key in a set of keys, and wherein instructions for determining the input include instructions for determining the corresponding key for the first designated region.

42. The computer-readable medium of claim 39, wherein instructions for detecting a sound include instructions for detecting the sound emitted by a voice of a person at the location corresponding to the source position, and wherein instructions for determining the input include instructions for determining at least a first range value for focusing an image capturing device on the source position.

43. The computer-readable medium of claim 42, wherein instructions for determining at least a first range value include instructions for determining range values in at least two dimensions.

44. The computer-readable medium of claim 39, wherein instructions for recording information about the sound at individual detection points include instructions for:

identifying an amplitude of the sound at each of the plurality of detection points; and

wherein instructions for determining the source position include instructions for;

comparing the amplitude identified at each of the plurality of detection points to the amplitude of the sound identified at one or more other detection point.

45. The computer-readable medium of claim 39, wherein instructions for recording information about the sound at individual detection points include instructions for:

determining a time value corresponding to a determination of an arrival of the sound at each of the plurality of detection points; and

wherein instructions for determining the approximate source position include instructions for;

comparing the time value determined at each of the plurality of detection points to the time value determined at one or more other detection points in the plurality of detection points.

46. The computer-readable medium of claim 44, wherein instructions for comparing the amplitude identified at each of the plurality of detection points include instructions for:

determining a first comparison value representing a difference between the amplitude at each of a first pair of detection points in the plurality of detection points; and

using the first value to determine a first range value of the source position from a designated position along an axis defining a first dimension between the source position and the designated position.

47. The computer-readable medium of claim 46, wherein instructions for comparing the amplitude identified at each of the plurality of detection points include instructions for:

determining a second comparison value representing a difference between the amplitude at each of a second pair of detection points in the plurality of detection points; and

using the second comparison value to determine a second range value of the source position from the designated position along an axis defining a second dimension between the source position and the designated position.

48. The computer-readable medium of claim 47, wherein instructions for comparing the amplitude identified at each of the plurality of detection points include instructions for:

determining a third comparison value representing a difference between the amplitude of each of a third pair of detection points in the plurality of detection points; and

using the third comparison value to approximate a third range value of the source position from the designated position along an axis defining a third dimension between the source position and the designated position.

49. The computer-readable medium of claim 39, wherein instructions for recording information about the sound at individual detection points in the plurality of detections points include instructions for:

identifying an amplitude of the sound at each of the plurality of detection points;

determining a time value corresponding to detection of the sound at each of the plurality of detection points; and

wherein instructions for determining the source position include instructions for;

comparing the amplitude identified at each of the plurality of detection points to the amplitude of the sound identified at one or more other detection point to determine at least a first amplitude-determined range value and a second amplitude-determined range value; and

comparing the time value determined at each of the plurality of detection points to the time value determined at one or more other detection points in the plurality of detection points to determine at least a first time-determined range value and a second time-determined range value.

50. The computer-readable medium of claim 49, wherein instructions for determining the source position include instructions for determining a first range value by determining an average of the first amplitude-determined range value and the first time-determined range value, the first range being along an axis defining a first dimension between the source position and the designated position.

51. The computer-readable medium of claim 50, wherein instructions for determining the source position include instructions for weighting the average of the first amplitude-determined range value and the first time-determined range value.

52. The computer-readable medium of claim 51, wherein instructions for determining the source position include instructions for determining a second range value by determining an average of the amplitude-determined second range value with the time-determined second range value, the second range value being along an axis defining a second dimension between the source position and the designated position.