Acoustic ranging based positioning of objects using sound recordings by terminals

• US 10,641,864 B2
• Filed: 02/12/2016
• Issued: 05/05/2020
• Est. Priority Date: 02/12/2016
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Isberg, Peter, Persson, Magnus
• Primary Examiner(s): Sabouri, Mazda
• Application Number:
  US16/073,030
• Publication Number:
  US 20190086508A1
• Time in Patent Office: 1,544 Days
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G01S 5/02585 at least one of the measurements being a non-radio measurement
• CPC Class Codes: G01S 5/22 Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements G01S5/28 takes precedence
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification
• CPC Class Codes: H04R 29/004 for microphones H04R29/007 takes precedence
• CPC Class Codes: H04W 64/006 with additional information processing, e.g. for direction or speed determination

Abstract

An electronic communications terminal records in a memory a sampled audio segment of sound sensed by a micro-phone. A time stamp associated with timing of the recording of the sampled audio segment is determined and stored in the memory associated with the sampled audio segment. The sampled audio segment and the time stamp are transmitted through a network interface toward a location service client of a network node that determines location of the electronic communications terminal based on correlating the sampled audio segment and other sampled audio segments of the sound received from other electronic communications terminals. A related network node and corresponding methods by terminals and network nodes are disclosed.

Claims

1. An electronic communications terminal comprising:

a network interface;

a microphone configured to sense sound;

a processor coupled to the microphone and the network interface; and

a memory coupled to the processor and storing computer readable program code that is executable by the processor to;

record in the memory a sampled audio segment of the sound sensed by the microphone;

determine a time stamp associated with timing of the recording of the sampled audio segment;

store the time stamp in the memory associated with the sampled audio segment; and

transmit the sampled audio segment and the time stamp, through the network interface, toward a location service client of a network node that determines location of a sound source and/or a location of the electronic communications terminal based on correlating the sampled audio segment and other sampled audio segments of the sound received from other electronic communications terminals,

wherein the determination of the time stamp associated with timing of the recording of the sampled audio segment, comprises;

determining time latency between sound being sensed by the microphone and a corresponding portion of the sampled audio segment being recorded in the memory;

obtaining a time value from an internal clock circuit when a defined event occurs associated with the recording of the sampled audio segment; and

generating the time stamp based on removing the time latency from the time value.

2. The electronic communications terminal of claim 1, wherein the processor is further configured by the computer readable program code to:

receive a location identification message from the location service client of the network node, via the network interface, the location identification message containing data identifying a geographic location of the electronic communications terminal; and

provide the data identifying the geographic location to a location service user application being executed by the processor.

3. The electronic communications terminal of claim 1, further comprising a cellular transceiver, wherein the determination of a time stamp associated with timing of the recording of the sampled audio segment, comprises:

repetitively synchronizing an internal clock circuit responsive to repetitive occurrences of a defined signal received through the cellular transceiver from a cellular radio base station; and

determining the time stamp based on a clock signal from the internal clock circuit when a defined event occurs associated with the recording of the sampled audio segment.

4. The electronic communications terminal of claim 1, further comprising a satellite signal receiver, wherein the determination of a time stamp associated with timing of the recording of the sampled audio segment, comprises:

repetitively synchronizing an internal clock circuit responsive to repetitive occurrences of a defined signal received through the satellite signal receiver from at least three satellites; and

determining the time stamp based on a clock signal from the internal clock circuit when a defined event occurs associated with the recording of the sampled audio segment.

5. The electronic communications terminal of claim 1, wherein:

the determination of a time stamp associated with timing of the recording of the sampled audio segment, comprises generating an indication of a level of accuracy of the time stamp; and

the transmission of the sampled audio segment and the time stamp toward a location service client, further comprises transmitting the indication of the level of accuracy of the time stamp through the network interface toward the location service client.

6. The electronic communications terminal of claim 5, further comprising a radio frequency transceiver, wherein:

generating an indication of a level of accuracy of the time stamp, comprises determining an elapsed time since a last successful synchronization of an internal clock circuit was completed based on timing of a defined signal received through the radio frequency transceiver.

7. An electronic communications terminal comprising:

a network interface;

a microphone configured to sense sound;

a processor coupled to the microphone and the network interface; and

a memory coupled to the processor and storing computer readable program code that is executable by the processor to perform operations comprising;

recording in the memory a sampled audio segment of the sound sensed by the microphone;

determining a time stamp associated with timing of the recording of the sampled audio segment;

storing the time stamp in the memory associated with the sampled audio segment;

transmitting the sampled audio segment and the time stamp, through the network interface, toward a location service client of a network node that determines location of a sound source and/or a location of the electronic communications terminal based on correlating the sampled audio segment and other sampled audio segments of the sound received from other electronic communications terminals;

processing the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment; and

transmitting the privacy protected audio segment and the time stamp through the network interface toward the location service client of the network node that determines location of the sound source and/or location of the electronic communications terminal based on correlating the privacy protected audio segment and other privacy protected audio segments of the sound received from other electronic communications terminals.

8. The electronic communications terminal of claim 7, wherein the processing of the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment, comprises:

determining indications of elapsed time between locations of waveform peaks in the sampled audio segment; and

generating the privacy protected audio segment to contain a listing of the indications of elapsed time.

9. The electronic communications terminal of claim 7, wherein the processing of the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment, further comprises:

measuring amplitude values of the waveform peaks in the sampled audio segment; and

generating the privacy protected audio segment to further contain the listing of the amplitude values of the waveform peaks.

10. The electronic communications terminal of claim 7, wherein the processing of the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment, comprises:

performing envelope detection on the sampled audio segment to generate the privacy protected audio segment.

11. The electronic communications terminal of claim 10, wherein the performing envelope detection on the sampled audio segment to generate the privacy protected audio segment, comprises:

constraining the envelope detection to being performed within one or more defined frequency bands outside a range of human voice.

12. The electronic communications terminal of claim 10, wherein the performing envelope detection on the sampled audio segment to generate the privacy protected audio segment, comprises:

performing a Hilbert transform on the sampled audio segment to generate the transformed sampled audio segment; and

down-sampling the transformed sampled audio segment to blur any human voice component thereof.

13. The electronic communications terminal of claim 7, wherein the processing of the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment, comprises:

filtering the sampled audio segment to remove a frequency range component that is within a major portion of a range of human voice.

14. The electronic communications terminal of claim 7, further comprising:

a plurality of the microphones,

wherein the processor is further configured by the computer readable program code to identify a near-field component of the sound and a far-field component of the sound based on correlating signals received by the plurality of the microphones, the near-field component being identified based on greater amplitude variations between the correlated signals compared to the fair-field component, and

wherein the processing of the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment, comprises;

removing the near-field component of the sound from the sampled audio segment of the sound recorded in the memory while leaving the far-field component of the sound in the sampled audio segment of the sound recorded in the memory.

15. The electronic communications terminal of claim 7, further comprising:

a sound activity detector circuit configured to identify start and stop sound transitions in the sampled audio segment of the sound recorded in the memory,

wherein the processing of the sampled audio segment to render unintelligible any human voice component thereof to generate a privacy protected audio segment, comprises;

measuring timing of the start and stop transitions identified by the sound activity detector circuit in the sampled audio segment of the sound recorded in the memory; and

generating the privacy protected audio segment to contain the timing of the start and stop transitions.

16. A network node providing a location service to electronic communications terminals, the network node comprising:

a network interface configured to communicate with a plurality of electronic communications terminals;

a processor coupled to the network interface; and

a memory coupled to the processor and storing computer readable program code that is executable by the processor to;

receive messages from the electronic communications terminals through the network interface, each of the messages containing a sampled audio segment and a time stamp, wherein the time stamp indicates timing associated with when the sampled audio segment was recorded by the electronic communications terminal;

identify a similar pattern in the sampled audio segments contained in one of the messages received from each of the electronic communications terminals;

identify timing offsets between the similar pattern identified in the sampled audio segments using the time stamps contained in the one of the messages received from each of the electronic communications terminals;

determine location of a sound source of the sound and/or a location of the electronic communications terminals based on the timing offsets; and

communicate the location to the at least one of the electronic communications terminals and/or another network node through the network interface;

wherein the processor is further configured by the computer readable program code to obtain an indication of a level of accuracy of the time stamp contained in the one of the messages received from each of the electronic communications terminals; and

wherein the identification of the similar pattern in the privacy protected audio segments contained in one of the messages received from each of the electronic communications terminals, comprises;

cross-correlating a pattern in the privacy protected audio segment contained in the one of the messages received from each of the electronic communications terminals using the time stamps; and

adjusting size of a window used for the cross-correlation of the one of the messages received from the electronic communications terminals based on the indications of the level of accuracy of the time stamps used for the cross-correlation.

17. The network node of claim 16, wherein the determination of the location of the sound source of the sound and/or the location of the electronic communications terminals based on the timing offsets, comprises:

determining acoustic range between the one of the electronic communications terminals and the other ones of the electronic communications terminals based on the timing offsets determined between the similar pattern identified in the sampled audio segments.

18. The network node of claim 17, wherein the determination of the location of the sound source of the sound and/or the location of the electronic communications terminals based on the timing offsets, further comprises:

determining a geographic location of a GNSS assisted one of the electronic communications terminals based on GNSS location reported by the GNSS assisted one of the electronic communications terminals to the network node; and

determining the location of the one of the electronic communications terminals based on a combination of the geographic location of the GNSS assisted one of the electronic communications terminals and the acoustic range between the one of the electronic communications terminals and the GNSS assisted one of the electronic communications terminals.