Privacy protection in collective feedforward

• US 9,703,524 B2
• Filed: 01/15/2016
• Issued: 07/11/2017
• Est. Priority Date: 11/25/2015
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Doppler Labs, Inc.
• Inventors: Lanman, III, Richard Fritz, Jaffe, Matthew J., Kraft, Noah, Klimanis, Gints, Baker, Jeff
• Primary Examiner(s): Goins, Davetta W
• Assistant Examiner(s): Ojo, Oyesola C
• Application Number:
  US14/997,320
• Publication Number:
  US 20170147281A1
• Time in Patent Office: 543 Days
• Field of Search: 381 12, 381 98, 381 99, 381100, 381101, 381102, 381103, 381 72, 381 731, 381 74, 381 92, 381 26, 381 61, 381 63, 381 66, 381 714, 381 716, 381 718, 381 719, 381 7111, 381 71, 4554141, 4554142, 4554042, 4554041, 455440, 4554221, 4554561, 4554562, 4554563, 455 45, 700 94
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: H04L 67/30 Profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2460/01 Hearing devices using active noise cancellation
• CPC Class Codes: H04R 2460/07 Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings

Abstract

Personal audio systems and methods are disclosed. A personal audio system includes a processor to generate a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, a circular buffer memory to store a most recent snippet of the ambient audio stream, and an event detector to detect a trigger event. In response to detection of the trigger event, a controller may extract audio feature data from the most recent snippet of the ambient audio stream and transmit the audio feature data and associated metadata to a knowledgebase remote from the personal audio system.

Claims

1. A personal audio system, comprising:

a processor to generate a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, wherein the active processing parameter set indicates a type and degree of one or more processes to be performed on the ambient audio stream;

a buffer memory to store a most recent snippet of the ambient audio stream;

an event detector to detect a trigger event, wherein the trigger event includes retrieving a selected processing parameter set from a dataset memory for use as the active processing parameter set and receiving, via a user interface, a command to modify the active processing parameter set; and

a controller configured to, in response to detection of the trigger event;

extract audio feature data from the most recent snippet of the ambient audio stream; and

transmit the audio feature data and associated metadata to a device remote from the personal audio system, wherein the associated metadata includes an identifier of the selected processing parameter set and the modified processing parameter set associated with the active processing parameter set.

2. The personal audio system of claim 1, wherein

the associated metadata includes the active processing parameter set before modification.

3. The personal audio system of claim 1, wherein

the trigger event includes receiving, via the user interface, a command to store the active processing parameter set in a dataset memory, and

the associated metadata transmitted to the device includes the active processing parameter set.

4. The personal audio system of claim 1, wherein

the associated metadata transmitted to the device includes a descriptor received from a user via a user interface.

5. The personal audio system of claim 1, further comprising:

a geo-locator to determine a location of the personal audio system, wherein

the associated metadata transmitted to the device includes data indicating the location of the personal audio system.

6. The personal audio system of claim 1, wherein

the device is a knowledgebase, and

the controller is further configured to

receive, from the knowledgebase, one or more processing parameter sets derived by the knowledgebase from audio feature data and associated metadata received by the knowledgebase from a plurality of personal audio systems, and

store the one or more processing parameter sets received from the knowledgebase in a parameter memory.

7. The personal audio system of claim 1, wherein

the audio feature data transmitted to the device includes one or more of arithmetic mean, geometric mean, harmonic mean, centroid, variance, standard deviation, spectral skew, kurtosis, spectral shape, rolloff, spread, flatness, spectral flux, Mel frequency cepstral coefficients, pitch, tonal power ratio, harmonic-to-average power ratio, maximum of autocorrelation function, zero crossing rate, RMS power, peak power, crest factor, amplitude/power envelope and attack/decay rates.

8. The personal audio system of claim 1, further comprising:

a housing;

a microphone, a preamplifier, and an analog to digital converter to convert ambient sound into the ambient audio stream; and

a digital to analog converter, an amplifier, and a speaker to convert the personal audio stream into personal output sound, wherein

the microphone, the preamplifier, the analog to digital converter, the processor, the digital to analog converter, the amplifier, and the speaker are disposed within the housing.

9. The personal audio system of claim 8, wherein:

the housing is an earbud housing configured to fit, at least partially, within and be supported by a user'"'"'s ear.

10. A method for processing ambient sound in a personal audio system, comprising:

generating a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, wherein the active processing parameter set indicates a type and degree of one or more processes to be performed on the ambient audio stream;

storing a most recent snippet of the ambient audio stream in a buffer memory;

detecting a trigger event, wherein the trigger event includes retrieving a selected processing parameter set from a dataset memory for use as the active processing parameter set and receiving, via a user interface, a command to modify the active processing parameter set; and

in response to detecting the trigger event;

extracting audio feature data from the most recent snippet of the ambient audio stream; and

transmitting the audio feature data and associated metadata to a device remote from the personal audio system, wherein the associated metadata includes an identifier of the selected processing parameter set and the modified processing parameter set associated with the active processing parameter set.

11. The method of claim 10, wherein

the associated metadata includes the active processing parameter set before modification.

12. The method of claim 10, wherein

the trigger event includes receiving, via a user interface, a command to store the active processing parameter set in a dataset memory, and

the associated metadata transmitted to the device includes the active processing parameter set.

13. The method of claim 10, wherein

the associated metadata transmitted to the device includes a descriptor received from a user via the user interface.

14. The method of claim 10, further comprising:

determining a geo-location of the personal audio system, wherein

the associated metadata transmitted to the device includes data indicating the geo-location of the personal audio system.

15. The method of claim 10, wherein

the device is a knowledgebase, and

the method further comprises:

receiving, from the knowledgebase, one or more processing parameter sets derived by the knowledgebase from snippets and associated metadata from a plurality of personal audio systems, and

storing the one or more processing parameter sets received from the knowledgebase in a parameter memory.

16. The method of claim 10, wherein extracting audio feature data from the most recent snippet of the ambient audio stream further comprises:

performing a preliminary analysis of the most recent snippet;

selecting relevant feature data based on results of the preliminary analysis; and

extracting the relevant feature data from the most recent snippet.

17. The method of claim 10, wherein extracting audio feature data from the most recent snippet of the ambient audio stream further comprises:

requesting, via the user interface, a user to identify a sound captured in the most recent audio snippet;

receiving, via the user interface, a response from the user;

selecting relevant feature data based on the response; and

extracting the relevant feature data from the most recent snippet.

18. A system, comprising:

a knowledgebase; and

a plurality of personal audio systems in communications with the knowledgebase via a network, each personal audio system comprising;

a processor to generate a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, wherein the active processing parameter set indicates a type and degree of one or more processes to be performed on the ambient audio stream;

a circular buffer memory to store a most recent snippet of the ambient audio stream;

an event detector to detect a trigger event, wherein the trigger event includes receiving, via a user interface, a command to modify the active processing parameter set; and

a controller configured to, in response to detection of the trigger event;

extract audio feature data from the most recent snippet of the ambient audio stream; and

transmit the audio feature data and associated metadata to a knowledgebase remote from the personal audio system, wherein the associated metadata includes the modified processing parameter set associated with the active processing parameter set,

wherein the knowledgebase is configured to create new and/or revised processing parameter sets based on the audio feature data and metadata collectively received from the plurality of personal audio systems and to transmit the new and/or revised processing parameter sets to at least some of the plurality of personal audio systems.

19. The system of claim 18, wherein the knowledgebase is configured to push the new and/or revised processing parameter sets to the plurality of personal audio system on an as-available basis.

20. The system of claim 18, wherein the knowledgebase is configured to transmit the new and/or revised processing parameter sets to a particular personal audio system from the plurality of personal audio systems in response to a request from the particular personal audio system.

21. The system of claim 18, wherein the knowledgebase is configured to transmit the new and/or revised processing parameter sets to the at least some of the plurality of personal audio systems at predetermined periodic intervals.

22. A method for processing ambient sound in a personal audio system, comprising:

generating a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set;

storing a most recent snippet of the ambient audio stream in a buffer memory;

detecting a trigger event; and

in response to detecting the trigger event;

determining whether or not the most recent audio snippet contains speech; and

in the event the most recent audio snippet contains speech;

extracting audio feature data from the most recent snippet of the ambient audio stream; and

transmitting the audio feature data and associated metadata to a device remote from the personal audio system; and

in the event the most recent audio snippet does not contain speech, transmitting the most recent audio snippet and associated metadata to the device remote from the personal audio system.