Personalized, real-time audio processing

• US 9,905,244 B2
• Filed: 02/02/2016
• Issued: 02/27/2018
• Est. Priority Date: 02/02/2016
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Glasgow, Dane, Ramadge, David, MacLaurin, Matthew Bret, Van Winkle, Justin, Sherman, Corinne Elizabeth
• Primary Examiner(s): ISLAM, MOHAMMAD K
• Application Number:
  US15/013,537
• Publication Number:
  US 20170221500A1
• Time in Patent Office: 756 Days
• Field of Search: 381 1408, 381313, 381 1416, 348169
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/022 Viewing apparatus G02B27/04, G02B27/06, G02B27/08 take precedence
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/403 Linear arrays of transducers
• CPC Class Codes: H04R 29/005 Microphone arrays
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

An apparatus and method for real-time audio processing employs a gaze detection sensor to detect a direction of a user'"'"'s gaze and output a gaze signal corresponding to the detected direction of the user'"'"'s gaze. A digital signal processing unit responds to a plurality of signals corresponding to a plurality of sounds received at the apparatus, and the determined direction of gaze to identify a signal of interest from the plurality of signals using the gaze signal. The signal of interest is processed for output to the user. In embodiments, a microphone array provides the plurality of signals. An imaging sensor may work with either the microphone array or the gaze detection sensor to identify the signal of interest.

Claims

1. An apparatus comprising:

a gaze detection sensor configured to detect a direction of a gaze of a user and output a gaze signal corresponding to the detected direction of the gaze of the user;

a digital signal processing unit configured to;

responsive to identifying a plurality of signals corresponding to a plurality of sounds received at the apparatus and the detected direction of the gaze;

generate a list identifying at least two of the plurality of signals using the gaze signal;

cause the list to be presented via a graphical interface;

receive, via the graphical interface, a selection of a signal of interest from among the at least two of the plurality of signals; and

process the signal of interest for output.

2. The apparatus of claim 1, further comprising a microphone array configured to receive the plurality of sounds and output the plurality of signals, the microphone array being further configured relative to a positioning of the gaze detection sensor to provide information indicating directionality of the plurality of signals.

3. The apparatus of claim 2, further comprising:

an imaging sensor configured to detect images and output an image signal corresponding to the detected image to the digital signal processing unit;

wherein the digital signal processing unit is further configured to;

determine whether the image signal identifies a person speaking in the direction of the user; and

identify the at least two of the plurality of signals using the image signal.

4. The apparatus of claim 3, wherein the gaze detection sensor works with the digital signal processing unit to identify the at least two of the plurality of signals.

5. The apparatus of claim 3, wherein the digital signal processing unit uses data from the gaze detection sensor and the microphone array to identify the at least two of the plurality of signals.

6. The apparatus of claim 3, wherein the digital signal processing unit uses data from the imaging sensor and the microphone array to identify the at least two of the plurality of signals.

7. The apparatus of claim 1, further comprising an override switch to override the gaze detection sensor.

8. The apparatus of claim 7, wherein, in response to a signal from the override switch, the digital signal processing unit ignores input from the gaze detection sensor for a finite period of time.

9. The apparatus of claim 1, wherein the digital signal processing unit includes a device to output a communication signal to indicate an indication of the user to communicate.

10. The apparatus of claim 1, wherein the digital signal processing unit includes a device configured to receive an identifying signal from a sound source, the identifying signal including metadata that identifies either the sound source itself, or a person associated with the sound source.

11. A method comprising:

detecting a direction of a gaze of a user;

outputting a gaze signal corresponding to the detected direction of the gaze of the user;

responsive to identifying a plurality of signals corresponding to a plurality of sounds received at a processor, and the detected direction of the gaze;

generating a list identifying at least two of the plurality of signals using the gaze signal;

causing the list to be presented via a graphical interface;

receiving a selection of a signal of interest from among the at least two of the plurality of signals; and

processing the signal of interest for output.

12. The method of claim 11, further comprising using a relative positioning of a microphone array which outputs the plurality of signals, and a gaze detection sensor which outputs the gaze signal, to provide information indicating directionality of the plurality of signals.

13. The method of claim 12, further comprising:

detecting an image of a person speaking in the direction of the user;

generating an image signal corresponding to the detected image; and

identifying the at least two of the plurality of signals using the image signal.

14. The method of claim 13, wherein the identifying further comprises using the gaze signal and the image signal to identify the at least two of the plurality of signals.

15. The method of claim 13, wherein the identifying further comprises using the gaze signal and the relative positioning of the microphone array and the gaze signal to identify the at least two of the plurality of signals.

16. The method of claim 13, wherein the identifying further comprises using the image signal and the relative positioning of the microphone array and the gaze signal to identify the at least two of the plurality of signals.

17. The method of claim 11, further comprising selectively overriding the gaze signal.

18. The method of claim 17, further comprising selectively overriding the gaze signal for a finite period of time.

19. The method of claim 11, further comprising outputting a communication signal indicating a user'"'"'s indication to communicate.

20. The method of claim 11, further comprising receiving an identifying signal from a sound source, the identifying signal including metadata that identifies either the sound source itself or a person associated with the sound source.