Gaze based directional microphone

• US 9,596,391 B2
• Filed: 08/05/2014
• Issued: 03/14/2017
• Est. Priority Date: 09/03/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Henderek, David Figgins, Skog, Mrten
• Primary Examiner(s): King, Simon
• Application Number:
  US14/452,178
• Publication Number:
  US 20150063603A1
• Time in Patent Office: 952 Days
• Field of Search: 381/122
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04R 1/32 for obtaining desired directional characteristic only specially adapted for hearing aids H04R25/40
• CPC Class Codes: H04R 2201/40 Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups

Abstract

According to the invention, a system for converting sound to electrical signals is disclosed. The system may include a gaze tracking device and a microphone. The gaze tracking device may determine a gaze direction of a user. The microphone may be more sensitive in a selected direction than at least one other direction and alter the selected direction based at least in part on the gaze direction determined by the gaze tracking device.

Claims

1. A system for converting sounds to electrical signals, wherein the system comprises:

a gaze tracking device, wherein the gaze tracking device determines a gaze direction of a user;

a microphone;

a camera;

a display device reproducing a scene captured by the camera, wherein;

the microphone is more sensitive in a selected direction than at least one other direction;

the microphone alters the selected direction based at least in part on the gaze direction determined by the gaze tracking device;

the gaze tracking device determining the gaze direction of the user comprises determining a gaze point of the user on the display device; and

the microphone altering the selected direction based at least in part on the gaze direction determined by the gaze tracking device comprises the microphone altering the selected direction towards a direction in the scene corresponding to the gaze point of the user on the reproduced scene.

2. The system for converting sounds to electrical signals of claim 1, wherein the system further comprises:

an output device.

3. The system for converting sounds to electrical signals of claim 2, wherein the output device comprises a selection from a group consisting of:

a speaker;

a cochlear implant; and

a recording device.

4. The system for converting sounds to electrical signals of claim 1, wherein the microphone altering the selected direction based at least in part on the gaze direction comprises:

the gaze tracking device determining a point in space representative of the gaze direction; and

the microphone altering the selected direction such that the selected direction intersects the point in space.

5. The system for converting sounds to electrical signals of claim 1, wherein the microphone altering the selected direction based at least in part on the gaze direction comprises:

the microphone altering the selected direction to be parallel with the gaze direction.

6. The system for converting sounds to electrical signals of claim 1, wherein the system further comprises:

a frame configured to be worn on the user'"'"'s head, wherein the eye tracking device is coupled with the frame.

7. The system for converting sounds to electrical signals of claim 6, wherein:

the microphone is coupled with the frame.

8. The system for converting sounds to electrical signals of claim 1, wherein the microphone altering the selected direction comprises:

physically reorienting the microphone.

9. The system for converting sounds to electrical signals of claim 1, wherein:

the microphone comprises a plurality of unidirectional microphones; and

the microphone altering the selected direction comprises;

processing outputs from the plurality of unidirectional microphones.

10. The system for converting sounds to electrical signals of claim 9, wherein processing outputs from the plurality of unidirectional microphones comprises:

adding delay to a signal received from a first unidirectional microphone to create a delayed signal

summing a signal received from a second unidirectional microphone with the delayed signal, wherein the second unidirectional microphone is relatively less aligned with the selected direction than the first unidirectional microphone.

11. The system for converting sounds to electrical signals of claim 1, wherein the system further comprises:

a user input device configured to allow the user to selectively enable and disable altering of the selected direction.

12. The system for converting sounds to electrical signals of claim 1, wherein the system further comprises:

a user input device configured to allow the user to adjust the sensitivity of the microphone in the selected direction compared to other directions.

13. A method for converting sounds to electrical signals, wherein the method comprises:

reproducing a scene captured by a camera on a display device;

determining a gaze direction of a user with a gaze tracking device, wherein determining the gaze direction comprises determining a gaze point of the user on the display device; and

altering, with a processor, a direction in which a microphone is directed based at least in part on the gaze direction, wherein altering the direction in which the microphone is directed based at least in part on the gaze direction comprises altering the direction towards a direction in the scene corresponding to the gaze point of the user on the reproduced scene.

14. The method for converting sounds to electrical signals of claim 13, wherein altering the direction in which the microphone is directed comprises:

physically reorienting the microphone.

15. The method for converting sounds to electrical signals of claim 13, wherein:

the microphone comprises a plurality of unidirectional microphones; and

altering the direction in which the microphone is directed comprises;

processing outputs from the plurality of unidirectional microphones.

16. The method for converting sounds to electrical signals of claim 15, wherein processing outputs from the plurality of unidirectional microphones comprises:

adding delay to a signal received from a first unidirectional microphone to create a delayed signal

summing a signal received from a second unidirectional microphone with the delayed signal, wherein the second unidirectional microphone is relatively less aligned with the selected direction than the first unidirectional microphone.

17. A non-transitory machine readable medium having instructions stored thereon, wherein instructions are executable by a processor to at least:

reproduce a scene captured by a camera on a display device;

receive, from a gaze tracking device, data representing a gaze direction of a user;

determining the gaze direction of the user from the data, wherein determining the gaze direction comprises determining a gaze point of the user on the display device; and

cause a direction in which a microphone is most sensitive to be altered based at least in part on the data representing the gaze direction, wherein causing the direction in which the microphone is most sensitive to be altered based at least in part on the data comprises altering the direction in which the microphone is most sensitive towards a direction in the scene corresponding to the gaze point of the user on the reproduced scene.

18. The non-transitory machine readable medium of claim 17, wherein causing the direction in which the directional microphone is most sensitive to be altered comprises:

causing the microphone to be physically reoriented.

19. The non-transitory machine readable medium of claim 17, wherein:

the microphone comprises a plurality of unidirectional microphones; and

causing the direction in which the microphone is most sensitive to be altered comprises;

processing outputs from the plurality of unidirectional microphones.

20. The non-transitory machine readable medium of claim 19, wherein processing outputs from the plurality of unidirectional microphones comprises:

adding delay to a signal received from a first unidirectional microphone to create a delayed signal

summing a signal received from a second unidirectional microphone with the delayed signal, wherein the second unidirectional microphone is relatively less aligned with the selected direction than the first unidirectional microphone.

21. A system for converting sounds to electrical signals, wherein the system comprises:

a gaze tracking device, wherein the gaze tracking device determines a gaze direction of a user; and

a microphone, wherein;

the microphone is more sensitive in a selected direction than at least one other direction;

the microphone alters the selected direction based at least in part on the gaze direction determined by the gaze tracking device;

the microphone comprises a plurality of unidirectional microphones;

the microphone altering the selected direction comprises processing outputs from the plurality of unidirectional microphones; and

processing outputs from the plurality of unidirectional microphones comprises;

adding delay to a signal received from a first unidirectional microphone to create a delayed signal; and

summing a signal received from a second unidirectional microphone with the delayed signal, wherein the second unidirectional microphone is relatively less aligned with the selected direction than the first unidirectional microphone.

22. A method for converting sounds to electrical signals, wherein the method comprises:

determining a gaze direction of a user; and

altering a direction in which a microphone is directed based at least in part on the gaze direction, wherein;

the microphone comprises a plurality of unidirectional microphones;

altering the direction in which the microphone is directed comprises processing outputs from the plurality of unidirectional microphones; and

processing outputs from the plurality of unidirectional microphones comprises;

adding delay to a signal received from a first unidirectional microphone to create a delayed signal; and

summing a signal received from a second unidirectional microphone with the delayed signal, wherein the second unidirectional microphone is relatively less aligned with the selected direction than the first unidirectional microphone.

23. A non-transitory machine readable medium having instructions stored thereon, wherein instructions are executable by a processor to at least:

receive, from a gaze tracking device, data representing a gaze direction of a user; and

cause a direction in which a microphone is most sensitive to be altered based at least in part on the data representing the gaze direction, wherein;

the microphone comprises a plurality of unidirectional microphones;

causing the direction in which the microphone is most sensitive to be altered comprises processing outputs from the plurality of unidirectional microphones; and

processing outputs from the plurality of unidirectional microphones comprises;

adding delay to a signal received from a first unidirectional microphone to create a delayed signal; and

summing a signal received from a second unidirectional microphone with the delayed signal, wherein the second unidirectional microphone is relatively less aligned with the selected direction than the first unidirectional microphone.