Accessibility for the hearing impaired using measurement and object based audio

• US 9,858,943 B1
• Filed: 05/09/2017
• Issued: 01/02/2018
• Est. Priority Date: 05/09/2017
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Carlsson, Gregory, Richman, Steven, Milne, James R.
• Primary Examiner(s): Islam, Mohammad
• Application Number:
  US15/590,795
• Time in Patent Office: 238 Days
• CPC Class Codes: A61B 5/125 objective methods
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: H04M 1/72478 for hearing-impaired users
• CPC Class Codes: H04R 2205/041 Adaptation of stereophonic signal reproduction for the hearing impaired
• CPC Class Codes: H04R 2430/01 Aspects of volume control, not necessarily automatic, in sound systems
• CPC Class Codes: H04R 2460/07 Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
• CPC Class Codes: H04R 25/356 Amplitude, e.g. amplitude shift or compression
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 1/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 3/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30

Abstract

A setup application in a TV, sound bar, surround sound speaker system, or AVR creates a hearing profile for a user, calibrating various frequencies according to an interactive calibration step. Then, object-based audio, which may include metadata describing the key aspects of the audio, is used to identify the key aspects so that they can be emphasized, using the calibration information, to allow for increased intelligibility and the most natural listening experience possible for the hearing-impaired. EQ calibration for each speaker also may be effected for each user.

Claims

1. A device comprising:

at least one computer memory that is not a transitory signal and that comprises instructions executable by at least one processor to;

receive from a user interface (UI) input pertaining to at least first and second speaker frequencies;

based at least in part on the input, establish a calibration for the first and second frequencies;

identify at least a first audio element in audio comprising at least first and second audio elements;

modify the first audio element according to the calibration of the first and second frequencies to render a calibrated audio element and not modify the second audio element according to the calibration of the first and second frequencies; and

play the calibrated audio element and the second audio element on at least one audio speaker.

2. The device of claim 1, comprising the at least one processor.

3. The device of claim 1, comprising the at least one audio speaker.

4. The device of claim 1, wherein the instructions are executable to:

modify the first audio element according to the calibration of the first and second frequencies at least in part based on metadata in the audio indicating that the first audio element is an instance of a first type of audio object; and

not modify the second audio element according to the calibration of the first and second frequencies at least in part based on metadata in the audio indicating that the second audio element is an instance of a second type of audio object.

5. The device of claim 1, wherein the instructions are executable to:

modify the first audio element according to the calibration of the first and second frequencies at least in part based on object based audio information indicating that the first audio element is an instance of a first type of audio object; and

not modify the second audio element according to the calibration of the first and second frequencies at least in part based on object based audio information indicating that the second audio element is an instance of a second type of audio object.

6. The device of claim 1, wherein the first audio element includes speech and the second audio element does not include speech.

7. The device of claim 1, wherein the instructions are executable to establish the calibration at least in part by:

allowing a user to adjust respective amplitudes of first and second tones played on the speaker at the respective first and second frequencies to achieve what the user perceives as equality in at least one respect between the first and second tones.

8. The device of claim 7, wherein the instructions are executable to modify the first audio element to render the calibrated audio element at least in part by adjusting first and second components having the respective first and second frequencies in the first audio element according to user adjustments of the respective first and second tones.

9. The device of claim 1, wherein the instructions are executable to:

modify the first audio element according to the calibration of the first and second frequencies at least in part based on metadata in the audio indicating that the first audio element is a key sound; and

not modify the second audio element according to the calibration of the first and second frequencies at least in part based on metadata in the audio indicating that the second audio element is not a key sound.

10. The device of claim 1, comprising:

at least first circuitry configured to modify the first audio element according to the calibration of the first and second frequencies to render the calibrated audio element (CAE); and

at least second circuitry configured to receive the CAE and modify at least one EQ parameter in the CAE for at least first and second audio channels according to at least one user profile.

11. A method, comprising:

receiving a person'"'"'s frequency response profile indicating respective first and second perceptions of the person to at least respective first and second frequencies;

receiving audio comprising first audio objects and second audio objects;

modifying an amplitude of at least one frequency in the first audio objects according to the person'"'"'s frequency response profile to render modified first audio objects;

not modifying an amplitude at least one frequency in the second audio objects according to the person'"'"'s frequency response profile; and

playing the modified first audio objects and the second audio objects on at least one audio speaker.

12. The method of claim 11, wherein at least one frequency in the first audio objects is modified based at least in part on identification information in metadata accompanying the audio.

13. The method of claim 12, wherein the metadata indicates that the first audio objects are key sounds.

14. The method of claim 12, wherein the metadata indicates that the first audio objects are of a type of object matching a key sound type.

15. An audio video assembly (AVA), comprising:

at least one display for presenting video under control of at least one processor;

at least one speaker for presenting audio under control of at least one processor; and

at least one storage comprising instructions executable by at least one processor to;

receive audio having at least first and second components with respective first and second frequencies;

change at least a first component in a first object in the audio according to first frequency information in a person'"'"'s frequency response profile;

change at least a second component in the first object in the audio according to second frequency information in the person'"'"'s frequency response profile; and

play at least the first audio object on the at least one speaker after changing the first and second components.

16. The AVA of claim 15, wherein the first and second components include volumes.

17. The AVA of claim 15, wherein the instructions are executable to:

not change any component in a second object in the audio according to the person'"'"'s frequency response profile.

18. The AVA of claim 15, wherein the components in the first object are changed based at least in art on metadata in the audio.

19. The AVA of claim 18, wherein the metadata indicates that the first audio object is a key sound.

20. The AVA of claim 18, wherein the metadata indicates that the first audio object is of a type of object matching a key sound type.