System and method for dynamic control of audio playback based on the position of a listener

• US 10,264,385 B2
• Filed: 05/26/2016
• Issued: 04/16/2019
• Est. Priority Date: 12/05/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Freeman, Daniel B.
• Primary Examiner(s): Kuntz, Curtis A
• Assistant Examiner(s): Zhu, Qin
• Application Number:
  US15/166,080
• Publication Number:
  US 20160353223A1
• Time in Patent Office: 1,055 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

An optimal listening position for the multi-channel audio sound system is determined. A listening area and other audio playback controls may also be defined or modified by a listener. During subsequent audio playback, an imager captures images of at least a portion of the listener. When the listener has moved to a position that is different from a previous position, one or more audio parameters are adjusted for at least one channel in the multi-channel audio system in order to reposition the initial or previous optimal listening position to the current position of the listener. When the listener moves outside the listening area, an out of bounds response is initiated to disable or control the audio playback until the listener moves to a position within the listening area.

Claims

1. A system with a multi-channel audio sound system that dynamically adjusts audio playback, the system comprising:

an input device operable to receive an entry of initial set up data, wherein the initial set up data includes a definition of an out of bounds response and a listening area within a room;

a sensor operable to track an orientation of a portion of interest of a listener and a location of the listener;

a processing module, operable to

receive data corresponding to the orientation of the portion of interest of the listener and the location of the listener,

determine whether the location of the listener is within the listening area, and

generate, in response to the location of the listener being within the listening area, an adjustment signal based on the orientation of the portion of interest of the listener; and

an audio module operable to;

receive the adjustment signal and to produce an audio signal for each channel in the multi-channel audio sound system and to control audio playback of the audio signals by the channels in the multi-channel audio sound system, and

initiate, in response to the location of the listener being outside the listening area, the out of bounds response.

2. The system of claim 1, further comprising a plurality of speakers each operable to receive a respective audio signal.

3. The system of claim 1, further comprising an audio input device operable to input a plurality of audio signals into the processing module.

4. The system of claim 1, wherein the sensor comprises a camera connected to the processing module.

15. The system of claim 1, wherein the portion of interest of the listener includes one or more of a head, an eye, or a face of the listener.

16. The system of claim 1, wherein the sensor is operable to track movement of the location of the listener, and wherein the processing module is operable to generate the adjustment signal based on the orientation of the portion of interest of the listener and the location of the listener.

5. A system with a multi-channel audio sound system that dynamically adjusts audio playback, the system comprising:

an input device operable to receive an entry of initial set up data, wherein the initial set up data includes a definition of an out of bounds response and a listening area within a room;

a sensor operable to track an orientation of a portion of interest of a listener and a location of the listener;

a sensor processor operable to

receive data corresponding to the orientation of the portion of interest of the listener and the location of the listener,

determine whether the location of the listener is within the listening area, and

store the determination in a memory;

a controller operable to

read the determination out of the memory, and

generate, in response to the location of the listener being within the listening area, an adjustment signal based on the orientation of the portion of interest of the listener; and

an audio processor operable to;

receive the adjustment signal and to produce an audio signal for each channel in the multi-channel audio sound system and to control audio playback of the audio signal by the channels in the multi-channel audio sound system, and

initiate, in response to the location of the listener being outside the listening area, the out of bounds response.

6. The system of claim 5, further comprising a plurality of speakers, each operable to receive a respective audio signal.

7. The system of claim 5, further comprising an audio input device, operable to input a plurality of audio signals into the audio processor.

8. The system of claim 5, wherein the sensor comprises a camera connected to the sensor processor.

17. The system of claim 5, wherein the portion of interest of the listener includes one or more of a head, an eye, or a face of the listener.

18. The system of claim 5, wherein the sensor is operable to track movement of the location of the listener, and wherein the controller is operable to generate the adjustment signal based on the orientation of the portion of interest of the listener and the location of the listener.

9. A method for dynamic control of audio playback in a system having a multi-channel audio sound system, the method comprising:

receiving initial set up data, wherein the initial set up data includes a definition of an out of bounds response and a listening area within a room;

capturing data corresponding to an orientation of a portion of interest of a listener and a location of the listener;

determining whether the listener is within the listening area; and

when the location of the listener is within the listening area;

generating an adjustment signal based on the data corresponding to the orientation of the portion of interest of the listener,

producing an audio signal for each channel in the multi-channel audio sound system, and

controlling audio playback of the audio signal by the channels in the multi-channel audio sound system based on the adjustment signal, and

when the location of the listener is outside the listening area, initiating the out of bounds response.

10. The method of claim 9, further comprising performing an initial set up prior to capturing the data corresponding to the orientation of the portion of interest of the listener.

11. The method of claim 10, wherein performing the initial set up prior to capturing the data corresponding to the orientation of the portion of interest of the listener comprises:

determining an initial listening position;

determining the listening area;

setting the out of bounds response that determines a first control response when the listener is outside the listening area; and

setting one or more playback controls that limit an adjustment for at least one audio parameter.

12. The method of claim 11, further comprising:

determining whether the listener has moved out of the listening area; and

when the listener has moved out of the listening area disabling dynamic control of audio playback according to the first control response.

13. The method of claim 12, further comprising

determining whether the listener has moved to a new position within the listening area subsequent to the listener being outside out of the listening area;

and

when the listener has moved to the new position, adjusting one or more audio parameters for one or more channels in the multi-channel audio sound system according to a second control response, wherein the second control response enables dynamic control of audio playback such that an optimal listening position is repositioned to the new position.

14. The method of claim 9, further comprising determining whether dynamic control of audio playback is enabled.

19. The method of claim 9, wherein the portion of interest of the listener includes one or more of a head, an eye, or a face of the listener.

20. The method of claim 9, wherein the data further corresponds to the location of the listener, and wherein the adjustment signal is based on the orientation of the portion of interest of the listener and the location of the listener.