System And Method Of Audio Processing

• US 20100323793A1
• Filed: 01/23/2009
• Published: 12/23/2010
• Est. Priority Date: 02/18/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Andall, Hogarth
• Granted Patent: US 8,932,134 B2
• Time in Patent Office: Days
• US Class Current: 463/35
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 2300/6081 generating an output signal, e.g. under timing constraints, for spatialization
• CPC Class Codes: A63F 2300/8088 involving concurrently several players in a non-networked game, e.g. on the same game console
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

A method of audio processing for an entertainment device operable to communicate with a game controller, the method comprising the steps of generating a source sound for reproduction by a plurality of loudspeakers at a current position where acoustic signals from the plurality of loudspeakers are coincident (hereafter referred to as a ‘sweet spot’), requesting that the user steers the sound using the game controller until the user considers the sound to be centred upon them, and interpreting user inputs from the game controller to adjust the output timing of the source sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the user directional inputs so as to move the sweet spot.

Claims

8. (canceled)

15. (canceled)

22. A computer-readable storage medium having instructions stored thereon, the instructions, when executed by a processor, cause the processor to perform method of audio processing for an entertainment device operable to communicate with a game controller, the method comprising the steps of:

generating a source sound for reproduction by a plurality of loudspeakers at a current position where acoustic signals from the plurality of loudspeakers are coincident at a sweet spot;

requesting that a user steers the sound using the game controller until the user considers the sound to be centered upon them; and

interpreting user inputs from the game controller to adjust output timing of the source sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the user inputs from the game controller so as to move the sweet spot.

23. A computer-readable storage medium according to claim 22, the method further comprising:

estimating the position of the sweet spot;

estimating the position of the user;

estimating the position of the user relative to the estimated position of the sweet spot; and

adjusting the output timing of sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the estimated position of the user relative to the estimated position of the sweet spot so as to move the estimated position of the sweet spot towards the estimated position of the user.

24. A computer-readable storage medium according to claim 23, in which the step of estimating the position of a sweet spot comprises:

generating test signals for a plurality of loudspeakers;

measuring acoustic signals from the loudspeakers with a microphone located substantially at the position of a non-acoustic sensor used in the estimation of the user'"'"'s position and itself placed in a position other than the user'"'"'s listening position; and

adjusting the output timing of the sound data signals for reproduction by respective loudspeakers so that simultaneous source sound data signals are then timed to cause corresponding acoustic signals from the loudspeakers to reach the microphone substantially simultaneously.

25. A method of audio processing for an entertainment device operable to communicate with a game controller, the method comprising the steps of:

generating a source sound for reproduction by a plurality of loudspeakers at a current position where acoustic signals from the plurality of loudspeakers are coincident at a sweet spot;

requesting that a user steers the sound using the game controller until the user considers the sound to be centered upon them; and

interpreting user inputs from the game controller to adjust output timing of source sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the user inputs from the game controller so as to move the sweet spot.

1. A method of audio processing according to claim 25, comprising the steps of:

estimating the position of the sweet spot;

estimating the position of the user;

estimating the position of the user relative to the estimated position of the sweet spot; and

adjusting the output timing of sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the estimated position of the user relative to the estimated position of the sweet spot so as to move the estimated position of the sweet spot towards the estimated position of the user.

2. A method according to claim 1, in which the step of estimating the position of the user comprises capturing and analysing data from a non-acoustic sensor.

3. A method according to claim 1, in which the step of estimating a position of a user comprises capturing an image encompassing some or all of the user, and analysing the image to identify the position of the user.

4. A method according to claim 3, comprising the step of augmenting the analysis of the captured image by the use of distance data corresponding to the captured image data and obtained from an image capture device further equipped with a distance measuring arrangement.

5. A method according to claim 1, in which the step of estimating the position of the user comprises receiving a radio signal from a radio transmitting device carried by the user by two or more phased array antennas, and analysing the respective angles of reception at the phased arrays to detect the position of transmission of the radio signal.

6. A method according to claim 1, in which the step of estimating the position of the user relative to the estimated position of the sweet spot comprises comparing the estimated position of the user with the estimated position of the sweet spot to determine the relative difference in position.

7. A method according to claim 1, in which the step of estimating the position of a sweet spot comprises:

generating test signals for a plurality of loudspeakers;

measuring acoustic signals from the loudspeakers with a microphone located substantially at the position of a non-acoustic sensor used in the estimation of the user'"'"'s position and itself placed in a position other than the user'"'"'s listening position; and

adjusting the output timing of the sound data signals for reproduction by respective loudspeakers so that simultaneous source sound data signals are then timed to cause corresponding acoustic signals from the loudspeakers to reach the microphone substantially simultaneously.

9. A method according to claim 1, in which the step of estimating the position of the sweet spot comprises:

capturing an image of a room housing the loudspeakers;

estimating the centre point of the room by analysis of the captured image; and

estimating the position of the sweet spot as the centre point of the room.

10. A method according to claim 1, in which the step of estimating the position of the sweet spot comprises:

capturing an image of a room housing the loudspeakers;

analysing the captured image to identify a seat, or if there are two or more seats, the seat closest to the lateral centre of the captured image; and

estimating the position of the identified seat by analysis of the captured image.

11. A method according to claim 9, comprising the step of augmenting the analysis of the captured image by the use of distance data corresponding to the captured image data and obtained from an image capture device further equipped with a distance measuring arrangement.

12. A method according to claim 1, the method being carried out by an entertainment device operable to interact with two or more users.

13. A method according to claim 12, where the two or more users interact with the entertainment device in turn, and in which the step of adjusting output timings comprises moving the effective position of the sweet spot to the position of the currently interacting user.

14. A method according to claim 12, where the two or more users interact with the entertainment device simultaneously, and in which the step of adjusting output timings comprises:

adjusting the output timing of some or all of respective sound data signals associated with each respective user'"'"'s interaction in accordance with their respective sweet spots.

16. A method according to claim 1, where two or more users are identified, in which the step of estimating the position of the user relative to the estimated position of the sweet spot comprises:

identifying a plurality of users within a captured video image;

defining a notional boundary dependent upon the lateral and axial extent of a group of some or all of these users; and

adjusting the output timing of sound data signals for reproduction by respective loudspeakers in accordance with the difference between the estimated position of a sweet spot and the notional boundary, so that the resultant acoustic signals reach the notional boundary substantially simultaneously.

26. An entertainment device, comprising:

a generator of a source sound for reproduction by a plurality of loudspeakers at a current position where acoustic signals from the plurality of loudspeakers are coincident at a sweet spot;

a multi-channel delay operable to adjust delays applied to the plurality of loudspeakers, thereby adjusting the position of the sweet spot; and

communication means for receiving user inputs from a remote game controller;

in which the multi-channel delay is operable to adjust the output timing of source sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the user inputs so as to move the sweet spot.

17. An entertainment device according to claim 26, comprising:

a non-acoustic sensor;

a coincident signal estimator operable to estimate a physical location at which acoustic signals from a plurality of loudspeakers are coincident at the sweet spot;

a sensor analyser operable to estimate a physical location of a user from non-acoustic sensor data generated by the non-acoustic sensor; and

wherein

the multi-channel delay is operable to adjust the output timing of sound data signals for reproduction by respective ones of the plurality of loudspeakers in accordance with the difference between the estimated position of the user and the estimated position of the sweet spot so as to move the effective position of the sweet spot towards the estimated position of the user.

18. An entertainment device according to claim 17, in which the non-acoustic sensor is a video camera.

19. An entertainment device according to claim 18, in which the video camera comprises a distance measuring arrangement.

20. An entertainment device according to claim 18, in which the video camera comprises a microphone.

21. An entertainment device according to claim 17, in which the non-acoustic sensor is a phased radio array.