Method And System For Achieving Self-Adaptive Surround Sound

• US 20160134986A1
• Filed: 06/27/2014
• Published: 05/12/2016
• Est. Priority Date: 09/25/2013
• Status: Active Grant

• Current Assignee: Goertek Incorporated
• Original Assignee: Goertek Incorporated
• Inventors: Liu, Song, Lou, Shasha, Li, Bo
• Granted Patent: US 9,807,536 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04R 1/403 loud-speakers
• CPC Class Codes: H04R 2203/12 Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

A method and a system for achieving a self-adaptive surround sound. The method comprises: recognizing specific positions of a room and a user in the room by using an object recognition technology, capturing focusing images of recognized objects by controlling a camera using a focusing control technology, and recording corresponding focusing parameters (S110); calculating position information of the room relative to the camera and position information of the user relative to the camera according to the images and the parameters (S120); calculating sound beams that can achieve the surround sound at the position of the user in said room according to aforesaid calculated position information of the room and the user (S130); obtaining parameters of a filter group according to the calculated sound beams, and adjusting the filter group of a loudspeaker array according to the parameters (S140); and playing an audio signal via the loudspeaker array after the audio signal is filtered by the filter group that has been adjusted according to the parameters to form surround sound at the position of the user in the room (S150).

Claims

1. A method for achieving a self-adaptive surround sound, wherein the method comprises:

recognizing specific positions of a room and a user in the room by using an object recognition technology, capturing focusing images of recognized objects by controlling a camera using a focusing control technology, and recording corresponding focusing parameters;

calculating position information of the room relative to the camera and calculating position information of the user relative to the camera according to the captured focusing images and the recorded corresponding focusing parameters;

calculating paths of sound beams that can achieve the surround sound at the position of the user in said room according to the calculated position information of the room relative to the camera and the position information of the user relative to the camera and the relative positional relation between the camera and a loudspeaker array;

determining parameters of a filter group of the loudspeaker array according to the calculated paths of the sound beams, and adjusting the filter group of the loudspeaker array using the parameters;

playing an audio signal via the loudspeaker array after the audio signal is filtered by said filter group that has been adjusted according to the parameters to form surround sound at the position of the user in said room.

2. The method according to claim 1, wherein said recognizing specific positions of a room and a user in the room by using an object recognition technology, capturing focusing images of recognized objects by controlling a camera using a focusing control technology comprises:

recognizing a left corner, a right corner, a midpoint on the left floor-wall boundary, and a midpoint on the right floor-wall boundary by using a floor-wall boundary recognition technology, wherein the midpoints on the floor-wall boundaries are the midpoints on the part of the floor-wall boundaries that appear in the focusing images;

recognizing a user'"'"'s face in the room by using a face recognition technology;

after the objects are recognized, calculating the areas where the objects are located, and making the camera precisely focus on the recognized objects and form clear images according to the calculating result;

correspondently,

said calculating position information of the room relative to the camera comprises;

calculating object distances of the left corner, the right corner, the midpoint on the left floor-wall boundary, and the midpoint on the right floor-wall boundary from the camera and the angles of the left corner, the right corner, the midpoint on the left floor-wall boundary, and the midpoint on the right floor-wall boundary relative to a central axis of the camera lens, and calculating an equation of the left floor-wall boundary, an equation of the right floor-wall boundary and an equation of the opposite floor-wall boundary;

said calculating position information of the user relative to the camera comprises;

calculating an object distance of the user from the camera and an angle of the user relative to the central axis of the camera lens.

3. The method according to claim 2, wherein said making the camera precisely focus on the recognized objects and form clear images according to the calculating result comprises:

driving a focusing motor to make the camera precisely focus on the recognized objects and form clear images according to the calculating result;

or

using light field reestablishment to make the camera precisely focus on the recognized objects and form clear images according to the calculating result.

4. The method according to claim 2, wherein said sound beams that can achieve surround sound at the position of the user in the room refer to sound beams reaching the position of said user after one reflection or two reflection by walls, comprising:

a left sound beam 1, a left sound beam 2, a right sound beam 1, and a right sound beam 2, wherein;

the left sound beam 1 is a sound beam that is pointed towards the left wall and is reflected once by the left wall, then reaches the position of the user;

the left sound beam 2 is a sound beam that is pointed towards the left wall and is reflected once by the left wall, and then reflected once by the opposite wall, then reaches the position of the user;

the right sound beam 1 is a sound beam that is pointed towards the right wall and is reflected once by the right wall, then reaches the position of the user;

the right sound beam 2 is a sound beam that is pointed towards the right wall and is reflected once by the right wall, and then reflected once by the opposite wall, then reaches the position of the user.

5. The method according to claim 4, wherein said calculating paths of sound beams that can achieve the surround sound at the position of the user in said room according to the calculated position information of the room relative to the camera and the position information of the user relative to the camera and relative positional relation between the camera and a loudspeaker array comprises:

converting the position information of the room relative to the camera and position information of the user relative to the camera into position information of the room relative to the loudspeaker array and position information of the user relative to the loudspeaker array according to the relative positional relation of the camera and the loudspeaker array;

by taking the position of the loudspeaker array as a zero point, first obtaining a mirror image point of the user relative to the left floor-wall boundary, and then obtaining the equation of the left sound beam 1 according to coordinates of the zero point and the mirror image point and on the basis of the principle of two points defining a straight line;

by taking the position of the loudspeaker array as a zero point, first obtaining a mirror image point 1 of the user relative to the opposite floor-wall boundary, and then obtaining a mirror image point 2 of the mirror image point 1 relative to the left floor-wall boundary according to the mirror image point 1, and then obtaining the equation of the left sound beam 2 according to the coordinates of the zero point and the mirror image point 2 and on the basis of the principle of two points defining a straight line;

similarly, obtaining the equations of the right sound beam 1 and right sound beam 2.

6. The method according to claim 4, wherein the method further comprises:

when the image of the left corner and/or the right corner cannot be captured, making directions of two sound beams on the corresponding side deflect certain angles from the user;

or,

when any of said left sound beam 1, left sound beam 2, right sound beam 1 and right sound beam 2 is blocked by the user, canceling the sound beam.

7. A system for achieving a self-adaptive surround sound, wherein the system comprises:

a camera, an object recognition and imaging unit, a position information acquiring unit, a sound beam control unit, and a loudspeaker array;

the camera configured to capture images;

the object recognition and imaging unit configured to recognize specific positions of a room and a user in the room by using an object recognition technology, capture focusing images of recognized objects by controlling the camera using a focusing control technology, and record corresponding focusing parameters;

the position information acquiring unit configured to calculate position information of the room relative to the camera and position information of the user relative to the camera according the captured focusing images and the recorded corresponding focusing parameters;

the sound beam control unit configured to calculate paths of sound beams that can achieve the surround sound at the position of the user in said room according to the calculated position information of the room relative to the camera, the position information of the user relative to the camera and the relative positional relation between the camera and the loudspeaker array; and

determine parameters of a filter group of the loudspeaker array according to the calculated paths of sound beams, and adjust the filter group of the loudspeaker array using the parameters;

the loudspeaker array configured to play an audio signal after the audio signal is filtered by the filter group that has been adjusted according to the parameters to form surround sound at the position of the user in said room.

8. The system according to claim 7, wherein,

said object recognition and imaging unit is specifically configured to recognize a left corner, a right corner, a midpoint on the left floor-wall boundary, and a midpoint on the right floor-wall boundary by using a floor-wall boundary recognition technology, wherein the midpoints on the floor-wall boundaries are the midpoints on the part of the floor-wall boundaries that appear in the focusing images;

to recognize a user'"'"'s face in the room by using a face recognition technology;

calculate the areas where the objects are located after the objects are recognized, and make the camera precisely focus on the recognized objects and form clear images according to the calculating result;

correspondently,

said position information acquiring unit is specifically configured to calculate object distances of the left corner, the right corner, the midpoint on the left floor-wall boundary, and the midpoint on the right floor-wall boundary from the camera and the angles of the left corner, the right corner, the midpoint on the left floor-wall boundary, and the midpoint on the right floor-wall boundary relative to the central axis of the camera lens, and calculate an equation of the left floor-wall boundary, an equation of the right floor-wall boundary and an equation of the opposite floor-wall boundary; and

calculate the object distance of the user from the camera and an angle of the user relative to the central axis of the camera lens.

9. The system according to claim 8, wherein said object recognition and imaging unit is specifically configured to drive a focusing motor to make the camera precisely focus on the recognized objects and form clear images according to the calculating result;

or use light field reestablishment to make the camera precisely focus on the recognized objects and form clear images according to the calculating result.

10. The system according to claim 8, wherein,

said sound beams that can achieve surround sound at the position of the user in the room refer to sound beams reaching the position of said user after one reflection or two reflection by walls, comprising:

a left sound beam 1, a left sound beam 2, a right sound beam 1, and a right sound beam 2, wherein;

the left sound beam 1 is a sound beam that is pointed towards the left wall and is reflected once by the left wall, then reaches the position of the user;

the left sound beam 2 is a sound beam that is pointed towards the left wall and is reflected once by the left wall, and then reflected once by the opposite wall, then reaches the position of the user;

the right sound beam 1 is a sound beam that is pointed towards the right wall and is reflected once by the right wall, then reaches the position of the user;

the right sound beam 2 is a sound beam that is pointed towards the right wall and is reflected once by the right wall, and then reflected once by the opposite wall, then reaches the position of the user;

correspondently,

said sound beam control unit is specifically configured to convert the position information of the room relative to the camera and position information of the user relative to the camera into position information of the room relative to the loudspeaker array and position information of the user relative to the loudspeaker array according to the relative positional relation of the camera and the loudspeaker array;

by taking the position of the loudspeaker array as a zero point, first obtain the mirror image point of the user relative to the left floor-wall boundary, and then obtain the equation of the left sound beam 1 according to the coordinates of zero point and the mirror image point and on the basis of the principle of two points defining a straight line;

by taking the position of the loudspeaker array as a zero point, first obtain a mirror image point 1 of the user relative to the opposite floor-wall boundary, and then obtain a mirror image point 2 of the mirror image point 1 relative to the left floor-wall boundary according to the mirror image point 1, and then obtain the equation of the left sound beam 2 according to the coordinates of the zero point and the mirror image point 2 and on the basis of the principle of two points defining a straight line;

similarly, obtain the equations of the right sound beam 1 and right sound beam 2.

11. The system according to claim 10, wherein said sound beam control unit is further configured to, when the images of the left corner and/or the right corner cannot be captured, making the directions of the two sound beams on the corresponding side deflect certain angles from the user;

or, when any of said left sound beam 1, left sound beam 2, right sound beam 1 and right sound beam 2 is blocked by the user, canceling the sound beam.

12. The system according to claim 7, wherein said the camera is a passive focusing camera, an active focusing camera, or a camera with depth perception.