Schemes for Emphasizing Talkers in a 2D or 3D Conference Scene

• US 20150052455A1
• Filed: 03/21/2013
• Published: 02/19/2015
• Est. Priority Date: 03/23/2012
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Boustead, Paul, Spittle, Gary
• Granted Patent: US 9,961,208 B2
• Time in Patent Office: Days
• US Class Current: 715/753
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30

Abstract

The present document relates to methods and systems for setting up and managing two-dimensional or three-dimensional scenes for audio conferences. A conference controller (111, 175) configured to place a plurality of upstream audio signals (123, 173) associated with a plurality of conference participants within a 2D or 3D conference scene to be rendered to a listener (211) is described. The conference controller (111, 175) is configured to set up a X-point conference scene with X different spatial talker locations (212) within the conference scene; assign the plurality of upstream audio signals (123, 173) to respective ones of the talker locations (212); determine a degree of activity of the plurality of upstream audio signals (123, 173); determine a dominant one of the plurality of upstream audio signals (123, 173); and emphasize the dominant upstream audio signal (123, 173).

Claims

1-36. -36. (canceled)

37. ) A conference controller configured to place a plurality of upstream audio signals associated with a plurality of conference participants within a 2D or 3D conference scene to be rendered to a listener, wherein the conference controller is configured to

set up a X-point conference scene with X different spatial talker locations within the conference scene, X being an integer, X>

0;

assign the plurality of upstream audio signals to respective ones of the talker locations;

determine a degree of activity of the plurality of upstream audio signals at a time instant;

determine a dominant one of the plurality of upstream audio signals at the time instant based on the degrees of activity of the plurality of upstream audio signals at the time instant;

assign the dominant upstream audio signal to a first of the X talker locations; and

emphasize the dominant upstream audio signal at the time instant by rotating the X talker locations around a head of the listener to yield updated talker locations such that the updated talker location of the dominant upstream audio signal is the updated talker location closest to a midline in front of a head of the listener.

38. ) The conference controller of claim 37, configured to

generate metadata identifying the talker locations of the plurality of upstream audio signals and enabling an audio processing unit to generate a spatialized audio signal based on a set of downstream audio signals;

wherein the set of downstream audio signals comprises the dominant upstream audio signal;

wherein when rendering the spatialized audio signal to the listener, the listener perceives the dominant upstream audio signal in an emphasized manner.

39. ) The conference controller of claim 37, wherein the conference controller is configured to determine the degree of activity of an upstream audio signal at the time instant by determining an energy of the upstream audio signal at the time instant.

40. ) The conference controller of claim 37, wherein the conference controller is configured to determine a dominant one of the plurality of upstream audio signals by determining an upstream audio signal having the highest degree of activity at the time instant.

41. ) The conference controller of claim 37, wherein

the conference controller is configured to re-assign an upstream audio signal already assigned to the talker location closest to the midline to another talker location.

42. ) The conference controller of claim 37, wherein the conference controller is configured to emphasize the dominant upstream audio signal at the time instant by increasing a rendering volume of the dominant upstream audio signal at the time instant.

43. ) The conference controller of claim 37, wherein the conference controller is configured to

emphasize the dominant upstream audio signal at the time instant by moving the first talker location closer to the listener.

44. ) The conference controller of claim 37, wherein

the X talker locations are positioned within a cone around the midline in front of the head of the listener;

a generatrix of the cone and the midline form an angle which is smaller than or equal to a pre-determined maximum cone angle; and

the conference controller is configured to rotate the conference scene such that all updated talker locations are positioned within the cone around the midline.

45. ) The conference controller of claim 37, wherein the conference controller is configured to reduce an angular distance between adjacent talker locations, in order to determine the updated talker locations.

46. ) The conference controller of claim 37, wherein the conference controller is configured to

determine a different new dominant one of the plurality of upstream audio signals at a second time instant after the time instant;

de-emphasize the former dominant upstream audio signal at the second time instant; and

emphasize the new dominant upstream audio signal at the second time instant.

47. ) The conference controller of claim 37, wherein the conference controller is configured to classify the X spatial talker locations into a plurality of clusters;

wherein a first of the plurality of clusters comprises at least two spatial talker locations;

wherein the spatial talker locations comprised within the first cluster are directly adjacent.

48. ) The conference controller of claim 47, wherein the conference controller is configured to classify the X spatial talker locations into the plurality of clusters dependent upon classification metadata.

49. ) The conference controller of claim 48, wherein the classification metadata comprises at least one of:

an identifier associated with an electronic means of communication of a conference participant; and

an identifier associated with a physical location of a conference participant.

50. ) The conference controller of claim 48, wherein the conference controller is configured to extract the classification metadata from one or more of the plurality of upstream audio signals.

51. ) The conference controller of claim 48, wherein the conference controller is configured to facilitate input of the classification metadata by a conference participant.

52. ) The conference controller of claim 44, wherein the conference controller is configured to calculate the X-point conference scene with X different spatial talker locations such that the X talker locations are positioned within the cone around the midline in front of the head of the listener.

53. ) The conference controller of claim 37, wherein the conference controller is configured to select the X-point conference scene with X different spatial talker locations from a set of pre-determined X-point conference scenes with X different pre-determined spatial talker locations.

54. ) The conference controller of claim 37, wherein the conference controller is configured to emphasize the dominant upstream audio signal at the time instant by modifying a height of the first talker location relative to the others of the X spatial talker locations.

55. ) An audio conferencing system comprising:

a plurality of talker terminals configured to generate a plurality of upstream audio signals associated with a plurality of conference participants, respectively;

a conference controller according to claim 37, configured to

assign the plurality of upstream audio signals to respective talker locations within a 2D or 3D conference scene;

determine a dominant one of the plurality of upstream audio signals;

assign the dominant upstream audio signal to a first of the X talker locations; and

emphasize a dominant one of the plurality of upstream audio signals by re-assigning the dominant upstream audio signal to a center location within the 2D or 3D conference scene;

wherein the center location corresponds to the talker location closest to a midline in front of a head of the listener; and

a listener terminal configured to render the dominant upstream audio signal to a listener, such that the listener perceives the dominant upstream audio signal in an emphasized manner.

56. ) A method for placing a plurality of upstream audio signals associated with a plurality of conference participants within a 2D or 3D conference scene to be rendered to a listener, wherein the method comprises

setting up a X-point conference scene with X different spatial talker locations within the conference scene, X being an integer, X>

0;

assigning the plurality of upstream audio signals to respective ones of the talker locations;

determining a degree of activity of the plurality of upstream audio signals, at a time instant;

determining a dominant one of the plurality of upstream audio signals at the time instant based on the degrees of activity of the plurality of upstream audio signals at the time instant;

assigning the dominant upstream audio signal to a first of the X talker locations; and

emphasizing the dominant upstream audio signal at the time instant by rotating the X talker locations around a head of the listener to yield updated talker locations such that the updated talker location of the dominant upstream audio signal is the updated talker location closest to a midline in front of a head of the listener.

57. ) A storage medium comprising a software program adapted for execution on a processor and for performing the method steps of claim 56 when carried out on a computing device.