System and method for providing enhanced audio in a video environment

• US 9,338,394 B2
• Filed: 11/15/2010
• Issued: 05/10/2016
• Est. Priority Date: 11/15/2010
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Li, Wei, Mauchly, J. William, Mackie, David J., Williford, Olin D. II, Huang, Jinshi, Paszkowski, Pawel, Gajendran, Indrajit Rajeev, Wales, Richard T., Friel, Joseph T.
• Primary Examiner(s): Berhan, Ahmed A
• Application Number:
  US12/946,695
• Publication Number:
  US 20120120270A1
• Time in Patent Office: 2,003 Days
• Field of Search: 348/14.08, 348/14.09, 348/14.1, 348/211.12, 348/211.11, 348/211.3, 348/14.07, 348/211.2
• US Class Current: 1/1
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4398 involving reformatting operations of audio signals details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 5/60 for the sound signals
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence
• CPC Class Codes: H04S 7/305 Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence

Abstract

A method is provided in one example and includes receiving audio data at a microphone array that includes a plurality of microphones. The microphone array is provisioned at a first endpoint, which includes a camera element configured to capture video data associated with a video session involving the first endpoint and a second endpoint. The method also includes formatting the audio data into a time division multiplex (TDM) stream, and communicating the stream to a port for a subsequent communication over a network and to the second endpoint.

Claims

1. A method, comprising:

converting an audio session on device to a video session on the same device, wherein the audio session and the video session involve a first endpoint and a second endpoint;

requesting verification of video connectivity by, at least in part, displaying a graphic on a display of the video session and requesting confirmation for the video connectivity;

receiving audio data at a microphone array that includes a plurality of microphones, wherein one of the microphones in the plurality of microphones is wireless, wherein the microphone array is provisioned at the first endpoint, which includes a camera element configured to capture video data associated with the video session;

formatting the audio data into a time division multiplex (TDM) stream; and

communicating the stream to a port for a subsequent communication over a network and to the second endpoint.

2. The method of claim 1, wherein the plurality of microphones is logarithmically spaced from each other, and wherein the microphone array includes Micro Electrical-Mechanical System (MEMS) microphones and an embedded watermark signature provided in speakers of the microphone array is used to determine a delay for the audio data.

3. The method of claim 1, wherein a plurality of audio codecs is configured to convert a digital output from the microphone array into a different format for interfacing with a transmitter integrated circuit.

4. The method of claim 1, further comprising:

measuring a delay associated with the audio data in order to cancel echoing effects associated with the video session, wherein the delay is measured by playing music during an initial startup phase associated with a video conferencing platform.

5. The method of claim 1, further comprising:

providing a delay for at least a portion of the audio data, wherein the delay is based on a type of display being used to render images associated with the video session.

6. The method of claim 1, wherein a delay for the audio data is dynamically updated based on a clocking mechanism associated with a display for rendering images associated with the video session.

7. Logic encoded in one or more non-transitory tangible media that includes code for execution and when executed by a processor operable to perform operations comprising:

converting an audio session on a device to a video session on the same device, wherein the audio session and the video session involve a first endpoint and a second endpoint;

requesting verification of video connectivity by, at least in part, displaying a graphic on a display of the video session and requesting confirmation for the video connectivity;

receiving audio data at a microphone array that includes a plurality of microphones, wherein one of the microphones in the plurality of microphones is wireless, wherein the microphone array is provisioned at the first endpoint, which includes a camera element configured to capture video data associated with the video session;

formatting the audio data into a time division multiplex (TDM) stream; and

communicating the stream to a port for a subsequent communication over a network and to the second endpoint.

8. The logic of claim 7, wherein the plurality of microphones is logarithmically spaced from each other, and wherein the microphone array includes Micro Electrical-Mechanical System (MEMS) microphones and an embedded watermark signature provided in speakers of the microphone array is used to determine a delay for the audio data.

9. The logic of claim 7, wherein a plurality of audio codecs is configured to convert a digital output from the microphone array into a different format for interfacing with a transmitter integrated circuit.

10. The logic of claim 7, the operations further comprising:

measuring a delay associated with the audio data in order to cancel echoing effects associated with the video session, wherein the delay is measured by playing music during an initial startup phase associated with a video conferencing platform.

11. The logic of claim 7, the operations further comprising:

providing a delay for at least a portion of the audio data, wherein the delay is based on a type of display being used to render images associated with the video session.

12. The logic of claim 7, wherein a delay for the audio data is dynamically updated based on a clocking mechanism associated with a display for rendering images associated with the video session.

13. An apparatus, comprising:

a microphone array that includes a plurality of microphones, wherein one of the microphones in the plurality of microphones is wireless;

a memory element configured to store data; and

a processor operable to execute instructions associated with the data, wherein the processor and the memory element cooperate such that the apparatus is configured to;

convert an audio session on a device to a video session on the same device, wherein the audio session and the video session involve a first endpoint and a second endpoint;

requesting verification of video connectivity by, at least in part, displaying a graphic on a display of the video session and requesting confirmation for the video connectivity;

receive audio data at the first endpoint, which includes the microphone array and a camera element configured to capture video data associated with the video session;

format the audio data into a time division multiplex (TDM) stream; and

communicate the stream to a port for a subsequent communication over a network and to the second endpoint.

14. The apparatus of claim 13, wherein the plurality of microphones is logarithmically spaced from each other, and wherein the microphone array includes Micro Electrical-Mechanical System (MEMS) microphones and an embedded watermark signature provided in speakers of the microphone array is used to determine a delay for the audio data.

15. The apparatus of claim 13, wherein a plurality of audio codecs is configured to convert a digital output from the microphone array into a different format for interfacing with a transmitter integrated circuit.

16. The apparatus of claim 13, the apparatus being further configured to:

measure a delay associated with the audio data in order to cancel echoing effects associated with the video session, wherein the delay is measured by playing music during an initial startup phase associated with a video conferencing platform.

17. The apparatus of claim 13, further comprising:

a display configured to render images associated with the video session, wherein a delay is provided to at least a portion of the audio data based on the display being a certain type.

18. The apparatus of claim 13, further comprising:

a handset configured to control one or more application program interfaces (APIs) during the video session.