Heterogeneous video conferencing

• US 8,249,237 B2
• Filed: 05/02/2011
• Issued: 08/21/2012
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Jeong, Hyeonkuk, Abuan, Joe, Normile, Jim, Salsbury, Ryan, Tung, Berkat Shing
• Primary Examiner(s): Goins, Davetta W
• Assistant Examiner(s): ISLAM, MOHAMMAD K
• Application Number:
  US13/099,331
• Publication Number:
  US 20110205332A1
• Time in Patent Office: 477 Days
• Field of Search: 705/8, 705/9, 707/10, 709/204, 379/202.01, 348/14.01, 348/14.02, 370260-267
• US Class Current: 379/202.01
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04Q 2213/13093 Personal computer, PC
• CPC Class Codes: H04Q 2213/1324 Conference call
• CPC Class Codes: H04Q 3/0045 involving hybrid, i.e. a mixture of public and private, or multi-vendor systems

Abstract

Some embodiments provide an architecture for establishing a multi-participant conference. This architecture has one participant'"'"'s computer in the conference act as a central content distributor for the conference. The central distributor receives data (e.g., video and/or audio streams) from the computer of each other participant, and distributes the received data to the computers of all participants. In some embodiments, the central distributor receives A/V data from the computers of the other participants. From such received data, the central distributor of some embodiments generates composite data (e.g., composite image data and/or composite audio data) that the central distributor distributes back to the participants. The central distributor in some embodiments can implement a heterogeneous audio/video conference. In such a conference, different participants can participate in the conference differently. For instance, different participants might use different audio or video codecs. Moreover, in some embodiments, one participant might participate in only the audio aspect of the conference, while another participant might participate in both audio and video aspects of the conference.

Claims

1. A non-transitory computer readable medium storing a computer program for execution by at least one processor of a central distributor device participating in a conference, the computer program comprising sets of instructions for:

receiving conference content data from (i) a first non-central distributor device according to a first set of conference parameters and (ii) a second non-central distributor device according to a second, different set of conference parameters, the first and second non-central distributor devices participating in the conference;

generating composite conference content data using (i) the received conference content data and (ii) conference content data captured locally at the central distributor device; and

distributing the composite conference content data to (i) the first non-central distributor device according to the first set of conference parameters and (ii) the second non-central distributor device according to the second set of conference parameters.

2. The non-transitory computer readable medium of claim 1, wherein the conference content data from the first non-central distributor device comprises video data and audio data while the conference content data from the second non-central distributor device comprises only audio data.

3. The non-transitory computer readable medium of claim 2, wherein the set of instructions for receiving conference content data comprises sets of instructions for:

defining a video-decoding pipeline for the conference content data from the first non-central distributor device;

defining an audio-decoding pipeline for the conference content data from the first non-central distributor device; and

defining an audio-decoding pipeline for the conference content data from the second non-central distributor device.

4. The non-transitory computer readable medium of claim 3, wherein no video-decoding pipeline is generated for the conference content data from the second non-central distributor device.

5. The non-transitory computer readable medium of claim 1, wherein the conference content data from the first non-central distributor device comprises video images having a first resolution and the conference content data from the second non-central distributor device comprises video images having a second, different resolution.

6. The non-transitory computer readable medium of claim 5, wherein the set of instructions for generating the composite conference content data comprises sets of instructions for:

resizing the video images from the first non-central distributor device from the first resolution to a composite buffer size; and

resizing the video images from the second non-central distributor device from the second resolution to the composite buffer size.

7. The non-transitory computer readable medium of claim 6, wherein the set of instructions for generating the composite conference content data further comprises a set of instructions for generating composite video images using the resized video images.

8. The non-transitory computer readable medium of claim 1, wherein the conference content data from the first non-central distributor device comprises video images encoded in a first format and the conference content data from the second non-central distributor device comprises video images encoded in a second format.

9. The non-transitory computer readable medium of claim 8, wherein the first format is H.263 and the second format is H.264.

10. The non-transitory computer readable medium of claim 8, wherein the set of instructions for receiving the composite conference content data comprises sets of instructions for:

decoding the video images from the first non-central distributor device using a first decoder; and

decoding the video images from the second non-central distributor device using a second decoder.

11. A method for providing a conferencing application for a device that participates in a conference as a central distributor of the conference, the method comprising:

providing a local content capture module for capturing conference content at the device;

providing a set of receiver modules for receiving, at the device, conference content from a plurality of non-central distributor devices, wherein at least two of the non-central distributor devices transmit conference content to the device according to different conference parameters;

providing a compositing module for producing composite conference content data comprising data captured by the local content capture module and data received by at least one of the receiver modules; and

providing an output module for distributing the composite conference content data to the plurality of non-central distributor devices.

12. The method of claim 11, wherein the set of receiver modules comprises a set of decoders for decoding video images from the non-central distributor devices, the set of decoders comprising at least two different decoders for decoding video images in different formats.

13. The method of claim 11, wherein the set of receiver modules comprises a set of resizers for resizing video images from the non-central distributor devices to a standard size, the set of decoders comprising at least two different resizers for resizing video images having different resolutions.

14. The method of claim 11, wherein the compositing module comprises:

a compositing buffer for storing video images from the non-central distributor devices and the local conference capture module; and

an encoder for encoding a composite video image from the video images stored in the compositing buffer.

15. The method of claim 14, wherein the output module comprises a redundancy remover for removing a particular video image from the composite video image when distributing the composite video image to a non-central distributor device from which the device received the particular video image.

16. The method of claim 11, wherein the local content capture module comprises a module for receiving images from a camera at the device.

17. The method of claim 11, wherein the local content capture module comprises a module for receiving audio from a microphone at the device.

18. A conference system comprising:

a first non-central distributor device for transmitting conference content data according to a first set of conference parameters;

a second non-central distributor device for transmitting conference content data according to a second set of conference parameters; and

a central distributor device for (i) receiving the transmitted conference content data from the first and second non-central distributor device, (ii) generating composite conference content data from the received conference content data and additional conference content data captured locally at the central distributor device, (iii) transmitting the generated composite conference content data to the first and second non-central distributor devices, and (iv) outputting the generated composite conference content data locally at the central distributor device.

19. The system of claim 18, wherein outputting the generated composite conference content data locally comprises displaying a composite video image generated using images received from the first and second non-central distributor devices and locally captured by a camera at the central distributor device.

20. The system of claim 18 further comprising at least one additional non-central distributor device for transmitting conference content according to a third set of conference parameters.