Video encoding in a video conference

• US 20060244819A1
• Filed: 04/28/2005
• Published: 11/02/2006
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Jeong, Hyeonkuk, Pun, Thomas, Wu, Hsi Jung
• Granted Patent: US 7,692,682 B2
• Time in Patent Office: Days
• US Class Current: 348/14.90
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Some embodiments provide an architecture for establishing multi-participant video conferences. This architecture has a central distributor that receives video images from two or more participants. From the received images, the central distributor generates composite images that the central distributor transmits back to the participants. Each composite image includes a set of sub images, where each sub image belongs to one participant. In some embodiments, the central distributor saves network bandwidth by removing each particular participant'"'"'s image from the composite image that the central distributor sends to the particular participant. In some embodiments, images received from each participant are arranged in the composite in a non-interleaved manner. For instance, in some embodiments, the composite image includes at most one sub-image for each participant, and no two sub-images are interleaved.

Claims

1. A method of encoding an image during video conference with multiple participants, the method comprising:

a) generating a composite image having at least two sub-images for at least two participants;

b) constraint encoding each sub-image separately, wherein said constraint encoding comprises encoding each sub-image based on a set of constraints that allow the encoding of the sub-image to be independent of the encoding of the other sub-images.

2. The method of claim 1,

a) wherein the encoding comprises performing a motion estimation process for an interblock encoding operation;

b) wherein the set of constraints includes a constraint on a location of a search window used during the motion estimation process.

3. The method of claim 2, wherein performing the motion estimation process comprises:

a) said motion estimation process entails performing a search within a target sub-image;

b) wherein the location constraint requires the search window to be located within the target sub-image.

4. The method of claim 3, further comprising:

a) identifying a search window that partially falls outside the target sub-image;

b) cropping the identified search window so that it entirely falls within the target sub-image.

5. The method of claim 3, further comprising:

a) identifying a search window that at least partially falls outside the target sub-image;

b) moving the identified search window so that it entirely falls within the target sub-image.

6. The method of claim 1,

a) wherein the encoding comprises examining blocks that are aligned with sub-pixel locations;

b) wherein the set of constraints includes a constraint on a region within which blocks aligned with sub-pixel locations can be examined.

7. The method of claim 6, wherein said examination comprises performing interpolation operations for sets of image values of sets of pixels.

8. The method of claim 7 further comprising establishing a set of boundary regions between sub-images, wherein the dimensions of the boundary regions are determined by the number of pixels in the sets of pixels used for said interpolation operations.

9. The method of claim 8 further comprising foregoing examining blocks aligned with a plurality of sub-pixel locations that are within the boundary region.

10. The method of claim 9, wherein the blocks that are not examined within the boundary regions are a plurality of blocks aligned with a plurality of half pixel locations.

11. The method of claim 10, further comprising examining a plurality of blocks aligned with a plurality of half pixel locations that are within the boundary regions and are located between two integer pixel locations within the same boundary region.

12. The method of claim 9, wherein the blocks that are not examined within the boundary regions are a plurality of blocks aligned with quarter pixel locations.

13. The method of claim 12, further comprising examining a plurality of blocks aligned with a plurality of quarter-pixel locations that are within the boundary regions and can be interpolated utilizing integer pixel locations and half-pixel locations within two integer locations in the same boundary region.

14. The method of claim 1, wherein the constraint encoding comprises:

a) performing intra prediction for a set of pixels in a particular sub-image;

b) excluding from the intra prediction a set of pixels that fall outside of the particular sub-image.

15. The method of claim 1 further comprising:

a) performing a loop-filter operation during the encoding of a particular sub-image; and

b) excluding pixels that are not in the particular sub-image from the loop-filter operation.

16. A method of distributing video content in a multi-participant video conference, said conference having one central distributor of video content, the method comprising:

a) at the central distributor, receiving video images from each participant;

b) generating composite images from the received video images;

c) encoding the composite images; and

d) transmitting said encoded composite images to said participants.

17. The method of claim 16, wherein said composite image includes an interblock encoded image for a first participant and an intrablock encoded image for a second participant.

18. The method of claim 16 further comprising:

a) detecting a condition requiring refreshing of a particular participant'"'"'s image data through intrablock encoded data;

b) intrablock encoding a portion of a first image of the particular participant;

c) transmitting the intrablock encoded portion of the first image in a first composite image;

d) intrablock encoding a first portion of a second image of the particular participant;

e) transmitting the intrablock encoded portion of the second image in a second composite image;

f) wherein the intrablock encoded portions of the first and second images refreshes the particular participant'"'"'s image data.

19. The method of claim 18 further comprising interblock encoding a second portion of the second image and transmitting the interblock encoding of the second portion, wherein the second portion corresponds to the portion of the first image that was intrablock encoded.

20. A method of distributing video content in a video conference between at least two participants, the method comprising:

a) detecting a condition requiring refreshing of a first participant'"'"'s image data through intrablock encoded data;

b) intrablock encoding a portion of a first image of the first participant;

c) transmitting the intrablock encoded portion of the first image to the second participant;

d) intrablock encoding a first portion of a second image of the first participant;

e) transmitting the intrablock encoded portion of the second image to the second participant;

f) wherein the intrablock encoded portions of the first and second images refreshes the first participant'"'"'s image data.

21. The method of claim 20 further comprising interblock encoding a second portion of the second image and transmitting the interblock encoding of the second portion to the second participant, wherein the second portion corresponds to the portion of the first image that was intrablock encoded.

22. A method of distributing video content in a video conference between two participants, the method comprising:

a) storing a long term reference frame at first participant;

b) first participant informing second participant that first participant has dropped video data transmitted by second participant;

c) second participant sending inter-encoded data from the second participant to the first participant where the inter-encoded data is defined by reference to the long term reference frame.