Sharing reference data between multiple encoders parallel encoding a sequence of video frames

• US 6,356,589 B1
• Filed: 01/28/1999
• Issued: 03/12/2002
• Est. Priority Date: 01/28/1999
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Vachon, Michael Patrick, Gebler, Charlene Ann, Hall, Barbara A., Ngai, Agnes Yee
• Primary Examiner(s): Le, Vu
• Application Number:
  US09/239,381
• Time in Patent Office: 1,139 Days
• Field of Search: 375/240.1, 375/240.08, 375/240.14, 375/240.16, 375/240.24, 348/397.1, 348/402.1, 348/407.1, 348/411.1, 348/413.1, 348/416.1, 348/420.1, 348/699, 382/234, 382/236, 382/238, 382/243, 386/109, 386/111, 358/261.2, 358/430
• US Class Current: 375/240.1
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements

Abstract

A digital video encode system, method and computer program product are described wherein a picture is divided into multiple active areas that are parallel encoded using multiple encoders. The encoders associated with adjoining active areas of the picture are configured to exchange reference data to facilitate avoiding appearance of a seam between the adjoining active areas of a subsequent picture in a sequence of pictures due to separate encoding thereof by the multiple encoders. The reference data transferred from one encoder to an adjoining encoder is an encoded and decoded section of the picture and is used by the adjoining encoder as an expanded reference area for motion estimation when encoding the subsequent picture.

Claims

1. A method for processing a sequence of video frames comprising:

employing a first encoder to encode a first area of a frame of said sequence of video frames and a second encoder to encode a second area of said frame, wherein said first area and said second area comprise adjoining areas within said frame; and

making reference data of said first encoder available to said second encoder, said reference data arising from encoding said first area of said frame and being made available to said second encoder for use in encoding said second area of a subsequent frame of said sequence of video frames to facilitate avoiding appearance of a seam between said adjoining first and second areas of said subsequent frame of the sequence of video frames due to separate encoding thereof by said first encoder and said second encoder.

2. The method of claim 1, wherein said reference data of said first encoder is used by said second encoder to determine motion vectors for said second area of said subsequent frame at a common boundary of said first area and said second area.

3. The method of claim 2, wherein said making of reference data of said first encoder available to said second encoder comprises transferring said reference data from said first encoder to said second encoder after encoding of said first area of said frame, wherein said reference data comprises an encoded and decoded section of said first area along said common boundary of said first area and said second area.

4. The method of claim 1, further comprising making reference data of said second encoder available to said first encoder for use by said first encoder in encoding said first area of said subsequent frame of said sequence of video frames to facilitate avoiding appearance of said seam between said adjoining first and second areas.

5. The method of claim 4, wherein said making reference data of said first encoder available to said second encoder comprises transferring said reference data from said first encoder to said second encoder, and wherein said making reference data of said second encoder available to said first encoder comprises transferring said reference data of said second encoder to said first encoder.

6. The method of claim 5, wherein said first area comprises an upper portion of said frame of said sequence of frames and said second area comprises a lower portion of said frame of said sequence of video frames, and wherein said reference data of said first encoder comprises an encoded and decoded lower section of said first area along a common boundary of said first and second areas, and wherein said reference data of said second encoder comprises an encoded and decoded upper section of said second area along said common boundary of said first and second areas.

7. The method of claim 1, further comprising employing N additional encoders to encode N additional areas of said frame of said sequence of video frames, wherein N≧

1, and wherein each area of said first area, second area, and N additional areas has at least one adjoining area within said frame, and wherein said method further comprises making reference data of an encoder of an adjoining area at least partially available to an encoder for use in encoding each area within said subsequent frame of said sequence of video frames to facilitate avoiding appearance of seams between adjoining areas due to encoding thereof by different encoders of said first encoder, second encoder and N additional encoders.

8. The method of claim 1, wherein said reference data comprises an expanded reference size for said second area for use in encoding said subsequent frame of said sequence of video frames.

9. The method of claim 8, further comprising repeating said method for each frame of said sequence of video frames.

10. The method of claim 1, wherein said sequence of video frames comprises a stream of high definition television pictures.

11. The method of claim 1, wherein said first encoder and said second encoder comprise two encoders of a plurality of encoders employed in processing said sequence of video frames, and wherein said plurality of encoders are coupled to encode a plurality of areas of said frame, said plurality of areas being disposed in a tile configuration, with said first area and said second area comprising diagonally adjoining areas within said frame, and wherein said making reference data of said first encoder available to said second encoder comprises transferring said reference data from said first encoder to said second encoder through a third encoder of said plurality of encoders, said third encoder encoding a third area of said frame, said third area having a first common boundary with said first area and a second common boundary with said second area.

12. The method of claim 11, wherein said transferring of said reference data from said first encoder to said second encoder through said third encoder comprises first transferring data from an active region of said first encoder to said third encoder, and then transferring data from an inactive region of said third encoder to said second encoder wherein said reference data transferred from said first encoder to said second encoder comprises at least a portion of said data from said inactive region transferred from said third encoder to said second encoder.

13. A method for processing a sequence of video frames employing a plurality of encoders, said plurality of encoders being coupled to parallel process a plurality of active areas of a frame of said sequence of video frames, said plurality of active areas being disposed in a tile configuration with said plurality of encoders also being coupled in said tile configuration, wherein said method comprises for each encoder of at least one encoder of said plurality of encoders:

determining for said encoder whether there is a top connection with a top encoder of said plurality of encoders, and if so, transferring to said encoder reference data from said active area being encoded by said top encoder;

determining for said encoder whether there is a bottom connection with a bottom encoder of said plurality of encoders, and if so, transferring to said encoder reference data from said active area being encoded by said bottom encoder;

determining for said encoder whether there is a left connection with a left encoder of said plurality of encoders, and if so, transferring to said encoder reference data R from said active area of said left encoder, and thereafter transferring top or bottom reference data aligned above or below, respectively, reference data R of the active area of said left encoder; and

determining for said encoder whether there is a right connection to a right encoder of said plurality of encoders, and if so, transferring to said encoder reference data L from said active area of said right encoder, and thereafter transferring top or bottom reference data aligned above or below, respectively, reference data L of the active area of said right encoder.

14. A method for processing a sequence of video frames comprising:

employing multiple encoders to encode multiple areas of a frame of said sequence of video frames, each area of said frame having at least one adjoining area within said frame so that each encoder has at least one adjoining encoder; and

making reference data available from each encoder to an adjoining encoder for use in encoding a subsequent frame of said sequence of video frames to facilitate avoiding appearance of seams between adjoining areas of said subsequent frame of said sequence of video frames due to separate encoding thereof by said multiple encoders.

15. The method of claim 14, further comprising for each frame of said sequence of video frames, dividing said frame into said multiple areas and forwarding each area to a respective encoder of said multiple encoders.

16. The method of claim 14, wherein said reference data made available to each adjoining encoder comprises an expanded reference area for use in motion estimation by said adjoining encoder when said subsequent frame is being encoded.

17. The method of claim 14, wherein said making reference data available from each encoder comprises making an encoded and decoded section of said area encoded by said encoder available to said adjoining encoder, wherein said encoded and decoded section is along a common boundary between said encoder and said adjoining encoder.

18. The method of claim 14, wherein said making reference data available from each encoder to said adjoining encoder comprises transferring said reference data from said encoder to said adjoining encoder for use in encoding said subsequent frame of said sequence of video frames.

19. The method of claim 14, further comprising repeating said method for multiple frames of said sequence of video frames, and wherein said sequence of video frames comprises one of a stream of high definition television pictures or a stream of digital video disc pictures.

20. The method of claim 14, wherein each area comprises a rectangular area within said frame, and wherein said making reference data available from each encoder comprises for each encoder determining whether there is a connection with an adjoining encoder, and if so, then making a predetermined amount of reference data available to said adjoining encoder.

21. The method of claim 20, wherein said determining whether said encoder has an adjoining encoder comprises evaluating its associated area for a top connection, bottom connection, right connection or left connection with at least one adjoining area.

22. An encode system for processing a sequence of video frames comprising:

a first encoder for encoding a first area of a frame of said sequence of video frames;

a second encoder for encoding a second area of said frame of said sequence of video frames, wherein said first area and said second area comprise adjoining areas within said frame; and

said first encoder being adapted to make available to said second encoder reference data arising from encoding of said first area of said frame, said reference data being employed by said second encoder in encoding said second area of a subsequent frame of said sequence of video frames to facilitate avoiding appearance of a seam between said adjoining first and second areas of said subsequent frame due to said separate encoding thereof by said first encoder and said second encoder.

23. The encode system of claim 22, wherein said reference data of said first encoder is used by said second encoder to determine motion vectors for said second area of said subsequent frame at a common boundary of said first area and said second area.

24. The encode system of claim 22, wherein said first encoder comprises means for transferring to said second encoder said reference data arising from encoding of said first area of said frame, wherein said reference data comprises an encoded and decoded section of said first area along a common boundary of said first area and said second area.

25. The encode system of claim 24, wherein said means for transferring said reference data from said first encoder to said second encoder comprises means for transferring said reference data from a memory of said first encoder to a memory of said second encoder subsequent to encoding and decoding of said first area.

26. The encode system of claim 22, wherein said reference data comprises an expanded reference size for said second area for use in encoding said subsequent frame of said sequence of video frames.

27. The encode system of claim 22, wherein said sequence of video frames comprises a stream of high definition television pictures.

28. The encode system of claim 22, wherein said second encoder is adapted to make available to said first encoder reference data arising from encoding of said second area of said frame, said reference data being employed by said first encoder in encoding said first area of a subsequent frame of said sequence of video frames to facilitate avoiding appearance of a seam between said adjoining first and second areas of said subsequent frame due to said separate encoding thereof by said first encoder and said second encoder.

29. The encode system of claim 22, wherein said first encoder and said second encoder comprise two encoders of a plurality of encoders of said encode system employed in processing said sequence of video frames, and wherein said plurality of encoders are coupled to encode a plurality of areas of said frame, said plurality of areas being disposed in a tile configuration, with said first area and said second area comprising diagonally adjoining areas within said frame, and wherein said first encoder being adapted to make available to said second encoder reference data comprises means for transferring said reference data of said first encoder to said second encoder through a third encoder of said plurality of encoders, said third encoder encoding a third area of said frame, said third area having a first common boundary with said first area and a second common boundary with said second area.

30. The encode system of claim 29, wherein said means for transferring of said reference data from said first encoder to said second encoder through said third encoder comprises means for first transferring data from an active region of said first encoder to said third encoder, and means for transferring thereafter data from an inactive region of said third encoder to said second encoder, wherein said reference data transferred from said first encoder to said second encoder comprises at least a portion of said data from said inactive region transferred from said third encoder to said second encoder.

31. An encode system for processing a sequence of video frames comprising:

a video divider for dividing a frame of said sequence of frames into multiple active areas;

multiple encoders coupled in parallel to encode said multiple active areas of said frame of said sequence of video frames, each active area of said frame having at least one adjoining active area within said frame so that each encoder has at least one adjoining encoder; and

wherein said multiple encoders are adapted to make reference data available from each encoder to its at least one adjoining encoder for use in encoding a subsequent frame of said sequence of video frames to facilitate avoiding appearance of seams between adjoining areas of said subsequent frame of said sequence of video frames due to separate encoding thereof by said multiple encoders.

32. The encode system of claim 31, wherein said video divider comprises means for dividing each frame of said sequence of frames into said multiple active areas.

33. The encode system of claim 31, wherein said reference data made available to each adjoining encoder comprises an expanded reference area for use in motion estimation by said adjoining encoder when said subsequent frame is being encoded.

34. The encode system of claim 31, wherein said multiple encoders being adapted to make reference data available comprises multiple encoders adapted for making an encode and decode section of said area encoded by each encoder available to said adjoining encoder, wherein said encoded and decoded section is along a common boundary between said encoder and said adjoining encoder.

35. The encode system of claim 31, further comprising means for transferring said reference data made available from each encoder to said adjoining encoder for use in encoding said subsequent frame of said sequence of video frames.

36. The encode system of claim 31, wherein each area comprises a rectangular area within said frame, and wherein said multiple encoders adapted to make reference data available comprises for each encoder means for determining whether there is a connection with an adjoining encoder, and if so, then for making a predetermined section of reference data available to said adjoining encoder.

37. The encode system of claim 36, wherein said means for determining whether said encoder has an adjoining encoder comprises means for evaluating its associated area for a top connection, bottom connection, right connection or left connection with at least one adjoining area.

38. An article of manufacture comprising:

a computer program product comprising a computer usable medium having computer readable program code means therein for use in processing a sequence of video frames, said computer readable program code means in said computer program product comprising;

(i) computer readable program code means for causing a computer to effect employing a first encoder to encode a first area of a frame of said sequence of video frames and a second encoder to encode a second area of said frame, wherein said first area and said second area comprise adjoining areas within said frame; and

(ii) computer readable program code means for causing a computer to effect making reference data of said first encoder available to said second encoder, said reference data arising from encoding said first area of said frame and being made available to said second encoder for use in encoding said second area of a subsequent frame of said sequence of video frames to facilitate avoiding appearance of a seam between said adjoining first and second areas of said subsequent frame of the sequence of video frames due to separate encoding thereof by said first encoder and said second encoder.

39. An article of manufacture comprising:

a computer program product comprising a computer usable medium having computer readable program code means therein for use in processing a sequence of video frames, said computer readable program code means in said computer program product comprising;

(i) computer readable program code means for causing a computer to effect employing multiple encoders to encode multiple areas of a frame of said sequence of video frames, each area of said frame having at least one adjoining area within said frame so that each encoder has at least one adjoining encoder; and

(ii) computer readable program code means for causing a computer to effect making reference data available from each encoder to an adjoining encoder for use in encoding a subsequent frame of said sequence of video frames to facilitate avoiding appearance of seams between adjoining areas of said subsequent frame of said sequence of video frames due to separate encoding thereof by said multiple encoders.