Interpolation of video compression frames

• US 20030112871A1
• Filed: 07/11/2001
• Published: 06/19/2003
• Est. Priority Date: 07/11/2001
• Status: Active Grant

• Current Assignee: Tagivan II LLC (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Incorporated, Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Demos, Gary A.
• Granted Patent: US 6,816,552 B2
• Time in Patent Office: Days
• US Class Current: 375/240.15
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop

Abstract

A method, system, and computer programs for improving the image quality of one or more bi-directionally predicted intermediate frames in a video image compression system, where each frame comprises a plurality of pixels. In one aspect, the invention includes determining the value of each pixel of each bi-directionally predicted intermediate frame as a weighted proportion of corresponding pixel values in non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames. In one embodiment, the weighted proportion is a function of the distance between the bracketing non-bidirectionally predicted frames. In another embodiment, the weighted proportion is a blended function of the distance between the bracketing non-bidirectionally predicted frames and an equal average of the bracketing non-bidirectionally predicted frames. In another aspect of the invention, interpolation of pixel values is performed on representations in a linear space, or in other optimized non-linear spaces differing from an original non-linear representation.

Claims

1. A method for improving the image quality of a sequence of two or more bi-directionally predicted intermediate frames in a video image compression system, each frame comprising a plurality of pixels, including determining the value of each pixel of each bi-directionally predicted intermediate frame as a weighted proportion of corresponding pixel values in non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames.

2. The method of claim 1, wherein the weighted proportion is a function of the distance between the bracketing non-bidirectionally predicted frames.

3. The method of claim 1, wherein the weighted proportion is a blended function of the distance between the bracketing non-bidirectionally predicted frames and an equal average of the bracketing non-bidirectionally predicted frames.

4. The method of claim 3, wherein the blended function for each bi-directionally predicted intermediate frame weighted proportion is:

weight=F·

(frame distance proportional weight)+(1−

F)·

(equal average weight)

where F is a selected blend factor ranging from 0 to 1, “

frame distance proportional weight”

is a function of the distance between the bracketing non-bidirectionally predicted frames, and “

equal average weight”

is an equal average of the bracketing non-bidirectionally predicted frames.

5. The method of claim 4, further including optimizing the blend factor F for a selected region of at least one frame.

6. The method of claim 4, further including optimizing the blend factor F for a selected range of frames.

7. The method of claim 4, further including optimizing the blend factor F as a function of scene characteristics within at least one frame.

8. The method of claim 4, wherein the blend factor F varies as a function of the position of a bi-directionally predicted intermediate frame with respect to the bracketing non-bidirectionally predicted frames.

9. The method of claim 4, further including:

(a) selecting at least two candidate blend factors F;

(b) applying the candidate blend factors F in determining pixel values for at least one bi-directionally predicted intermediate frame to determine corresponding evaluation sets of frames;

(c) coding each evaluation set of frames;

(d) evaluating each such coded evaluation set of frames with respect to at least one compression characteristic;

(e) selecting one such evaluation set of frames having a desired compression characteristic; and

(f) selecting, as a final blend factor F, the candidate blend factor F corresponding to the selected evaluation set of frames.

10. The method of claim 9, wherein the compression characteristic is the number of bits generated during coding.

11. The method of claim 9, wherein the compression characteristic is a signal to noise ratio measurement.

12. A method for improving the image quality of a sequence of one or more bi-directionally predicted intermediate frames in a video image compression system, each frame comprising a plurality of pixels having luminance and chroma characteristics in a first non-linear representation, including:

(a) converting at least one of the luminance and chroma characteristics of such plurality of pixels to a second representation;

(b) determining the value of each pixel of each bi-directionally predicted intermediate frame in such second representation as a weighted proportion of corresponding pixel values in non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames; and

(c) converting such plurality of pixels from the second representation back to the first representation.

13. The method of claim 12, wherein the second representation is a linear representation.

14. The method of claim 12, wherein the second representation is a non-linear representation differing from the first non-linear representation.

16. The computer program of claim 12, wherein the weighted proportion is a function of the distance between the bracketing non-bidirectionally predicted frames.

17. The computer program of claim 12, wherein the weighted proportion is a blended function of the distance between the bracketing non-bidirectionally predicted frames and an equal average of the bracketing non-bidirectionally predicted frames.

18. The computer program of claim 17, wherein the blended function for each bi-directionally predicted intermediate frame weighted proportion is:

weight=F·

(frame distance proportional weight)+(1−

F)·

(equal average weight)

where F is a selected blend factor ranging from 0 to 1, “

frame distance proportional weight”

is a function of the distance between the bracketing non-bidirectionally predicted frames, and “

equal average weight”

is an equal average of the bracketing non-bidirectionally predicted frames.

19. The computer program of claim 18, further including instructions for causing a computer to optimize the blend factor F for a selected region of at least one frame.

20. The computer program of claim 18, further including instructions for causing a computer to optimize the blend factor F for a selected range of frames.

21. The computer program of claim 18, further including instructions for causing a computer to optimize the blend factor F as a function of scene characteristics within at least one frame.

22. The computer program of claim 18, wherein the blend factor F varies as a function of the position of a bidirectionally predicted intermediate frame with respect to the bracketing non-bidirectionally predicted frames.

23. The computer program of claim 18, further including instructions for causing a computer to:

(a) select at least two candidate blend factors F;

(b) apply the candidate blend factors F in determining pixel values for at least one bi-directionally predicted intermediate frame to determine corresponding evaluation sets of frames;

(c) code each evaluation set of frames;

(d) evaluate each such coded evaluation set of frames with respect to at least one compression characteristic;

(e) select one such evaluation set of frames having a desired compression characteristic; and

(f) select, as a final blend factor F, the candidate blend factor F corresponding to the selected evaluation set of frames.

24. The computer program of claim 23, wherein the compression characteristic is the number of bits generated during coding.

25. The computer program of claim 23, wherein the compression characteristic is a signal to noise ratio measurement.

27. The computer program of claim 12, wherein the second representation is a linear representation.

28. The computer program of claim 12, wherein the second representation is a non-linear representation differing from the first non-linear representation.

15. A computer program, stored on a computer-readable medium, for improving the image quality of a sequence of two or more bi-directionally predicted intermediate frames in a video image compression system, each frame comprising a plurality of pixels, the computer program comprising instructions for causing a computer to determine the value of each pixel of each bi-directionally predicted intermediate frame as a weighted proportion of corresponding pixel values in non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames.

26. A computer program, stored on a computer-readable medium, for improving the image quality of a sequence of one or more bi-directionally predicted intermediate frames in a video image compression system, each frame comprising a plurality of pixels having luminance and chroma characteristics in a first non-linear representation, the computer program comprising instructions for causing a computer to:

(a) convert at least one of the luminance and chroma characteristics of such plurality of pixels to a second representation;

(b) determine the value of each pixel of each bidirectionally predicted intermediate frame in such second representation as a weighted proportion of corresponding pixel values in non-bidirectionally predicted frames bracketing the sequence of bidirectionally predicted intermediate frames; and

(c) convert such plurality of pixels from the second representation back to the first representation.

29. A system for improving the image quality of a sequence of two or more bi-directionally predicted intermediate frames in a video image compression system, each frame comprising a plurality of pixels, including:

(a) means for inputting at least two non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames; and

(b) means for determining the value of each pixel of each bi-directionally predicted intermediate frame as a weighted proportion of corresponding pixel values in the non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames.

30. The system of claim 29, wherein the weighted proportion is a function of the distance between the bracketing non-bidirectionally predicted frames.

31. The system of claim 29, wherein the weighted proportion is a blended function of the distance between the bracketing non-bidirectionally predicted frames and an equal average of the bracketing non-bidirectionally predicted frames.

32. The system of claim 31, wherein the blended function for each bi-directionally predicted intermediate frame weighted proportion is:

weight=F·

(frame distance proportional weight)+(1−

F)·

(equal average weight)

where F is a selected blend factor ranging from 0 to 1, “

frame distance proportional weight”

is a function of the distance between the bracketing non-bidirectionally predicted frames, and “

equal average weight”

is an equal average of the bracketing non-bidirectionally predicted frames.

33. The system of claim 32, further including means for optimizing the blend factor F for a selected region of at least one frame.

34. The system of claim 32, further including means for optimizing the blend factor F for a selected range of frames.

35. The system of claim 32, further including means for optimizing the blend factor F as a function of scene characteristics within at least one frame.

36. The system of claim 32, wherein the blend factor F varies as a function of the position of a bi-directionally predicted intermediate frame with respect to the bracketing non-bidirectionally predicted frames.

37. The system of claim 32, further including:

(a) means for selecting at least two candidate blend factors F;

(b) means for applying the candidate blend factors F in determining pixel values for at least one bidirectionally predicted intermediate frame to determine corresponding evaluation sets of frames;

(c) means for coding each evaluation set of frames;

(d) means for evaluating each such coded evaluation set of frames with respect to at least one compression characteristic;

(e) means for selecting one such evaluation set of frames having a desired compression characteristic; and

(f) means for selecting, as a final blend factor F, the candidate blend factor F corresponding to the selected evaluation set of frames.

38. The system of claim 37, wherein the compression characteristic is the number of bits generated during coding.

39. The system of claim 37, wherein the compression characteristic is a signal to noise ratio measurement.

40. A system for improving the image quality of a sequence of one or more bi-directionally predicted intermediate frames in a video image compression system, each frame comprising a plurality of pixels having luminance and chroma characteristics in a first non-linear representation, including:

(a) means for converting at least one of the luminance and chroma characteristics of such plurality of pixels to a second representation;

(b) means for determining the value of each pixel of each bi-directionally predicted intermediate frame in such second representation as a weighted proportion of corresponding pixel values in non-bidirectionally predicted frames bracketing the sequence of bi-directionally predicted intermediate frames; and

(c) means for converting such plurality of pixels from the second representation back to the first representation.

41. The system of claim 40, wherein the second representation is a linear representation.

42. The system of claim 40, wherein the second representation is a non-linear representation differing from the first non-linear representation.