Method and system for video encoding using a variable number of B frames

• US 20050147167A1
• Filed: 12/24/2003
• Published: 07/07/2005
• Est. Priority Date: 12/24/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Haskell, Barin Geoffry, Dumitras, Adriana
• Granted Patent: US 7,889,792 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression

Abstract

An effective method for dynamically selecting the number of I, P and B frames during video coding is proposed. Short-term look-ahead analysis of a video sequence yields a variable number of B frames to be coded between any two stored pictures. The first picture of a group of frames (GOF) may be coded as a B picture. Motion speed is calculated for each picture of the GOF with respect to the first picture of the GOF. Subject to exceptions, as long as the subsequent pictures exhibit motion speeds that are similar and motion vector displacements that are co-linear with those of the first picture in the GOF, they may be coded as B pictures. When a picture is encountered having a motion speed that is not the same as that of the first picture in the GOF, the picture may be coded as a P picture. In some embodiments, a sequence of B pictures that terminates in a P picture may be called a “group of frames” (GOF).

Claims

1. A method of assigning frame types for coding of pictures in a video sequence, comprising:

determining movement between image data of a reference picture and each of a plurality of pictures in temporal order,

comparing the movement of a first picture in the plurality, temporally closest to the reference picture, to movement of other pictures therein,

if the plurality of pictures exhibits consistent motion speed, assigning the pictures therein as B pictures.

2. The method of claim 1, further comprising assigning a picture in the sequence that does not exhibit consistent motion speed with the first picture as a P picture.

3. The method of claim 1, further comprising:

determining whether a scene change occurs in the plurality of pictures, and, if so;

coding a first picture temporally after the scene change as a P picture, and

coding all pictures in the plurality that occur from the first picture to a picture immediately prior to the scene change as B picture long as they exhibit consistent motion speed.

4. The method of claim 3, further comprising coding the picture before the scene cut as a P picture at full quality or low quality.

5. The method of claim 1, further comprising coding a picture as a P picture when the picture, if coded as a B picture, would cause a number of consecutive B pictures to be larger than a predetermined maximum.

6. The method of claim 1, further comprising coding select pictures as I pictures pursuant to a random access policy.

7. The method of claim 6, further comprising coding a picture temporally adjacent to and before the I picture as a P picture.

8. The method of claim 7, further comprising coding a picture temporally adjacent to and before the I picture as a P picture using full quality of low quality.

9. The method of claim 1, wherein the determining comprises computing motion vectors between pixelblocks of the plurality of pictures and of the reference picture and the comparing comprises comparing motion vectors of the respective pictures.

10. A video coding method, comprising, from a sequence of video data:

determining a motion speed between a first picture and a reference picture;

for each picture following the first picture, until a termination condition is met;

determining a motion speed for the respective picture,

comparing the motion speed of a respective picture with the motion speed of the first picture, and

coding the respective picture as a B picture if the motion speeds are consistent with each other; and

when the termination is met, coding a picture as a P picture.

11. The video coding method of claim 10, wherein the termination condition is met when motion speed of a picture is not consistent with the motion speed of the first picture.

12. The video coding method of claim 10, wherein the termination condition is met when a scene change is detected.

13. The video coding method of claim 12, further comprising coding a frame immediately prior to the scene change in display order as a P frame and coding a frame immediately after the scene change in display order as an I frame.

14. The video coding method of claim 12, further comprising coding frames immediately prior to the scene change in display order and immediately after the scene change P frames.

15. The video coding method of claim 12, further comprising coding frames immediately adjacent to the scene change in display order as B frames within a group of frames.

16. The video coding method of claim 12, further comprising detecting a scene change by comparing a correlation coefficient C to a predetermined threshold, the correlation coefficient given by:

17. The video coding method of claim 10,

wherein the termination condition is met when a random access policy dictates that a picture be coded as an I picture, and

a picture temporally adjacent to and before the I picture is coded as a P picture.

18. Apparatus, comprising:

a memory to store pictures of a video sequence,

a video coder coupled to the memory, to code each stored picture as one of an I picture, a P picture and a B picture,

a motion vector generator coupled to the memory to generate motion vectors for a plurality of stored pictures, the motion vectors measuring motion between a respective picture and a stored reference picture,

a colinearity detector having an input coupled to the output of the motion vector generator, and

a picture type decision unit to assign pictures having generally consistent motion speeds for B picture coding and to assign a picture that does not exhibit consistent motion speed for coding as a P picture.

19. The apparatus of claim 18, wherein the picture type decision unit further controls the video coder to cause it to code the B and P pictures.

20. The apparatus of claim 18, wherein the video coder codes B and P frames using motion vectors generated by the motion vector generator.

21. The apparatus of claim 18, further comprising a scene change detector coupled to the memory and to the picture type decision unit, to identify stored frames that follow a scene change.

22. The apparatus of claim 21, wherein the picture type decision unit assigns a frame immediately prior to the scene change in display order for coding as a P frame and assigns a frame immediately after the scene change in display order as an I frame.

23. The apparatus of claim 21, wherein the picture type decision unit assigns a frames immediately prior to the scene change in display order and immediately after the scene change for coding as P frames.

24. The apparatus of claim 21, wherein the picture type decision unit assigns frames immediately adjacent to the scene change in display order as B frames within a group of frames.

25. A coded video signal created from a method comprising:

from a sequence of video data, determining a motion speed between a first picture and a reference picture;

for each picture following the first picture in the sequence, until a termination condition is met;

determining a motion speed for the respective picture,

coding the respective picture as a B picture if its motion speed is consistent with the motion speed of the first picture;

when the termination is met, coding a picture as a P picture; and

outputting coded data of the B pictures and the P picture to a channel.

26. The coded video signal of claim 25, wherein the termination condition is met when motion speeds of a following picture is not constant with respect to the motion speed of the first picture.

27. The coded video signal of claim 25, wherein the termination condition is met when a scene change is detected.

28. The coded video signal of claim 25,

wherein the termination condition is met when a random access policy dictates that a picture be coded as an I picture, and

a picture temporally adjacent to and before the I picture is coded as a P picture.

29. A video coding assignment method, comprising, for each of a plurality of pictures following a reference picture:

adding a first picture that follows the reference picture in display order to a group of frames,

determining a motion speed of the first picture with respect to the reference picture,

iteratively, for pictures subsequent to the first picture in display order;

adding the subsequent picture to the group of frames,

determining a motion speed of the subsequent picture with respect to the reference picture,

if the motion speed of the subsequent picture is consistent with the motion speed of the first picture, performing a next iteration, and

if not, coding the last picture of the group of frames as a P picture and coding all other pictures in the group of frames as a B picture.

30. The video coding assignment method of claim 29, wherein the motion speed determinations are based on a pixelblock-by-pixelblock comparison of motion vectors between the respective pictures.

31. The video coding assignment method of claim 30, wherein the motion speed determinations are:

32. The video coding assignment method of claim 29, wherein colinearity is based on: