Method for estimating the type of the group of picture structure of a plurality of video frames in a video stream

• US 9,241,156 B2
• Filed: 07/26/2011
• Issued: 01/19/2016
• Est. Priority Date: 07/30/2010
• Status: Active Grant

• Current Assignee: Deutsche Telekom AG
• Original Assignee: Deutsche Telekom AG
• Inventors: Argyropoulos, Savvas, Feiten, Bernhard, Garcia, Marie-Neige, List, Peter, Raake, Alexander
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): FINDLEY, CHRISTOPHER G
• Application Number:
  US13/812,871
• Publication Number:
  US 20130219443A1
• Time in Patent Office: 1,638 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 17/00 Diagnosis, testing or measuring for television systems or their details
• CPC Class Codes: H04N 17/004 for digital television systems

Abstract

A method for estimating the type of the GoP structure of a plurality of video frames in a video stream by estimating their frame types includes: capturing frame sizes in bytes of every video frame subsequent to an initial I-frame to obtain an array of frame sizes by exploiting features of a transport layer carrying the video stream; converting, after a number of frames, the array of frame sizes into an array of zeros and ones; matching the binarized array of frame sizes to a number of predefined short basic binary patterns, said predefined binary patterns depicting all GoP structures to be considered; converting the result of said matching to form a single score value; and determining the particular pattern of the number of predefined patterns of binaries having the best score value, according to a predefined metric.

Claims

1. A method for estimating the type of the Group of Picture (GoP) structure of a plurality of video frames in a video stream by estimating their frame types, comprising the steps of:

a) capturing frame sizes in bytes of every video frame subsequent to an initial Intra-frame (I-frame) to obtain an array of frame sizes by exploiting features of a transport layer carrying the video stream;

b) converting, after a number of frames, the array of frame sizes obtained in step a) into an array of zeros and ones, where zero stands for small frame sizes, as assumed for bidirectional frames (B-frames), and one stands for large frame sizes, as assumed for predicted frames (P-frames);

c) matching the binarized array of frame sizes obtained in step b), which contains a specific repeated short basic binary pattern that is characteristic to the GoP structure to be analyzed, to a number of predefined short basic binary patterns, said predefined binary patterns depicting all GoP structures to be considered;

d) converting the result of said matching in step c) to form a single score value; and

e) determining the particular pattern of the number of predefined patterns of binaries having the best score value, according to a predefined metric.

2. The method of claim 1, wherein in step a) the boundaries of video frames in the bitstream are detected by either observing the changes in certain timestamps of the transport layer, or by certain bits in the header structures that are used to signal new frames and decide Intra-frames from other frames.

3. The method of claim 1, wherein in step b) the number of frames comprises all frames until the next I-frame.

4. The method of claim 1, wherein in step b) the conversion of the frame sizes to binaries comprises the usage of a reference value, that is dependant on constant or fluctuating features of the video sequence to be analyzed, or a dynamically adaptive reference value.

5. The method of claim 4, wherein the dynamically adaptive reference value is dependent on at least one of the values of previously captured frame sizes or other dynamic features of the video sequence to be analyzed.

6. The method of claim 1, wherein said predefined short basic binary patterns depict possible GoP structures for different numbers of consecutive B-frames in frame based coding, field based coding, open GoP coding, or closed GoP coding.

7. The method of claim 1, wherein for the matching in step c) the predefined short basic binary patterns are repeated with a possible truncation of the last basic pattern until the series of repeated basic patterns has the same length as the series of binarized frame sizes obtained from the array of frame sizes.

8. The method of claim 1, wherein in step c) the matching is performed by an element-wise exclusive NOR or by an element-wise exclusive OR operation, wherein in step d) the results of every exclusive NOR or exclusive OR is added up to form the score value, and wherein in step e) the score value metric corresponds to the maximum or minimum value, respectively.

9. The method of claim 1, wherein in step c) for the pattern having the best score value the associated GoP-pattern, field/frame status and open GoP/closed GoP status are selected as estimation result for the GoP-structure of the current series of analyzed frames.

10. The method of claim 1, wherein for GoP-structure estimation of subsequent GoPs either new arrays of frame sizes are captured for each of the subsequent GoPs, or any kind of average of the elements of current and previous arrays of frame sizes arc being used.

11. The method of claim 1, wherein after the estimation of the GoP-pattern a further frame type classification, is performed, wherein every subset of consecutive B-frames between two P-frames is further examined as to whether the relation of frame sizes according to the output of step a) for those elements in said subsets that may contain reference B-frames to those elements that may only contain non-reference B-frames is larger than a threshold value, which is sufficiently larger than 1,as reference B-frames are assumed to be on average larger than non-reference B-frames, and wherein, if this is the case for a statistically significant number of said subsets of B-frames in the array of frame sizes, said frames are then assumed to be reference B-frames and the estimated GoP-structure is assumed to be a GoP-structure of a so called hierarchical coding sequence.

12. The method of claim 8, further comprising the step of using the estimated GoP structure as an estimate for the following GoP-structures, and using this estimated GoP-structure as aid to detect video frame losses due to packet losses, or to label lost or present frames with their specific frame-type.

13. A method for estimating the type of the Group of Picture (GoP) structures of a plurality of video frames in a video stream by estimating their frame types, comprising the steps of:

a) capturing frame sizes in bytes of every video frame n subsequent to an initial I-frame by exploiting features of the transport layer carrying the video stream, where the size of every frame n is added to a specific element of each array, Array[BP], of a multiple of arrays, where each array of these multiple of arrays has a different size, ArraySize[BP], equal to the length of the basic frame type pattern, BP, of each considered GoP structure, where the specific element in which the frame size of frame n is added up is the (n % ArraySize[BP])'"'"'th element of each array, Array[BP], and where the number of frame sizes added up in every element of every array is noted, which is depicted by the following symbolic code assuming Array and ArrayCount have meaningful content;

14. The method of claim 13, wherein the number of frames n, comprises all frames until the next Intra-frame.

15. The method of claim 13, wherein in step c) for every array two relation values are produced as described below to be able to decide between open GoP and closed GoP coding:

i) if the examined video stream comprises of open GoP coding, the assumed added up P-frames of basic pattern BP that is represented by Array[BP], would be located in the last element of said array[BP] and all other elements of said arrays would be occupied by added up B-frames so that the relation R_O for open GoP coding streams is calculated as the last array element divided by the average of all other array elements, with the equation;

R_O[BP]=Array[BP][ArraySize[BP]−

1]/(Σ

Array[BP][i]/[ArraySize[BP]−

1])

with i=0 . . . ArraySize[BP]−

2;

or

ii) if the examined video stream comprises of closed GoP coding, the assumed added up P-frames of the basic pattern BP that is represented by Array[BP], would he located in the first element of said array[BP] and all other elements of said arrays would be occupied by added up B-frames so that the relation R_C for closed GoP coding streams is calculated as the first array element divided by the average of all other array elements, with the equation;

R_C[BP]=Array[BP][0]/(Σ

Array[BP][i]/[ArraySize[BP]−

1])

with i=1 . . . ArraySize[BP]−

1.

16. The method of claim 15, where the largest relation in step d) is found by choosing the maximum of all values R_C and R_O as calculated in claim 14, where, if one of the values R_C comprises the maximum, the sequence can be considered a Closed-GoP coded sequence, and where if one of the values R_O comprises the maximum, the sequence can be considered an Open-GoP coded sequence.

17. The method of claim 13, wherein the GoP-structure associated with the determined array according to step d) is chosen to be the valid estimation of the GoP-pattern, field/frame status and open GoP/closed GoP status of the currently analyzed series of frames.

18. The method of claim 13, wherein after the estimation of the GoP-pattern a further frame type classification is performed wherein the subset of B-frames in the array representing the estimated GoP-structure according to step d) is examined as to whether the relation of array elements that may contain reference B-frames (first element in Open-GoP coding, second element in Closed-GoP coding) to those elements that only may contain non-reference B-frames is larger than a threshold value, which is sufficiently larger than 1, as reference B-frames are assumed to be on average larger than non-reference B-frames, and wherein, if this is the case, said frames are then assumed to be reference B-frames and the estimated GoP-structure is assumed to be a GoP-structure of a so called hierarchical coding sequence.

19. The method of claim 17, further comprising the step of using the estimated GoP structure as an estimate for the following GoP-structures, and using this estimated GoP-structure as aid to detect video frame losses due to packet losses, or to label lost or present frames with their specific frame-type.