Analyzer and methods for detecting and processing video data types in a video data stream

• US 5,828,786 A
• Filed: 12/02/1993
• Issued: 10/27/1998
• Est. Priority Date: 12/02/1993
• Status: Expired due to Term

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Rao, R. Padmanabha, Chin, Amanda L.
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US08/162,511
• Time in Patent Office: 1,790 Days
• Field of Search: 382/232, 382/236, 382/239, 348/439, 348/441, 348/443, 348/384, 348/97, 348/700, 348/412, 348/415, 358/430
• US Class Current: 382/236
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 7/01 Conversion of standards , e.g. involving analogue television standards or digital television standards processed at pixel level
• CPC Class Codes: H04N 7/0112 one of the standards corresponding to a cinematograph film standard
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229

Abstract

A video data stream analyzer modifies an input digital video signal so that the resulting output digital signal can be optimally compressed by a digital video encoder. The video data stream analyzer eliminates redundancy in the input video signal, and reorganizes the input video signal so that the spatial and temporal redundancy is increased. In addition, the video data stream analyzer generates side channel information that is supplied to the video encoder. The side channel information tells the video encoder whether vertical frame-based filtering or vertical field-based filtering is preferable. Additional side channel information specifies the order and duration of the display of the fields after decoding and this information preferably is encoded with the video signal. The video data stream analyzer provides scan detection of the incoming video digital data, and automatically and reliably detects scene cuts, repeated fields, and mixed-field frames in the incoming digital video data in real time independent of the video source. The video data stream analyzer modifies the input video data stream by dropping repeated fields and replacing a frame with a scene cut with a frame having identical fields for video, cartoon, telecine video sources as well as arbitrary combinations of these video sources.

Claims

1. A method for improving encoding efficiency of a video data stream, said method comprising:

analyzing each field in said video data stream to detect repeated fields wherein said video data stream includes combinations of a video sequence, a cartoon sequence, and a telecine sequence and further wherein repeated fields in said cartoon sequence are detected and repeated fields in said telecine sequence are detected; and

dropping each detected repeated field from said video data stream to generate a modified video data stream thereby eliminating encoding of redundant information and so improving encoding efficiency.

2. The method of claim 1 wherein said analyzing each field and said dropping each detected repeated field are performed in real time.

3. The method as in claim 1 further comprising:

generating a display code for each frame in said modified video data stream wherein said display code indicates a display combination for fields in said modified video data stream to recreate said video data stream.

4. The method of claim 1 wherein said video data stream includes a plurality of frames and each frame includes a field of a first parity and a field of a second parity, and further wherein said analyzing each field further comprises:

comparing a field activity of a field with a first threshold; and

identifying said field as a repeated field if said field activity of said field is less than said first threshold and said field has said second parity.

5. The method of claim 4 further comprising:

comparing a strip activity of each strip of said field with a second threshold; and

identifying said field as a repeated field if a predetermined number of said strip activities of said field are less than said second threshold and said field has said second parity.

6. The method of claim 5 further comprising:

repeating said field activity and strip activity comparisons for a second field of said second parity if (i) said field has said first parity;

(ii) said field activity of said field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said field are less than said second threshold; and

identifying said field as a repeated field if (i) said field activity of said second field is greater than said first threshold;

or (ii) a predetermined number of said strip activities of said second field are greater than said second threshold.

7. The method of claim 5 further comprising:

repeating said field activity and strip activity comparisons for a second field of said second parity if (i) said field has said first parity;

(ii) said field activity of said field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said field are less than said threshold;

comparing a measure of field activity of said field with a third threshold if (i) said field activity of said second field is less than said first threshold; and

(ii) a predetermined number of said strip activities of said second field are less than said second threshold; and

identifying said field as a repeated field if said measure of field activity of said field is less than said third threshold.

8. The method of claim 5 wherein said strip activity is a sum, for all pixels in a strip, of an absolute difference between a pixel in a first field of one parity and a pixel of a second field of said one parity.

9. The method of claim 8 wherein said field activity is a sum of each strip activity of a field.

10. The method of claim 4 further comprising

initializing said first threshold to four times the number of pixels in said field upon power-up.

11. The method of claim 5 further comprising

initializing said first threshold to six times the number of pixels in a strip of said field upon detection of a scene cut in said video data stream.

12. The method of any one of claims 4, 5, 6, or 7 further comprising:

updating said first threshold upon identifying said field as a repeated field.

13. The method of claim 12 wherein said updating said first threshold upon identifying said field as a repeated field further comprises:

comparing said field activity with a stored field activity;

replacing said stored field activity with said field activity if said field activity is greater than said stored field activity; and

defining said first threshold in terms of said stored field activity.

14. The method of claim 12 further comprising:

updating said second threshold after updating said first threshold.

15. The method of claim 13 wherein said updating said second threshold further comprises:

defining said second threshold in terms of said first threshold.

16. The method of claim 1 wherein said analyzing each field further comprises:

measuring a strip activity between a strip in a first field of a first parity and a strip of a second field of said first parity in said video data steam for a plurality of strips thereby obtaining a plurality of strip activities for said first field;

generating a field activity for said first field using said plurality of strip activities;

comparing said field activity to a first threshold;

comparing each of said plurality of strip activities to a second threshold; and

identifying said first field as a repeated field if (i) said first parity is an even parity;

(ii) said field activity for said first field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said first field are less than said second threshold.

17. The method of claim 16 further comprising:

performing the following operations if (i) said first parity is an odd parity;

(ii) said field activity for said first field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said first field are less than said second threshold;

measuring a strip activity between a strip in a first field of an even parity and a strip of a second field of said even parity in said video data stream for a plurality of strips to obtain a plurality of strip activities for said first field of said even parity;

generating a field activity for said first field of said even parity using said plurality of strip activities for said first field of said even parity;

comparing said field activity for said first field of said even parity to said first threshold;

comparing each of said plurality of strip activities for said first field of said even parity to said second threshold; and

identifying said first field of said first parity as a repeated field if (i) said field activity for said first field of said even parity is greater than said first threshold;

or (ii) a predetermined number of said strip activities for said first field of said even parity are greater than said second threshold.

18. The method of claim 16 further comprising:

performing the following operations if (i) said first parity is an odd parity;

(ii) said field activity for said first field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said first field are less than said second threshold;

measuring a strip activity between a strip in a first field of an even parity and a strip of a second field of said even parity in said video data stream for a plurality of strips thereby obtaining a plurality of strip activities for said first field of said even parity;

generating a field activity for said first field of said even parity using said plurality of strip activities for said first field of said even parity;

comparing said field activity for said first field of said even parity to said first threshold;

comparing each of said plurality of strip activities for said first field of said even parity to said second threshold

comparing a measure of field activity of said field with a third threshold if (i) said field activity of said first field of even parity is less than said first threshold; and

(ii) a predetermined number of said strip activities of said first field of even parity are less than said second threshold; and

identifying said field as a repeated field if said measure of field activity of said field is less than a third threshold.

19. The method of any one claims 16, 17, or 18 wherein said strip activity is a sum, for all pixels in a strip, of an absolute difference between a pixel in a first field of one parity and a pixel of a second field of said one parity.

20. The method of any one of claims 16, 17, or 18 further comprising:

updating said first threshold upon identifying said first field as a repeated field.

21. The method of claim 20 wherein said updating said first threshold further comprising:

comparing said field activity of said first field with a stored field activity;

replacing said stored field activity with said field activity of said first field if said field activity of said first field is greater than said stored field activity; and

defining said first threshold in terms of said stored field activity.

22. The method of claim 21 further comprising:

updating said second threshold after updating said first threshold.

23. The method of claim 22 wherein said updating said second threshold further comprises:

defining said second threshold in terms of said first threshold.

24. The method of claim 1 wherein said video data stream includes a plurality of frames and each frame includes a field of a first parity and a field of a second parity, and further wherein said analyzing each field further comprises:

determining if a primary criterion is true for a first field in a frame; and

identifying the first field as an unrepeated field upon determining that the primary criterion is not true.

25. The method of claim 24 further comprising:

determining if the primary criterion is true for the other field in said frame upon determining that the primary criterion is true for the first field.

26. The method of claim 25 further comprising:

identifying the first field as a repeated field upon determining that the primary criterion is not true for the other field in said frame.

27. The method of claim 26 further comprising:

determining if a secondary criterion is true for the first field upon determining that the primary criterion is true for the other field in said frame.

28. The method of claim 27 further comprising:

identifying the first field as an unrepeated field upon determining that the secondary criterion is not true.

29. The method of claim 27 further comprising:

determining if a secondary criterion is true for the other field in said frame upon determining that the secondary criterion is true for the first field.

30. The method of claim 29 further comprising:

identifying the first field as a repeated field upon determining that the secondary criterion is not true for the other field in said frame.

31. The method of claim 24 wherein the primary criterion is true if a predetermined number of strip activities for the first field do not exceed a first threshold, and if a field activity for the first field does not exceed a second threshold.

32. The method of claim 31 wherein determining if the primary criterion is true further comprises:

measuring a strip activity between a strip in a first field of a given parity and a strip of a second field of said given parity in said video data stream for a plurality of strips thereby obtaining a plurality of strip activities for said first field of said first parity;

generating a field activity for said first field of said given parity using said plurality of strip activities for said first field of said given parity.

33. The method of claim 28 wherein the secondary criterion is true if a measure of a field activity for the first field does not exceed a third threshold.

34. The method of claim 1 wherein said video data stream includes a plurality of frames and each frame includes a field of a first parity and a field of a second parity, and further wherein said analyzing each field further comprises:

measuring a differential parameter using pixels in a line of a first field of a given parity and pixels in a line of a second field of said given parity; and

comparing said differential parameter with a threshold to detect a repeated field.

35. The method of claim 34 wherein said measuring a differential parameter for pixels in a line of a first field of a given parity and pixels in a line of a second field of said given parity, and said comparing said differential parameter with a threshold are performed in real time.

36. A method for improving encoding efficiency of a video data stream including a plurality of frames and each frame includes a field of a first parity and a field of a second parity, said method comprising:

analyzing each field in said video data stream to detect repeated fields wherein said analyzing process further comprises;

comparing a field activity of a field with a first threshold; and

identifying said field as a repeated field if said field activity of said field is less than said first threshold and said field has said second parity;

comparing a strip activity of each strip of said field with a second threshold; and

identifying said field as a repeated field if a predetermined number of said strip activities of said field are less than said second threshold and sail field has said second parity;

repeating said field activity and strip activity comparisons for a second field of said second parity if (i) said field has said first parity;

(ii) said field activity of said field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said field are less than said second threshold;

identifying said field as a repeated field if (i) said field activity of said second field is greater than said first threshold;

or (ii) a predetermined number of said strip activities of said second field are greater than said second threshold; and

dropping each detected repeated field from said video data stream to generate a modified video data stream thereby eliminating encoding of redundant information and so improving encoding efficiency.

40. The method of any one of claims 36, 37, 38, or 39 further comprising:

updating said first threshold upon identifying said field as a repeated field.

41. The method of claim 40 wherein said updating said first threshold upon identifying said field as a repeated field further comprises:

comparing said field activity with a stored field activity;

replacing said stored field activity with said field activity if said field activity is greater than said stored field activity; and

defining said first threshold in terms of said stored field activity.

42. The method of claim 40 further comprising:

updating said second threshold after updating said first threshold.

43. The method of claim 42 wherein said updating said second threshold further comprises:

defining said second threshold in terms of said first threshold.

37. A method for improving encoding efficiency of a video data stream including a plurality of frames and each frame includes a field of a first parity and a field of a second parity, said method comprising:

analyzing each field in said video data stream to detect repeated fields wherein said analyzing process further comprises;

comparing a field activity of a field with a first threshold; and

identifying said field as a repeated field if said field activity of said field is less than said first threshold and said field has said second parity;

comparing a strip activity of each strip of said field with a second threshold; and

identifying said field as a repeated field if a predetermined number of said strip activities of said field are less than said second threshold and said field has said second parity;

repeating said field activity and strip activity comparisons for a second field of said second parity if (i) said field has said first parity;

(ii) said field activity of said field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said field are less than said threshold;

comparing a measure of field activity of said field with a third threshold if (i) said field activity of said second field is less than said first threshold; and

(ii) a predetermined number of said strip activities of said second field are less than said second threshold; and

identifying said field as a repeated field if said measure of field activity of said field is less than said third threshold; and

dropping each detected repeated field from said video data stream to generate a modified video data stream thereby eliminating encoding of redundant information and so improving encoding efficiency.

38. A method for improving encoding efficiency of a video data stream including a plurality of frames and each frame includes a field of a first parity and a field of a second parity, said method comprising:

analyzing each field in said video data stream to detect repeated fields wherein repeated fields in a telecine sequence in said video data stream are detected and repeated fields in a cartoon sequence in said video data stream are detected and further wherein said analyzing process further comprises;

comparing a field activity of a field with a first threshold; and

identifying said field as a repeated field if said field activity of said field is less than said first threshold and said field has said second parity; and

dropping each detected repeated field from said video data stream to generate a modified video data stream thereby eliminating encoding of redundant information and so improving encoding efficiency.

39. The method of claim 38 further comprising:

comparing a strip activity of each strip of said field with a second threshold; and

identifying said field as a repeated field if a predetermined number of said strip activities of said field are less than said second threshold and said field has said second parity.

44. A method for improving encoding efficiency of a video data stream, said method comprising:

analyzing each field in said video data stream to detect repeated fields wherein said analyzing each field further comprises;

measuring a strip activity between a strip in a first field of a first parity and a strip of a second field of said first parity in said video data steam for a plurality of strips thereby obtaining a plurality of strip activities for said first field;

generating a field activity for said first field using said plurality of strip activities;

comparing said field activity to a first threshold;

comparing each of said plurality of strip activities to a second threshold; and

identifying said first field as a repeated field if (i) said first parity is an even parity;

(ii) said field activity for said first field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said first field are less than said second threshold; and

dropping each detected repeated field from said video data stream to generate a modified video data stream thereby eliminating encoding of redundant information and so improving encoding efficiency.

45. The method of claim 44 further comprising:

performing the following operations if (i) said first parity is an odd parity;

(ii) said field activity for said first field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said first field are less than said second threshold;

measuring a strip activity between a strip in a first field of an even parity and a strip of a second field of said even parity in said video data stream for a plurality of strips to obtain a plurality of strip activities for said first field of said even parity;

generating a field activity for said first field of said even parity using said plurality of strip activities for said first field of said even parity;

comparing said field activity for said first field of said even parity to said first threshold;

comparing each of said plurality of strip activities for said first field of said even parity to said second threshold; and

identifying said first field of said first parity as a repeated field if (i) said field activity for said first field of said even parity is greater than said first threshold;

or (ii) a predetermined number of said strip activities for said first field of said even parity are greater than said second threshold.

46. The method of claim 44 further comprising:

performing the following operations if (i) said first parity is an odd parity;

(ii) said field activity for said first field is less than said first threshold; and

(iii) a predetermined number of said strip activities of said first field are less than said second threshold;

measuring a strip activity between a strip in a first field of an even parity and a strip of a second field of said even parity in said video data stream for a plurality of strips thereby obtaining a plurality of strip activities for said first field of said even parity;

generating a field activity for said first field of said even parity using said plurality of strip activities for said first field of said even parity;

comparing said field activity for said first field of said even parity to said first threshold;

comparing each of said plurality of strip activities for said first field of said even parity to said second threshold

comparing a measure of field activity of said field with a third threshold if (i) said field activity of said first field of even parity is less than said first threshold; and

(ii) a predetermined number of said strip activities of said first field of even parity are less than said second threshold; and

identifying said field as a repeated field if said measure of field activity of said field is less than a third threshold.

47. The method of any one claims 44, 45, or 46, wherein said strip activity is a sum, for all pixels in a strip, of an absolute difference between a pixel in a first field of one parity and a pixel of a second field of said one parity.

48. The method of any one of claims 44, 45, or 46 further comprising:

updating said first threshold upon identifying said first field as a repeated field.

49. The method of claim 48 wherein said updating said first threshold further comprising:

comparing said field activity of said first field with a stored field activity;

replacing said stored field activity with said field activity of said first field if said field activity of said first field is greater than said stored field activity; and

defining said first threshold in terms of said stored field activity.

50. The method of claim 49 further comprising:

updating said second threshold after updating said first threshold.

51. The method of claim 50 wherein said updating said second threshold further comprises:

defining said second threshold in terms of said first threshold.