Video compression method and apparatus

• US 20050008077A1
• Filed: 03/31/2003
• Published: 01/13/2005
• Est. Priority Date: 03/31/2003
• Status: Active Grant

• Current Assignee: Broadcast Microwave Services, Inc. (COHU Incorporated)
• Original Assignee: Duma Video, Inc.
• Inventors: Weatherspoon, Sultan
• Granted Patent: US 7,519,115 B2
• Time in Patent Office: Days
• US Class Current: 375/240.240
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A video compression system includes first and second dual-port memory devices, a third memory device, and first and second processors that may provide enhanced processing, including motion estimation. The first processor may be configured to store in the second memory device first and second video frames and to transfer sequential sets of data from the first video frame corresponding to fields of search. A second set of a plurality of adjacent macroblocks of the second video frame may be compared to macroblocks selected from the field of search. Dual-port memory devices may allow for the concurrent use of shared memory by the two processors as well as data transfer during processing.

Claims

1. A video compression system comprising:

a first memory device;

a first processor configured to process video data according to a first video compression process, and to write processed data to the memory device; and

a second processor configured to read video data processed by the first video compression process from the memory device while the first processor is writing processed data to the memory device, and process the read data by a second video compression process.

2. The system of claim 1, where the first and second processes include at least one of motion estimation, motion compensation, discrete-cosine transformation, quantization, and entropy encoding.

3. The system of claim 2, where at least one of the first and second processes includes motion estimation, and further includes a stage of a hierarchical search.

4. The system of claim 3, where the one process is a search of a first set of a plurality of adjacent macroblocks of a first video frame relative to a field of search of a second set of macroblocks of a second video frame.

5. The system of claim 4, where the first set includes an array of J macroblocks, and the second set includes an array of M sets of J adjacent macroblocks, where J and M are integers greater than one.

6. The system of claim 5, where a plurality of macroblocks of one of the first and second sets is compared concurrently with at least one macroblock of the other set.

7. The system of claim 2, where the first and second processes include motion estimation having overlapping fields of search, the system further comprising a second memory device configured to store video data representative of the combined fields of search of the first and second processes, the first and second processors further configured to read data stored in the second memory device.

8. The system of claim 7, where the first and second processors are configured to read concurrently data from the second memory device.

9. The system of claim 7, further comprising a third memory device coupled to the second memory device for storing a plurality of frames of the video data, and a controller configured to write data into the third memory device and transfer data from the third memory device to the second memory device.

10. The system of claim 9, where the first processor processes a first set of data for a first frame while the field of search for a subsequent set of data for the first frame is being written into the second memory device.

11. The system of claim 1, where the first and second processors are configured to process sequential sets of data, and the first processor is further configured to write a third set of processed data over a first set of processed data while the second processor reads a second set of processed data.

12. The system of claim 11, where the first processor is configured to process sets of data with each set including a plurality of adjacent blocks of data, and to perform at least one process step on each block of data.

13. The system of claim 12, where the first processor is further configured to write into the memory device a set of processed data prior to processing a subsequent set of data.

14. A video compression system comprising:

a first memory device;

a second memory device; and

a first processor configured to store in the first memory device data representative of at least a portion of a first video frame, to transfer sequential sets of data representative of corresponding portions of the frame from the first memory device to the second memory device, to process each current set of data stored in the second memory device according to a first video compression process while transferring a sequentially next set of data from the first memory device to the second memory device.

15. The system of claim 14, where the processor is configured to process sequential sets of overlapping data in which the sequentially next set of data overlaps with the current set of data, the processor being further configured to read from the first memory that portion of the next sequential set of data not included in the current set of data during processing of the current set of data.

16. The system of claim 15, where the video compression process is motion estimation, the processor is configured to store in the first memory device at least portions of each of a plurality of video frames, and the sets of data are fields of search.

17. The system of claim 16, where the search is a search of a first set of a plurality of adjacent macroblocks of a first video frame relative to a field of search of a second set of macroblocks of a second video frame.

18. The system of claim 17, where the first set includes an array of J macroblocks, and the second set includes an array of M sets of J adjacent macroblocks, where J and M are integers greater than one.

19. The system of claim 18, where a plurality of macroblocks of one of the first and second sets is compared concurrently with at least one macroblock of the other set.

20. The system of claim 15, further comprising a second processor configured to read data stored in the second memory device and to process a portion of each current set of data stored in the second memory device according to a second video compression process.

21. The system of claim 20, where the first and second processors are configured to read concurrently data from the second memory device.

22. A video compression system comprising:

a first memory device; and

a first processor configured to store in the first memory device a set of video data representative of a first field of search including a first set of a plurality of macroblocks of a first video frame, and to search the first set of macroblocks relative to a second set of a plurality of adjacent macroblocks of a second video frame.

23. The system of claim 22, where the first set includes an array of M subsets of J adjacent macroblocks, and the second set includes an array of J macroblocks, where J and M are integers greater than one.

24. The system of claim 23, where the array of M macroblocks includes an array of K×

L sets of J macroblocks, where K and L are integers greater than 1.

25. The system of claim 23, where a plurality of macroblocks of one of the first and second sets is compared concurrently with at least one macroblock of the other set.

26. The system of claim 25, where the macroblocks in the array of J macroblocks are contained within the same lines of the second video frame.

27. The system of claim 25, where the macroblocks in the second set are compared concurrently with each of a plurality of macroblocks selected from the first set.

28. The system of claim 27, further comprising a second processor configured to search an associated plurality of macroblocks selected from the first set of macroblocks, relative to each macroblock of the second set of macroblocks.

29. The system of claim 28, where the second processor is further configured to compare sequentially each macroblock in the second set with overlapping macroblocks included in the associated plurality of macroblocks selected from the first set.

30. The system of claim 29, where the second processor is configured to compare each macroblock of the second set concurrently with a plurality of macroblocks from the first set.

31. The system of claim 25, where the macroblocks in the second set are sequentially compared with macroblocks selected from the first set, with each macroblock of the second set being compared concurrently with a plurality of macroblocks from the first set.

32. A video compression system comprising:

first and second dual-port memory devices;

a third memory device;

a first processor configured to store in the second memory device data representative of first and second video frames, to transfer sequential sets of data representative of corresponding portions of the first frame from the second memory device to the third memory device, each set of data representative of a first field of search including a first set of a plurality of macroblocks of a first video frame, and to process each set of data stored in the third memory device according to a first motion estimation process, the first motion estimation process including searching the first set of macroblocks relative to a second set of a plurality of adjacent macroblocks of a second video frame, the first motion estimation process being performed while transferring, from the second memory device to the third memory device, a portion of a sequentially next set of data not included in the set of data currently stored in the third memory device, the first processor further being configured to write processed data processed according to the first motion estimation process to the first memory device; and

a second processor configured to read video data processed by the first processor from the first memory device while the first processor is writing processed data to the first memory device, and process the read data by a second motion estimation process including searching an associated plurality of macroblocks selected from the first set of macroblocks, relative to each macroblock of the second set of macroblocks, and comparing sequentially each macroblock in the second set with overlapping macroblocks included in the data corresponding to an associated plurality of macroblocks selected from the first set, and comparing each macroblock of the second set concurrently with a plurality of macroblocks from the first set.

33. A method of compressing video data comprising:

processing video data according to a first video compression process;

writing processed data to a first memory device;

while writing processed data, reading processed data from the memory device; and

processing the read data by a second video compression process.

34. The method of claim 33, where processing the video data and processing the read data each includes at least one of motion estimating, motion compensating, discrete-cosine transforming, quantizing, and entropy encoding.

35. The method of claim 33, where at least one of processing the video data and processing the read data includes motion estimating, and further includes performing a stage of a hierarchical search.

36. The method of claim 35, where performing a stage of a hierarchical search includes searching a first set of a plurality of adjacent macroblocks of a first video frame relative to a field of search of a second set of a plurality of macroblocks of a second video frame.

37. The method of claim 36, where searching the first set includes searching an array of J macroblocks relative to an array of M sets of J adjacent macroblocks, where J and M are integers greater than one.

38. The method of claim 37, where searching an array includes comparing a plurality of macroblocks of one of the first and second sets concurrently with at least one macroblock of the other set.

39. The method of claim 33, where processing the video data and processing the read data each includes motion estimating, and further includes searching with overlapping fields of search, the method further comprising storing video data representative of the combined fields of search in a second memory device, and reading data stored in the second memory device.

40. The method of claim 39, where processing the video data and processing the read data includes reading concurrently data from the second memory device.

41. The method of claim 39, further comprising storing a plurality of frames of the video data in a third memory device, and writing data into the third memory device and transferring data from the third memory device to the second memory device.

42. The method of claim 41, where processing the video data includes processing a first set of data for a first frame while writing into the second memory device a field of search for a subsequent set of data for the first frame.

43. The method of claim 33, where processing the video data includes writing a third set of processed data over a first set of processed data, and processing the read data includes reading a second set of processed data during writing of the third set of data.

44. The method of claim 43, where processing the video data includes processing sequential sets of data with each set including a plurality of adjacent blocks of data, including processing each block in each set of data.

45. The method of claim 44, where processing the video data includes writing into the memory device a set of processed data prior to processing a subsequent set of data.

46. A method of compressing video data comprising:

storing in a first memory device data representative of at least a portion of a first video frame;

transferring sequential sets of data representative of corresponding portions of the frame from the first memory device to a second memory device;

processing each set of data stored in the second memory device according to a first video compression process; and

during processing of each set of data stored in the second memory device, transferring a sequentially next set of data from the first memory device to the second memory device.

47. The method of claim 46, where processing data stored in the second memory device includes processing sequential sets of overlapping data in which a sequentially next set of data overlaps with a current set of data, and reading from the first memory that portion of the next sequential set of data not included in the current set of data during processing of the current set of data.

48. The method of claim 47, where processing each set of data includes estimating motion by searching in a field of search, and storing data in the first memory device includes storing at least portions of each of a plurality of video frames, and transferring sets of data includes transferring sets of data corresponding to fields of search.

49. The method of claim 48, where searching includes searching a first set of a plurality of adjacent macroblocks of a first video frame relative to a field of search of a second set of macroblocks of a second video frame.

50. The method of claim 49, where the first set includes an array of J macroblocks, and the second set includes an array of M sets of J adjacent macroblocks, where J and M are integers greater than one.

51. The method of claim 50, where searching a first set of a plurality of adjacent macroblocks includes comparing concurrently a plurality of macroblocks of one of the first and second sets with at least one macroblock of the other set.

52. The method of claim 47, further comprising processing a portion of each current set of data stored in the second memory device according to a second video compression process.

53. The method of claim 52, where processing each set of data stored in the second memory device includes reading a first portion of data from the second memory device, and processing a portion of each current set of data stored in the second memory device includes reading a second portion of data from the second memory device concurrently with reading the first portion of data from the second memory device.

54. A method of compressing video data comprising:

storing in a first memory device a set of data representative of a first field of search including a first set of a plurality of macroblocks of a first video frame; and

searching the first set of macroblocks relative to a second set of a plurality of adjacent macroblocks of a second video frame.

55. The method of claim 54, where the first set includes an array of M subsets of J adjacent macroblocks, and the second set includes an array of J macroblocks, where J and M are integers greater than one.

56. The method of claim 55, where the array of M macroblocks includes K×

L sets of J macroblocks, where K and L are integers greater than 1.

57. The method of claim 55, where searching includes comparing concurrently a plurality of macroblocks of one of the first and second sets with at least one macroblock of the other set.

58. The method of claim 57, where the macroblocks in the array of J macroblocks are contained within the same lines of the second video frame.

59. The method of claim 57, where comparing includes comparing concurrently the macroblocks in the second set with each of a plurality of macroblocks selected from the first set.

60. The method of claim 59, further comprising selecting a plurality of macroblocks from the first set of macroblocks associated with each macroblock of the second set of macroblocks, and searching the associated plurality of macroblocks selected from the first set of macroblocks relative to each macroblock of the second set of macroblocks.

61. The method of claim 60, where searching the associated plurality of macroblocks includes comparing sequentially each macroblock in the second set with overlapping macroblocks included in the associated plurality of selected macroblocks.

62. The method of claim 61, where comparing each macroblock in the second set includes comparing concurrently each macroblock of the second set with a plurality of macroblocks from the first set.

63. The method of claim 57, where comparing includes comparing sequentially the macroblocks in the second set with macroblocks selected from the first set, including comparing concurrently each macroblock of the second set with a plurality of macroblocks from the first set.

64. A storage medium having a plurality of machine-readable instructions that, when executed, provide for:

processing video data according to a first video compression process;

writing processed data to a first memory device;

while writing processed data, reading processed data from the memory device; and

processing the read data by a second video compression process.

65. The storage medium of claim 64, where processing the video data and processing the read data each includes at least one of motion estimating, motion compensating, discrete-cosine transforming, quantizing, and entropy encoding.

66. The storage medium of claim 64, where at least one of processing the video data and processing the read data includes motion estimating, and further includes performing a stage of a hierarchical search.

67. The storage medium of claim 66, where performing a stage of a hierarchical search includes searching a first set of a plurality of adjacent macroblocks of a first video frame relative to a field of search of a second set of a plurality of macroblocks of a second video frame.

68. The storage medium of claim 67, where searching the first set includes searching an array of J macroblocks relative to an array of M sets of J adjacent macroblocks, where J and M are integers greater than one.

69. The storage medium of claim 64, where processing the video data and processing the read data each includes motion estimating, and further includes searching with overlapping fields of search, the instructions further provide for storing video data representative of the combined fields of search in a second memory device, and reading data stored in the second memory device.

70. The storage medium of claim 69, where processing the video data and processing the read data includes reading concurrently data from the second memory device.

71. The storage medium of claim 64, where processing the video data includes writing a third set of processed data over a first set of processed data, and processing the read data includes reading a second set of processed data during writing of the third set of data.

72. The storage medium of claim 71, where processing the video data includes processing sequential sets of data with each set including a plurality of adjacent blocks of data, including processing each block in each set of data.

73. A storage medium having a plurality of machine-readable instructions that, when executed, provide for:

storing in a first memory device data representative of at least a portion of a first video frame;

transferring sequential sets of data representative of corresponding portions of the frame from the first memory device to a second memory device;

processing each set of data stored in the second memory device according to a first video compression process; and

during processing of each set of data stored in the second memory device, transferring a sequentially next set of data from the first memory device to the second memory device.

74. The storage medium of claim 73, where processing data stored in the second memory device includes processing sequential sets of overlapping data in which a sequentially next set of data overlaps with a current set of data, and reading from the first memory that portion of the next sequential set of data not included in the current set of data during processing of the current set of data.

75. The storage medium of claim 74, where processing each set of data includes estimating motion by searching in a field of search, and storing data in the first memory device includes storing at least portions of each of a plurality of video frames, and transferring sets of data includes transferring sets of data corresponding to fields of search.

76. The storage medium of claim 75, where searching includes searching a first set of a plurality of adjacent macroblocks of a first video frame relative to a field of search of a second set of macroblocks of a second video frame.

77. The storage medium of claim 74, where the instructions further provide for processing a portion of each current set of data stored in the second memory device according to a second video compression process.

78. The storage medium of claim 77, where processing each set of data stored in the second memory device includes reading a first portion of data from the second memory device, and processing a portion of each current set of data stored in the second memory device includes reading a second portion of data from the second memory device concurrently with reading the first portion of data from the second memory device.

79. A storage medium having a plurality of machine-readable instructions that, when executed, provide for:

storing in a first memory device a set of data representative of a first field of search including a first set of a plurality of macroblocks of a first video frame; and

searching the first set of macroblocks relative to a second set of a plurality of adjacent macroblocks of a second video frame.

80. The storage medium of claim 79, where the first set includes an array of M subsets of J adjacent macroblocks, and the second set includes an array of J macroblocks, where J and M are integers greater than one.

81. The storage medium of claim 80, where searching includes comparing concurrently a plurality of macroblocks of one of the first and second sets with at least one macroblock of the other set.

82. The storage medium of claim 81, where the macroblocks in the array of J macroblocks are contained within the same lines of the second video frame.

83. The storage medium of claim 81, where comparing includes comparing concurrently the macroblocks in the second set with each of a plurality of macroblocks selected from the first set.

84. The storage medium of claim 83, where the instructions further provide for selecting a plurality of macroblocks from the first set of macroblocks associated with each macroblock of the second set of macroblocks, and searching the associated plurality of macroblocks selected from the first set of macroblocks relative to each macroblock of the second set of macroblocks.

85. The storage medium of claim 81, where comparing includes comparing sequentially the macroblocks in the second set with macroblocks selected from the first set, including comparing concurrently each macroblock of the second set with a plurality of macroblocks from the first set.