Parallel, pipelined, integrated-circuit implementation of a computational engine

• US 20100177828A1
• Filed: 02/04/2009
• Published: 07/15/2010
• Est. Priority Date: 01/12/2009
• Status: Active Grant

• Current Assignee: Bring Technology Inc.
• Original Assignee: Maxim Integrated Products, Inc. (Analog Devices, Inc.)
• Inventors: Rooyakkers, Albert, Rubinstein, Jorge
• Granted Patent: US 8,660,193 B2
• Time in Patent Office: Days
• US Class Current: 375/240.260
• CPC Class Codes: G06F 15/8015 One dimensional arrays, e.g. rings, linear arrays, buses
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/11 among a plurality of spatial predictive coding modes
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Embodiments of the present invention are directed to parallel, pipelined, integrated-circuit implementations of computational engines designed to solve complex computational problems. One embodiment of the present invention is a family of video encoders and decoders (“codecs”) that can be incorporated within cameras, cell phones, and other electronic devices for encoding raw video signals into compressed video signals for storage and transmission, and for decoding compressed video signals into raw video signals for output to display devices. A highly parallel, pipelined, special-purpose integrated-circuit implementation of a particular video codec provides, according to embodiments of the present invention, a cost-effective video-codec computational engine that provides an extremely large computational bandwidth with relatively low power consumption and low-latency for decompression and compression of compressed video signals and raw video signals, respectively.

Claims

1. A integrated-circuit computational engine comprising:

processing-element subcomponents, each of which carries out a high-level computational step of a stepwise computational process, the processing-element subcomponents arranged in one or more assembly-line-like series, which operate concurrently on different computational objects;

an object cache that stores computational objects, the computational objects comprising data-structure values input to processing elements prior to each high-level computational step and output from processing elements during each high-level computational step; and

an object bus that provides computational-object-level transmission transactions and through which computational objects are exchanged between the processing elements and the object cache.

2. The integrated-circuit computational engine of claim 1 implemented as a single integrated circuit.

3. The integrated-circuit computational engine of claim 1 implemented as two or more single-integrated-circuit computational engines.

4. The integrated-circuit computational engine of claim 1 implemented as one or more single-integrated-circuit computational engines and one or more additional integrated circuits.

5. The integrated-circuit computational engine of claim 1 wherein computational objects and data are passed between adjacent processing elements in the one or more assembly-line-like series of processing elements through one or more pipeline memories.

6. The integrated-circuit computational engine of claim 5 wherein a computational object is input to a first processing element in an assembly-line-like series of processing elements, operated on by the first processing element in a first high-level computational step, and output from the first processing element to a next processing element in the assembly-line-like series of processing elements for operation on by the next processing element in a next high-level processing step.

7. The integrated-circuit computational engine of claim 6 wherein the computational object is operated on, in turn, by each successive processing element in the assembly-line-like series of processing elements in subsequent high-level computational steps.

8. The integrated-circuit computational engine of claim 1 further including:

a clock component that provides a high-level-processing clock cycle according to which stepwise processing of computational objects along the one or more assembly-line-like series of processing-elements is synchronized and that provides one or more higher-frequency clock cycles, each of which controls internal processing steps of one or more of the processing elements.

9. The integrated-circuit computational engine of claim 5 further including:

a controller subcomponent that, in accordance with the high-level-processing clock cycle, controls launching of a next high-level processing step in each processing element, the controller subcomponent ensuring that current high-level processing steps are completed and that computational objects and other input data needed by each processing element for the next high-level processing step are available to the processing element prior to launching the next high-level processing step in each processing element.

10. The integrated-circuit computational engine of claim 9 wherein the controller subcomponent transmits a start signal to all processing elements of an assembly-line-like series of processing elements to launch the next high-level processing step.

11. The integrated-circuit computational engine of claim 9 wherein the controller subcomponent receives a done signal from each processing element of an assembly-line-like series of processing elements when the processing element completes a high-level processing step.

12. The integrated-circuit computational engine of claim 1 further including:

an object memory controller that controls exchange of computational objects between the object cache and a larger-capacity random-access memory.

13. The integrated-circuit computational engine of claim 12 wherein the object memory controller maps computational objects to data units in the larger-capacity random-access memory, the data units one of bytes or multi-byte words.

14. A integrated-circuit video codec comprising:

processing-element subcomponents, each of which carries out a high-level computational step of a stepwise computational process, the processing-element subcomponents arranged in one or more assembly-line-like series, which operate concurrently on different computational objects;

an object cache that stores computational objects, the computational objects comprising data-structure values input to processing elements prior to each high-level computational step and output from processing elements during each high-level computational step; and

an object bus that provides computational-object-level transmission transactions and through which computational objects are exchanged between the processing elements and the object cache.

15. The integrated-circuit computational engine of claim 14 implemented as a single integrated circuit.

16. The integrated-circuit computational engine of claim 14 implemented as two or more single-integrated-circuit computational engines.

17. The integrated-circuit computational engine of claim 14 implemented as one or more single-integrated-circuit computational engines and one or more additional integrated circuits.

18. The integrated-circuit video codec of claim 14 wherein computational objects and data are passed between adjacent processing elements in the one or more assembly-line-like series of processing elements through one or more pipeline memories.

19. The integrated-circuit video codec of claim 18 wherein a computational object is input to a first processing element in an assembly-line-like series of processing elements, operated on by the first processing element in a first high-level computational step, and output from the first processing element to a next processing element in the assembly-line-like series of processing elements for operation on by the next processing element in a next high-level processing step.

20. The integrated-circuit video codec of claim 19 wherein the computational object is operated on, in turn, by each successive processing element in the assembly-line-like series of processing elements in subsequent high-level computational steps.

21. The integrated-circuit video codec of claim 14 including two assembly-line-like series of processing elements, a first assembly-line-like series of processing elements carrying out stepwise compression of a sequence of video frames of a raw video signal to produce an output compressed video signal and a second assembly-line-like series of processing elements carrying out stepwise decompression of a compressed video signal to produce a sequence of frames of a raw video signal.

22. The integrated-circuit video codec of claim 21 wherein the first assembly-line-like series of processing elements includes:

a motion-estimation processing element;

an intra-prediction-and-inter-prediction processing element;

a residual-block calculation processing element;

a direct-integer-transform processing element;

a quantizing-and-scaling processing element; and

an entropy-encoding processing element.

23. The integrated-circuit video codec of claim 21 wherein the second assembly-line-like series of processing elements includes:

a entropy-decoding processing element;

a dequantizing-and-descaling processing element;

an inverse-integer-transform processing element;

a motion-estimation processing element;

an intra-prediction-and-inter-prediction processing element; and

a deblocking-filter processing element.

24. The integrated-circuit video codec of claim 14 wherein computational objects include:

macroblocks;

macroblock partitions;

motion vectors; and

data-and-parameter containing objects.

25. The integrated-circuit video codec of claim 14 further including:

a clock component that provides a high-level-processing clock cycle according to which stepwise processing of computational objects along the and one or more assembly-line-like series of processing-elements is synchronized and that provides one or more higher-frequency clock cycles, each of which controls internal processing steps of one or more of the processing elements.

26. The integrated-circuit video codec of claim 14 further including:

a controller subcomponent that, in accordance with the high-level-processing clock cycle, controls launching of a next high-level processing step in each processing element, the controller subcomponent ensuring that current high-level processing steps are completed and that computational objects and other input data needed by each processing element for the next high-level processing step are available to the processing element prior to launching the next high-level processing step in each processing element.

27. The integrated-circuit video codec of claim 26 wherein the controller subcomponent transmits a start signal to all processing elements of an assembly-line-like series of processing elements to launch the next high-level processing step.

28. The integrated-circuit video codec of claim 26 wherein the controller subcomponent receives a done signal from each processing element of an assembly-line-like series of processing elements when the processing element completes a high-level processing step.

29. The integrated-circuit video codec of claim 14 further including:

an object memory controller that controls exchange of computational objects between the object cache and a larger-capacity random-access memory.

30. The integrated-circuit video codec of claim 29 wherein the object memory controller maps computational objects to data units in the larger-capacity random-access memory, the data units one of bytes or multi-byte words.

31. The integrated-circuit video codec of claim 14 wherein the integrated-circuit video codec compresses raw video signals and decompresses compressed video signals according to the H.264 video decompression standard.