Data processing system and method

• US 6,215,898 B1
• Filed: 04/15/1997
• Issued: 04/10/2001
• Est. Priority Date: 04/15/1997
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Interval Research Corporation
• Inventors: Woodfill, John Iselin, Baker, Henry Harlyn, Herzen, Brian Von, Alkire, Robert Dale
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US08/839,767
• Time in Patent Office: 1,456 Days
• Field of Search: 382/154, 382/278, 382/303, 382/304, 348/42, 348/43, 348/46, 348/47, 356/12, 364/728.03, 364/728.05
• US Class Current: 382/154
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A powerful, scaleable, and reconfigurable image processing system and method of processing data therein is described. This general purpose, reconfigurable engine with toroidal topology, distributed memory, and wide bandwidth I/O are capable of solving real applications at real-time speeds. The reconfigurable image processing system can be optimized to efficiently perform specialized computations, such as real-time video and audio processing. This reconfigurable image processing system provides high performance via high computational density, high memory bandwidth, and high I/O bandwidth. Generally, the reconfigurable image processing system and its control structure include a homogeneous array of 16 field programmable gate arrays (FPGA) and 16 static random access memories (SRAM) arranged in a partial torus configuration. The reconfigurable image processing system also includes a PCI bus interface chip, a clock control chip, and a datapath chip. It can be implemented in a single board. It receives data from its external environment, computes correspondence, and uses the results of the correspondence computations for various post-processing industrial applications. The reconfigurable image processing system determines correspondence by using non-parametric local transforms followed by correlation. These non-parametric local transforms include the census and rank transforms. Other embodiments involve a combination of correspondence, rectification, a left-right consistency check, and the application of an interest operator.

Claims

1. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set; and

generating a disparity result for the first element before substantially all elements of the first and second data sets have been received; and

receiving successive rows of elements of the first and second data sets, the rows representing successive scanlines of image data;

wherein the generation of a disparity result for the first element occurs before X rows of elements of the first and second data sets have been received, where X is equal to the number of scanlines necessary to represent a frame of image data.

2. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set; and

generating a disparity result for the first element before substantially all elements of the first and second data sets have been received;

receiving successive rows of elements of the first and second data sets;

selecting a transform window having a transform window height equal to an integral number of such rows; and

selecting a correlation window having a correlation window height equal to an integral number of such rows, wherein the correlation window height is greater than the transform window height;

wherein the generation of a disparity result for the first element occurs before X rows of elements of the first and second data sets have been received, where X is equal to the correlation window height.

3. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity;

performing transformations on such received elements, including the first element and each of the plurality of selected elements; and

correlating the transformed first element with each of the plurality of transformed selected elements to generate a correlation result for each disparity, while concurrently performing transformations on one or more elements of each of the first and second data sets.

4. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity;

performing transformations on such received elements, including the first element and each of the plurality of selected elements; and

correlating the transformed first element with each of the plurality of transformed selected elements to generate a correlation result for each disparity, while concurrently receiving one or more elements of each of the first and second data sets.

5. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity;

performing transformations on such received elements, including the first element and each of the plurality of selected elements;

correlating the transformed first element with each of the plurality of transformed selected elements to generate a correlation result for each disparity, while concurrently performing transformations on one or more elements of each of the first and second data sets; and

generating a disparity result for the first element by selecting an optimal one of the correlation results, in accordance with predetermined optimization criteria, while concurrently correlating transformed elements of each of the first and second data sets.

6. A pipelined system for processing elements of a first data set and a second data set, each having a plurality of elements, and generating disparity results for such elements with low latency, the system comprising:

a transform unit for receiving elements of the first and second data sets, including a first element of the first data set, and for performing transformations on such received elements, including the first element;

a correlation unit, coupled to the transform unit, for receiving transformed elements of the first and second data sets, including the transformed first element; and

a disparity generation unit, coupled to the correlation unit, for generating a disparity result for the first element before substantially all elements of the first and second data sets have been received in the transform unit.

7. The system of claim 6, wherein:

the transform unit can receive and perform transformations on successive rows of elements of the first and second data sets, such transformations being performed with respect to a transform window having a transform window height equal to an integral number of such rows;

the correlation unit can generate correlation results for such transformed elements with respect to a correlation window having a correlation window height equal to an integral number of such rows; and

the disparity generation unit can generate a disparity result for the first element before X rows of elements of the first and second data sets have been received in the transform unit, where X is less than one-half of the sum of the transform window height and the correlation window height.

8. The system of claim 6, wherein:

the transform unit can receive and perform transformations on successive rows of elements of the first and second data sets, such transformations being performed with respect to a transform window having a transform window height equal to an integral number of such rows;

the correlation unit can generate correlation results for such transformed elements with respect to a correlation window having a correlation window height equal to an integral number of such rows; and

the disparity generation unit can generate a disparity result for the first element before X rows of elements of the first and second data sets have been received in the transform unit, where X is equal to the correlation window height.

9. The system of claim 6, wherein:

the transform unit can receive and perform transformations on successive rows of elements of the first and second data sets, the rows representing successive scanlines of image data; and

the disparity generation unit can generate a disparity result for the first element before X rows of elements of the first and second data sets have been received in the transform unit, where X is equal to the number of scanlines necessary to represent a frame of image data.

10. A pipelined system for processing elements of a first data set and a second data set, each having a plurality of elements, and generating disparity results for such elements with low latency, the system comprising:

a transform unit for receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity, and for performing transformations on such received elements, including the first element and each of the plurality of selected elements; and

a correlation unit for correlating the transformed first element with each of the plurality of transformed selected elements to generate a correlation result for each disparity, while the transform unit concurrently performs transformations on one or more received elements of each of the first and second data sets.

11. A pipelined system for processing elements of a first data set and a second data set, each having a plurality of elements, and generating disparity results for such elements with low latency, the system comprising:

a transform unit for receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity, and for performing transformations on such received elements, including the first element and each of the plurality of selected elements; and

a correlation unit for correlating the transformed first element with each of the plurality of transformed selected elements to generate a correlation result for each disparity, while the transform unit concurrently receives one or more elements of each of the first and second data sets.

12. A pipelined system for processing elements of a first data set and a second data set, each having a plurality of elements, and generating disparity results for such elements with low latency, the system comprising:

a transform unit for receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity, and for performing transformations on such received elements, including the first element and each of the plurality of selected elements;

a correlation unit for correlating the transformed first element with the plurality of transformed selected elements to generate a correlation result for each disparity, while the transform unit concurrently performs transformation on one or more received elements of each of the first and second data sets; and

a disparity generation unit for generating a disparity result for the first element by selecting an optimal one of the correlation results, in accordance with predetermined optimization criteria, while the correlation unit concurrently correlates transformed elements of each of the first and second data sets.

13. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set, including a plurality of ordered elements <

X₁, X₂, . . . X<sub>M−

1</sub>, X_M>

, and a second data set, including a plurality of ordered elements <

Y₁, Y₂, . . . Y<sub>N−

1</sub>, Y_N>

, where M and N respectively represent the total number of elements in the first and second data sets, comprising;

performing a transformation on elements of the first and second data sets, including elements X_i and X_k of the first data set and elements Y_j and Y_l of the second data set, wherein element X_i is offset from element Y_j by a first disparity, and element X_k is offset from element Y_l by a second disparity; and

correlating transformed element X_i with transformed element Y_j to generate a first correlation result, while concurrently correlating transformed element X_k with transformed element Y_l to generate a second correlation result.

14. The method of claim 13, wherein:

the elements of the first and second data sets represent pixel intensities; and

the transformed elements of the first and second data sets represent relative orderings of such elements to neighboring elements.

15. The method of claim 13, further comprising:

receiving successive rows of elements of the first and second data sets, the rows representing successive scanlines of image data; and

generating a disparity result for element X_i before R rows of elements of the first and second data sets have been received, where R is equal to the number of scanlines necessary to represent a frame of image data.

16. The method of claim 13, wherein the transformed elements of the first and second data sets are census vectors.

17. The method of claim 13, wherein each of the first and second correlation results includes a summed hamming distance.

18. The method of claim 17, wherein each of the first and second correlation results is set to a predetermined saturation threshold if its summed hamming distance exceeds that saturation threshold.

19. The method of claim 13, further comprising:

performing a transformation on an element Y<sub>j−

1</sub> of the second data set, wherein element X_i is offset from element Y<sub>j−

1</sub> by a third disparity;

correlating transformed element X_i with an element Y<sub>j−

1</sub> of the second data set to generate a third correlation result; and

selecting an optimal correlation result for element X_i from a set of correlation results including the first and third correlation results, based upon predetermined optimization criteria.

20. The method of claim 19, further comprising:

generating a disparity result for element X_i, wherein the disparity result is equal to the disparity at which element X_i is offset from the element that, when correlated with element X_i, generated the optimal correlation result for element X_i.

21. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set, including a plurality of ordered elements <

X₁, X₂, . . . X<sub>M−

1</sub>, X_M>

, and a second data set, including a plurality of ordered elements <

Y₁, Y₂, . . . Y<sub>N−

1</sub>, Y_N>

, where M and N respectively represent the total number of elements in the first and second data sets, comprising;

performing a transformation on elements of the first and second data sets, including elements X_i and X<sub>i−

k</sub> of the first data set and elements Y_j and Y<sub>j−

2k</sub> of the second data set, wherein k is an integer greater than zero, element X_i is offset from element Y_j by a first disparity, and element X<sub>i−

k</sub> is offset from element Y<sub>j−

2k</sub> by a second disparity; and

correlating transformed element X_i with transformed element Y_j to generate a first correlation result, while concurrently correlating transformed element X<sub>i−

k</sub> with transformed element Y<sub>j−

2k</sub> to generate a second correlation result.

22. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set, including a plurality of ordered elements <

X₁, X₂, . . . X<sub>M−

1</sub>, X_M>

, and a second data set, including a plurality of ordered elements <

Y₁, Y₂, . . . Y<sub>N−

1</sub>, Y_N>

, where M and N respectively represent the total number of elements in the first and second data sets, comprising;

performing a transformation on elements of the first and second data sets, including a plurality of elements X_i, X<sub>i−

1</sub>, X<sub>i−

2</sub>, . . . X<sub>i−

(D−

1)</sub> of the first data set, and a plurality of elements Y_j, Y<sub>j−

1</sub>, Y<sub>j−

2</sub>, . . . Y<sub>(j−

k)−

(D−

1)</sub> of the second data set, where D is a desired number of correlation disparities greater than one and k is an integer greater than zero and less than or equal to (D−

1), and wherein element X_i is offset from elements Y_j, Y<sub>j−

1</sub>, Y<sub>j−

2</sub>, . . . Y<sub>j−

(D−

1)</sub> by a first plurality of respective disparities, and an element X<sub>i−

k</sub> is offset from elements Y<sub>j−

k</sub>, Y<sub>j−

k)−

1</sub>, Y<sub>(j−

k)−

2</sub>, . . . Y<sub>(j−

k)−

(D−

1)</sub> by the first plurality of respective disparities;

correlating transformed element X_i with transformed elements Y_j, Y<sub>j−

1</sub>, Y<sub>j−

2</sub>, . . . Y<sub>j−

(D−

1)</sub>, respectively, to generate a first plurality of D correlation results for element X_i; and

correlating transformed element X<sub>i−

k</sub> with transformed elements Y<sub>(j−

k)</sub>, Y<sub>(j−

k)−

1</sub>, . . . Y<sub>(j−

k)−

(D−

1)</sub>, respectively, to generate a second plurality of D correlation results for element X<sub>i−

k</sub>, wherein element X_i is correlated with element Y_j at substantially the same time as element X<sub>i−

k</sub> is correlated with element Y<sub>(j−

2k)</sub>.

23. A pipelined system for generating disparity results with low latency for elements of a first data set, including a plurality of ordered elements <

X₁, X₂, . . . X<sub>M−

1</sub>, X_M>

, and a second data set, including a plurality of ordered elements <

Y₁, Y₂, . . . Y<sub>N−

1</sub>, Y_N>

, where M and N respectively represent the number of elements in the first and second data sets, comprising;

a first transform unit for performing a transformation on elements of the first data set, including elements X_i and X_k of the first data set;

a second transform unit for performing a transformation on elements of the second data set, including elements Y_j and Y_l of the second data set, wherein element X_i is offset from element Y_j by a first disparity, and element X_k is offset from element Y_l by a second disparity;

a first correlation unit, associated with the first disparity, for correlating transformed element X_i with transformed element Y_j to generate a first correlation result; and

a second correlation unit, associated with the second disparity, for correlating transformed element X_k with transformed element Y_l to generate a second correlation result, wherein the first and second correlation units operate concurrently to generate, respectively, the first and second correlation results.

24. The system of claim 23, wherein:

the elements of the first and second data sets represent pixel intensities; and

the transformed elements of the first and second data sets represent relative orderings of such elements to neighboring elements.

25. The system of claim 23, wherein the first and second transform units can receive successive rows of elements of the first and second data sets, the rows representing successive scanlines of image data, the system further comprising:

a disparity generation unit for generating a disparity result for element X_i before R rows of elements of the first and second data sets have been received in the first and second transform units, where R is equal to the number of scanlines necessary to represent a frame of image data.

26. The system of claim 23, wherein the first and second transform units can transform elements of the first and second data sets into census vectors.

27. The system of claim 23, wherein the first and second correlation units can calculate summed hamming distances to correlate transformed element X_i with transformed element Y_j, and transformed element X_k with transformed element Y_l.

28. The system of claim 27, wherein the first and second correlation units respectively can set the first and second correlation results to a predetermined saturation threshold if the calculated hamming distances exceed that saturation threshold.

29. The system of claim 23, wherein the second transformation unit can perform a transformation on an element Y_j−

1 of the second data set, where element X_i is offset from element Y<sub>j−

1</sub> by a third disparity, the system further comprising;

a third correlation unit, associated with the third disparity, for correlating transformed element X_i with transformed element Y<sub>j−

1</sub> to generate a third correlation result; and

a disparity generation unit for selecting an optimal correlation result for element X_i from a set of correlation results including the first and third correlation results, based upon predetermined optimization criteria.

30. The system of claim 29, wherein the disparity generation unit can generate a disparity result for element X_i that is equal to the disparity at which element X_i is offset from the element that, when correlated with element X_i, generated the optimal correlation result for element X_i.

31. The system of claim 29, further comprising a queueing buffer to store temporarily the disparity associated with the correlation unit that generated the optimal correlation result to perform a consistency check.

32. A pipelined system for generating disparity results with low latency for elements of a first data set, including a plurality of ordered elements <

X₁, X₂, . . . X<sub>M−

1</sub>, X_M>

, and a second data set, including a plurality of ordered elements <

Y₁, Y₂, . . . Y<sub>N−

1</sub>, Y_N>

, where M and N respectively represent the number of elements in the first and second data sets, comprising;

a first transform unit for performing a transformation on elements of the first data set, including elements X_i and X<sub>i−

k</sub> of the first data set;

a second transform unit for performing a transformation on elements of the second data set, including elements Y_j and Y<sub>j−

2k</sub> of the second data set, wherein element X_i is offset from element Y_j by a first disparity, and element X<sub>i−

k</sub> is offset from element Y<sub>j−

2k</sub> by a second disparity;

a first correlation unit, associated with the first disparity, for correlating transformed element X_i with transformed element Y_j to generate a first correlation result; and

a second correlation unit, associated with the second disparity, for correlating transformed element X<sub>i−

k</sub> with transformed element Y<sub>j−

2k</sub> to generate a second correlation result, wherein the first and second correlation units can operate concurrently to generate, respectively, the first and second correlation results.

33. A pipelined system for generating disparity results with low latency for elements of a first data set, including a plurality of ordered elements <

X₁, X₂, . . . X<sub>M−

1</sub>, X_M>

, and a second data set, including a plurality of ordered elements <

Y₁, Y₂, . . . Y<sub>N−

1</sub>, Y_N>

, where M and N respectively represent the number of elements in the first and second data sets, comprising;

a first transform unit for performing a transformation on elements of the first data set, including elements X_i, X<sub>i−

1</sub>, X<sub>i−

2</sub>, . . . X<sub>i−

(D−

1)</sub> of the first data set, where D is a desired number of correlation disparities greater than one;

a second transform unit for performing a transformation on elements of the second data set, including elements Y_j, Y<sub>j−

1</sub>, Y<sub>j−

2</sub>, . . . Y<sub>(j−

k)−

(D−

1)</sub> of the second data set, where k is an integer greater than zero and less than or equal to (D−

1), and wherein element X_i is offset from elements Y_j, Y<sub>j−

1</sub>, Y<sub>j−

2</sub>, . . . Y<sub>j−

(D−

1)</sub> by a first plurality of respective disparities, and an element X<sub>i−

k</sub> is offset from elements Y<sub>j−

k</sub>, Y<sub>(j−

k)−

1</sub>, Y<sub>(j−

k)−

2</sub>, . . . Y<sub>(j−

k)−

(D−

1)</sub> by the first plurality of respective disparities; and

a plurality of correlation units, each associated with a particular one of the D correlation disparities, for correlating transformed element X_i with transformed elements Y_j, Y<sub>j−

1</sub>, Y<sub>j−

2</sub>, . . . Y<sub>j−

(D−

1)</sub>, respectively, to generate a first plurality of D correlation results for element X_i, and correlating transformed element X<sub>i−

k</sub> with transformed elements Y<sub>(j−

k)</sub>, Y<sub>(j−

k)−

1</sub>, . . . Y<sub>(j−

k)−

(D−

1)</sub>, respectively, to generate a second plurality of D correlation results for element X<sub>i−

k</sub>, wherein element X_i is correlated with element Y_j at substantially the same time as element X<sub>i−

k</sub> is correlated with element Y<sub>(j−

2k)</sub>.

34. A method of generating disparity results in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set that is offset from each of a plurality of selected elements of the second data set by a particular disparity;

performing transformations on such received elements, including the first element and each of the plurality of selected elements;

correlating the transformed first element with each of the plurality of transformed selected elements to generate a correlation result for each disparity; and

generating a disparity result for the first element via the following steps;

generating a preliminary disparity result by selecting an optimal one of the correlation results, in accordance with predetermined optimization criteria;

selecting, as first and second adjacent correlation results, the correlation results for the next higher and next lower disparities from the preliminary disparity result; and

calculating the disparity result via a sub-pixel estimation function of the first and second adjacent correlation results and the preliminary disparity result.

35. The method of claim 34, wherein the sub-pixel estimation function is a piecewise function shaped substantially in the form of a V.

36. The method of claim 34, wherein the sub-pixel estimation function includes the following steps:

calculating a first difference equal to the first adjacent correlation result minus the preliminary disparity result;

calculating a second difference equal to the second adjacent correlation result minus the preliminary disparity result;

calculating an offset equal to 0.5 minus the minimum of the first and second differences divided by twice the maximum of the first and second differences; and

calculating the disparity result as the sum of the preliminary disparity result and the offset.

37. A method for determining the distance of image elements from an aligned first camera and second camera by generating disparity results for such image elements with low latency, comprising:

obtaining image elements of a first data set and a second data set, representing pixel intensities captured respectively by the first camera and the second camera, the first and second data sets each having a plurality of image elements, including a first image element of the first data set that is offset from each of a plurality of selected image elements of the second data set by a particular disparity;

performing transformations on such received image elements, including the first image element and each of the plurality of selected image elements;

correlating the transformed first image element with each of the plurality of transformed selected image elements to generate a correlation result for each disparity, while concurrently performing transformations on one or more image elements of each of the first and second data sets;

generating a disparity result for the first image element by selecting an optimal one of the correlation results, in accordance with predetermined optimization criteria, while concurrently correlating transformed image elements of each of the first and second data sets; and

determining the distance of the first image element from the first and second cameras, based upon the disparity result.

38. A scalable correlation unit, for generating disparity results with low latency for elements of a first data set and a second data set, each having a plurality of elements, in a pipelined system having a plurality of such correlation units connected in series, each associated with a particular disparity, the scalable correlation unit comprising:

a set of inputs for receiving a first transformed element from the first data set, a second transformed element from the second data set, a first current correlation result and a first current disparity associated with the first current correlation result;

correlation processor for correlating the first and second transformed elements and generating a second current correlation result and a second current disparity associated with the second current correlation result, and for selecting an optimal current correlation result and associated current disparity based upon predetermined optimization criteria; and

a set of delayed outputs for providing the first transformed element and the optimal current correlation result and associated current disparity to be delayed by a first delay, and the second transformed element to be delayed by a second delay.

39. A scalable correlation unit, for generating and checking the consistency of a plurality of disparity results with low latency for elements of a first data set and a second data set, each having a plurality of elements, in a pipelined system having a plurality of such correlation units connected in series, each associated with a particular disparity, the scalable correlation unit comprising:

a set of inputs including a first transformed element from the first data set, a second transformed element from the second data set, a first current correlation result and first associated disparity for elements of the first data set, and a second current correlation result and second associated disparity for elements of the second data set;

a correlation processor for correlating the first and second transformed elements and generating a new correlation result and new associated disparity, and for selecting a first and second optimal current correlation result and associated current disparity, based upon predetermined optimization criteria, for the first and second data sets respectively; and

a set of delayed outputs including the first transformed element and the first optimal current correlation result and associated current disparity to be delayed by a first delay, and the second transformed element and the second optimal current correlation result and associated current disparity to be delayed by a second delay.

40. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set; and

generating a disparity result for the first element before substantially all elements of the first and second data sets have been received; and

receiving successive rows of elements of the first and second data sets;

selecting a transform window having a transform window height equal to an integral number of such rows;

selecting a correlation window having a correlation window height equal to an integral number of such rows, wherein the correlation window height is greater than the transform window height; and

wherein the generation of a disparity result for the first element occurs before X rows of elements of the first and second data sets have been received, where X is equal to the correlation window height.