Data processing system and method

• US 6,456,737 B1
• Filed: 08/17/2000
• Issued: 09/24/2002
• Est. Priority Date: 04/15/1997
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Interval Research Corporation
• Inventors: Baker, Henry Harlyn, Herzen, Brian Von, Alkire, Robert Dale, Woodfill, John Iselin
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US09/641,610
• Time in Patent Office: 768 Days
• Field of Search: 382/154, 382/303, 382/304, 382/278, 382/106, 382/107, 708/422, 708/424, 348/42, 348/43, 348/46, 348/47
• US Class Current: 382/154
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A powerful, scaleable, and reconfigurable image processing system and method of processing data therein is described. This general purpose, reconfigurable engine with toroidal topology, distributed memory, and wide bandwidth I/O are capable of solving real applications at real-time speeds. The reconfigurable image processing system can be optimized to efficiently perform specialized computations, such as real-time video and audio processing. This reconfigurable image processing system provides high performance via high computational density, high memory bandwidth, and high I/O bandwidth. Generally, the reconfigurable image processing system and its control structure include a homogeneous array of 16 field programmable gate arrays (FPGA) and 16 static random access memories (SRAM) arranged in a partial torus configuration. The reconfigurable image processing system also includes a PCI bus interface chip, a clock control chip, and a datapath chip. It can be implemented in a single board. It receives data from its external environment, computes correspondence, and uses the results of the correspondence computations for various post-processing industrial applications. The reconfigurable image processing system determines correspondence by using non-parametric local transforms followed by correlation. These non-parametric local transforms include the census and rank transforms. Other embodiments involve a combination of correspondence, rectification, a left-right consistency check, and the application of an interest operator.

Claims

1. A parallel and pipelined image processing system for providing optimal correspondence information between a first data set of image data and a second data set of image data of a scene, comprising:

a vector generator for generating, for each selected image data in the first data set and second data set, a plurality of first vectors and a plurality of second vectors, each first vector and second vector representing the ordered relative values between said selected image data and a plurality of selected image data surrounding said selected image data; and

a correlation unit, coupled to the vector generator and receiving the plurality of first vectors and the plurality of second vectors, for generating a first correspondence information between a selected first vector and a second vector offset from each other by a first offset while said correlation unit generates a second correspondence information between another selected first vector and another second vector offset from each other by a second offset, the optimal correspondence information determined by selecting either the first correspondence information or the second correspondence information in accordance with optimization criteria.

2. A method of generating disparity results with low latency in a data processing system that processes elements of a first data set and a second data set, each having a plurality of elements, comprising:

receiving elements of the first and second data sets, including a first element of the first data set; and

generating a disparity result for the first element before substantially all elements of the first and second data sets have been received.