Parallel-pipelined image processing system

• US 6,044,166 A
• Filed: 02/23/1996
• Issued: 03/28/2000
• Est. Priority Date: 01/17/1995
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Hsu, Stephen C., Bhatt, Bhavesh B., Call, Bill J., Bassman, Robert G, van der Wal, Gooitzen S., Hansen, Michael W., Wixson, Lambert E.
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Bella, Matthew C.
• Application Number:
  US08/606,171
• Time in Patent Office: 1,495 Days
• Field of Search: 382/103, 382/107, 382/296, 382/307, 382/325, 395/800, 395/312, 395/309, 364/516, 348/149, 348/716, 701/1, 710/126, 710/129, 710/131, 710/132
• US Class Current: 382/103
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

Apparatus for image processing a sequence of images containing a parallel-pipelined image processor comprised of image memories, a pyramid processing circuit, an arithmetic logic unit, a crossbar switch for video routing through the various components of the processor, signal processors to provide hardware programming through a global bus and also perform image processing operations. Images can be passed directly from the crossbar switch to internal static RAM of the signal processors through a first-in, first-out interface at full video rates.

Claims

1. Apparatus for image processing a sequence of images representing a scene comprising:

a cross-bar switch for selectively routing said sequence of images to a plurality of image storage elements, a pyramid processing circuit, and an arithmetic logic unit;

a buffer, connected to an output port of the crossbar switch, for buffering images from the crossbar switch at video rates;

a first digital signal processor and a second digital signal processor, connected to said buffer, for processing said images from said crossbar switch where said buffered images from said buffer are coupled directly to a random access memory of said first and second digital signal processors; and

a global bus, connected to said crossbar switch and said first and second digital signal processors, for communicating information between said first digital signal processor, said second digital signal processor, said crossbar switch and an output connector.

2. The apparatus of claim 1 wherein said arithmetic logic unit transforms a two-dimensional image portion into a one-dimensional representation of the image portion.

3. The apparatus of claim 1 wherein said output connector is a VME P1 connector.

4. The apparatus of claim 1 further comprising a dynamic random access memory (DRAM), connected to said global bus, for storing information derived from processing said images.

5. The apparatus of claim 1 wherein said crossbar switch is an eight channel by ten bit switch that provides non-blocking interconnectivity of images.

6. The apparatus of claim 1 wherein the pyramid processing unit generates a Gaussian and Laplacian pyramid from images provided by the crossbar switch.

7. The apparatus of claim 1 wherein said image storage elements are video random access memories (VRAMs).

8. The apparatus of claim 1 further comprising a look up table connected to said arithmetic logic unit and said cross-bar switch.

9. The apparatus of claim 8 wherein said arithmetic logic unit further comprises means for pointwise subtracting landmark image patches in a reference image from corresponding image patches in a current image, and an illumination-compensated reference image from a current image.

10. The apparatus of claim 8 wherein said arithmetic logic unit further comprises means for performing image accumulation where pointwise image match measures are summed over an area.

11. The apparatus of claim 8 wherein said arithmetic logic unit further comprises means for performing row-wise masked image summation and energy computations, whereby a two-dimensional image area is reduced to a one-dimensional image.

12. The apparatus of claim 11 wherein said arithmetic logic unit further comprises means for generating image transpositions, where said row-wise masked image summation is performed column-wise.

13. The apparatus of claim 8 wherein said arithmetic logic unit further comprises:

means for quantizing an image within said image sequence; and

means for run-length encoding adjacent zeroes within a quantized image for performing video compression of said image within said sequence of images.

14. The apparatus of claim 1 further comprising an electrically erasable programmable read-only memory (EEPROM), connected to the global bus, for storing application programs.

15. The apparatus of claim 1 further comprising a static random access memory (SRAM) connected to said digital signal processor.

16. The apparatus of claim 1 further comprising a first static random access memory (SRAM) connected to said first digital signal processor and a second static random access memory (SRAM) connected to said second digital signal processor.

17. Apparatus for image processing comprising:

an imaging sensor for generating a sequence of images representing a scene;

a cross-bar switch for selectively routing said sequence of images to a plurality of image storage elements, a pyramid processing circuit, and an arithmetic logic unit, where said image storage elements are video random access memories (VRAMs), said pyramid processing circuit generates a Gaussian and Laplacian pyramid from images provided by the crossbar switch, and said arithmetic logic unit provides object detection functions;

a first-in, first-out buffer, connected to an output port of the crossbar switch, for buffering images from the crossbar switch at video rates;

a first digital signal processor, connected to said buffer, for processing said images from said crossbar switch;

a second digital signal processor, connected to said buffer, for processing said images from said crossbar switch, where said buffered images from said first-in, first-out buffer are coupled directly to a random access memory of said second digital signal processor;

a first static random access memory (SRAM) connected to said first digital signal processor, where said buffered images from said first-in, first-out buffer are coupled directly to said first SRAM;

a second static random access memory (SRAM) connected to said second digital signal processor, where said buffered images from said first-in, first-out buffer are coupled directly to said second SRAM;

a global bus, connected to said crossbar switch and said digital signal processor, for communicating between said digital signal processor, said crossbar switch and an output connector;

a dynamic random access memory (DRAM), connected to said global bus, for storing information derived from processing said images by said first and second digital signal processors; and

a EEPROM, connected to the global bus, for storing application programs to be executed by said arithmetic logic unit and said digital signal processors.

18. The apparatus of claim 17 further comprising a look up table connected to said arithmetic logic unit and said cross-bar switch.

19. The apparatus of claim 17 wherein said arithmetic logic unit further comprises means for pointwise subtracting landmark image patches in a reference image from corresponding image patches in a current image, and an illumination-compensated reference image from a current image.

20. The apparatus of claim 17 wherein said arithmetic logic unit further comprises means for performing image accumulation where pointwise image match measures are summed over an area.

21. The apparatus of claim 17 wherein said arithmetic logic unit further comprises means for performing row-wise masked image summation and energy computations, whereby a two-dimensional image area is reduced to a one-dimensional image.

22. The apparatus of claim 21 wherein said arithmetic logic unit further comprises means for generating image transpositions, where said row-wise masked image summation is performed column-wise.

23. The apparatus of claim 17 wherein said arithmetic logic unit further comprises:

means for quantizing an image within said image sequence; and

means for run-length encoding adjacent zeroes within a quantized image for performing video compression of said image within said sequence of images.

24. A traffic monitoring system for processing a sequence of images representing a scene of traffic comprising:

a cross-bar switch for selectively routing said sequence of images to a plurality of image storage elements, a pyramid processing circuit, and an arithmetic logic unit, where said pyramid processing circuit and said arithmetic logic unit process the sequence of images to generate a reference image representing stationary objects within the scene and generate motion images representing objects that are moving relative to the reference image;

a buffer, connected to an output port of the crossbar switch, for buffering the reference image and the motion images from the crossbar switch at video rates;

a digital signal processor, connected to said buffer, for processing said motion images and said reference image from said crossbar switch, where the buffered images from said buffer are coupled directly to a random access memory of said digital signal processor and said digital signal processor identifies the moving objects; and

a global bus, connected to said crossbar switch and said digital signal processor, for communicating information between said digital signal processor, said crossbar switch and an output connector.

25. The traffic monitoring system of claim 24 wherein said arithmetic logic unit and digital signal processor further comprises means for updating said reference image with information within said images, where said information used for updating the reference image only represents stationary objects within the scene and substantially disregards moving objects and temporarily stationary objects within the scene.

26. The traffic monitoring system of claim 24 wherein said arithmetic logic unit and digital signal processor further comprises means for modifying said reference image using image information within said images regarding changes to illumination of the scene.