Data modifier apparatus and method for machine vision systems

• US 4,445,137 A
• Filed: 09/11/1981
• Issued: 04/24/1984
• Est. Priority Date: 09/11/1981
• Status: Expired due to Fees

• Current Assignee: Machine Intelligence Corp.
• Original Assignee: Machine Intelligence Corp.
• Inventors: Panofsky, Edward F.
• Primary Examiner(s): Masinick, Michael A.
• Assistant Examiner(s): Coles, Edward L.
• Application Number:
  US06/301,428
• Time in Patent Office: 956 Days
• Field of Search: 358/93, 358/107, 358/101, 358/133, 364/515, 382/41
• US Class Current: 382/302
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023

Abstract

An image data processing system, including a camera for producing a serial image data system corresponding to an array of pixels having a preselected relationship to a scene viewed thereby and producing a separate format signal stream accompanying the image data stream. The system further includes an image data modification system comprising multistage digital delay circuit for separately delaying the image data stream and the accompanying format signal stream in a synchronous manner, a circuit arrangement for withdrawing at least a portion of the image data stream from at least one stage of the delay circuit, a circuit arrangement for processing the withdraw image data stream portion in accordance with a preselected algorithmic function to produce a modified image data stream, and a circuit arrangement for reinserting the modified image data stream into a subsequent stage of the delay circuit. An image utilization device such as an image processing system is coupled to the image data modifier system for utilizing the modified image data stream.

Claims

1. In an image data processing system, in combination:

imaging means for producing a serial image data stream corresponding to an array of pixels having a preselected relationship to a scene viewed thereby and producing a separate format signal stream accompanying said image data stream;

image data modification means including multistage digital delay means for separately delaying said image data stream and said accompanying format signal stream in a synchronous manner, means for withdrawing at least a portion of said image data stream from at least one stage of said delay means, means for processing said withdrawn image data stream portion in accordance with a preselected algorithmic function to produce a modified image data stream, and means for reinserting said modified image data stream into a subsequent stage of said delay means; and

image utilization means coupled to said image data modifier means for utilizing said modified image data stream.

2. The system of claim 1, wherein said processing means includes a function processor receiving said withdrawn image data stream and adapted to process said data stream together with at least a second data stream in accordance with a preselected algorithm to produce said modified image data stream, and a modifier data source for supplying a modifier data stream to said function processor in synchronism with said image data stream.

3. The system of claim 2, wherein said imaging means produces an image data stream comprising successive data words of n bit length, each data word corresponding to an image pixel;

and said modifier data source comprises memory means storing a plurality of preselected modifier data words each associated with one of said image data words and means for reading said modifier data words out of said memory means and presenting said modifier data words said function processor in synchronism with the arrival of associated image data words.

4. The system of claim 3, wherein n=1, and said memory means comprises at least one random access memory means, and said reading and presenting means comprises buffer storage means coupled to said memory means for receiving and storing a word of said modifier data;

a shift register coupled to buffer storage means for receiving a word of modifier data therefrom and communicating successive bits of said modifier data word to said function processor one at a time;

address circuit means for addressing said memory to read out a word therefrom into said buffer storage means; and

control circuit means receiving at least one format signal for controlling said address circuit means, said buffer register means and said shift register means to present successive modifier data bits to said function processor in synchronism with corresponding bits of said image data stream presented thereto.

5. The system of claim 3, wherein n is greater than 1, said function processor is adapted to process first and second data streams comprising successive n and m bit words, respectively, in accordance with a preselected algorithm;

and said memory means comprises m random access memory means, each storing one bit of each m bit word in said modifier data stream; and

said reading and presenting means comprises a separate buffer storage means coupled to each of said random access memory means for receiving and storing a word of said modifier data therefrom;

a shift register coupled to each of said buffer storage means for receiving a word of modifier data therefrom and communicating successive bits of said word to said function processor one at a time;

address circuit means for addressing each of said memories simultaneously to read out a word therefrom into associated buffer storage means; and

control circuit means receiving at least one format signal for controlling said address circuit means, each of said buffer register means, and each of said shift register means to present successive modifier bits to said function processor in synchronism with corresponding bits of said image data stream.

6. The system of either claim 4 or 5, wherein said image data stream is a video data stream organized as a rectangular array of pixels having y lines of x pixels each, and said format signal streams includes at least a frame start pulse on a first format data line preceding the first row of pixel data and an end of line signal substantially coincident with the last pixel in each line;

each said random access memory means is organized as s words of r bits each, r being substantially greater than the memory cycle time of said random access memory means multiplied by the pixel frequency, and s being a sufficiently high value to store an entire modifier memory plane corresponding to a complete video frame;

said address circuit means comprises a word counter and a line counter each having their outputs coupled as address lines into each of said memory means; and

said control circuit means comprises a pixel counter incremented by video shift clock pulses, means for resetting said line counter in response to each frame start signal, means for resetting said word counter and said pixel counter in response to each frame start signal and each end of line signal, means initiated by each frame start signal and each end of line signal for generating a sequence of a first memory read cycle, a buffer register load and word counter increment pulse and coincident shift register load and video shift clock pulse prior to the arrival of the first pixel data word at said function processor;

means for supplying a sequence of video clock pulses timed to shift said shift register and increment said pixel counter substantially coincident with the arrival of the second and subsequent pixel data words at said function processor;

means for generating a sequence of a second memory address cycle and a buffer register load and word counter increment pulse prior to the occurrence of the rth video shift clock pulse; and

means initiated by a carry out signal from said pixel counter after each rth video shift clock pulse for generating a sequence of a shift register load signal, a memory read cycle, and a buffer register load and word counter increment pulse.

7. The system of claim 6, wherein said memory means includes a plurality of modifier memory data planes each having modifier data corresponding to each pixel in said video data stream, and said address circuit means includes at least one settable memory plane address register for supplying at least one selectable memory address bit to said memory means for selecting one of said modifier memory planes to be read out from said memory means.

8. The system of claim 2, wherein said function processor is a programmable function processor adapted to process said video data stream and said second data stream in accordance with one of a plurality of selectable algorithms, and said system further includes function select means for selecting one of said algorithms for active implementation by said programmable function processor.

9. The system of claim 2, wherein n=1 and said function processor is a programmable function processor and comprises a multiplexer having a plurality of multiplexing control terminals and a plurality of data input terminals, one of said multiplexor control terminals receiving either said image data stream of said modifier data stream and the other of said multiplexing control terminals receiving function select signals;

said data input terminals receiving an arrangement of ground and pull-up voltages together with at least one logical version of the other of said image data stream and said modifier data stream, said arrangement being predetermined such that the output of said multiplexer is one of a plurality of logic functional combinations of said image data stream and said modifier data stream selectable by said function select signals.

10. In an image data processing system, in combination;

imaging means for producing a serial image data stream corresponding to an array of pixels in a scene viewed thereby;

image data modification means including a source of a modifier data stream and a function processing means receiving said modifier data stream and said video data stream for processing both said data streams in real time in accordance with a preselected algorithm to produce a modified image data stream; and

image utilization means coupled to said image data modifier means for utilizing said modified video data stream.

11. The system of claim 10, wherein said imaging means produces a video data stream comprising successive data words of n bit length each corresponding to an image pixel;

and said modifier data source comprises memory means storing a plurality of preselected modifier bits, each associated with one of said data words in said video data stream, and means for reading said modifier data bits out of said memory and presenting said modifier bits to said function processing means in synchronism with the arrival of said associated data words of said video data stream.

12. The system of claim 11, wherein n=1, and said imaging means also produces a separate format signal stream accompanying said image data stream;

said memory means comprises at least one random access memory means; and

said reading and presenting means comprises buffer storage means coupled to said memory means for receiving and storing a word of said modifier data;

a shift register coupled to buffer storage means for receiving a word of modifier data therefrom and communicating successive bits of said modifier data word to said function processor one at a time;

address circuit means for addressing said memory to read out a word therefrom into said buffer storage means; and

control circuit means receiving at least one format signal for controlling said address circuit means, said buffer register means and said shift register means to present successive modifier data bits to said function processor in synchronism with corresponding bits of said image data stream presented thereto.

13. The system of claim 11, wherein n is greater than 1, and said imaging means also produces a separate format signal stream accompanying said image data stream;

said function processor means is adapted to process first and second data streams comprising successive m and n bit words, respectively, in accordance with a preselected algorithm; and

said memory means comprises m random access memory means, each storing one bit of each m bit word in said modifier data stream; and

said reading and presenting means comprises a separate buffer storage means coupled to each of said random access memory means for receiving and storing a word of said modifier data therefrom;

a shift register coupled to each of said buffer storage means for receiving a word of modifier data therefrom and communicating successive bits of said word to said function processor one at a time; and

address circuit means for addressing each of said memories simultaneously to read out a word therefrom into associated buffer storage means; and

control circuit means receiving at least one format signal for controlling said address circuit means, each of said buffer register means, and each of said shift register means to present successive modifier bits to said function processor in synchronism with corresponding bits of said image data stream.

14. The system of either claim 12 or 13, wherein said video data stream is a video data stream organized as a rectangular array of pixels having y lines of x pixels each, and said format signal streams includes at least a frame start pulse on a first format data line preceding the first row of pixel data and an end of line signal substantially coincident with the last pixel in each line;

each said random access memory means is organized as s words of r bits each, r being substantially greater than the memory cycle time of said random access memory means multiplied by the pixel frequency, and s being a sufficiently high value to store an entire modifier memory plane corresponding to a complete video frame, said address circuit means comprises a word counter and a line counter each having their outputs coupled as address lines into each of said memory means; and

said control circuit means comprises a pixel counter incremented by video shift clock pulses, means for resetting said line counter in response to each frame start signal, means for resetting said word counter and said pixel counter in response to each frame start signal and each end of line signal, means initiated by each frame start signal and each end of line signal for generating a sequence of a first memory read cycle, a buffer register load and word counter increment pulse and coincident shift register load and video shift clock pulses prior to the arrival of the first pixel data word at said function processor;

means for supplying a sequence of video shift clock pulses timed to shift said shift register and increment said pixel counter substantially coincident with the arrival of the second and subsequent pixel data words at said function processor;

means for generating a sequence of a second memory read cycle and a buffer register load and word counter increment pulse prior to the occurrence of the rth video shift clock pulse; and

means initiated by a carry out signal from said pixel counter after each rth video shift clock pulse for generating a sequence of a shift register load signal, a memory read cycle, and a buffer register load and word counter increment pulse.

15. The system of claim 14, wherein said memory means includes a plurality of modifier memory planes each having modifier data corresponding to each pixel in said video data stream and said address circuit means includes at least one settable memory plane address register for supplying at least one selectable memory address bit to said memory means for selecting one of said modifier memory planes to be read out from said memory means.

16. The system of claim 10, wherein said function processor is a programmable function processor adapted to process said video data stream and said second data stream in accordance with one of a plurality of selectable algorithms, and said system further includes function select means for selecting one of said algorithms for active implementation by said programmable function processor.

17. The system of claim 10, wherein n=1 and said function processor is a programmable function processor and comprises a multiplexer having a plurality of multiplexing control terminals and a plurality of data input terminals, one of said multiplexor control terminals receiving either said video data stream or said modifier data stream and the other of said multiplexing control terminals receiving function select signals;

said data input terminals receiving an arrangement of ground and pull-up voltages together with at least one logical version of the other of said video data stream and said modifier data stream, said arrangement being predetermined such that the output of said multiplexer is one of a plurality of logic functional combinations of said video data stream and said modifier data stream selectable by said function select signals.

18. The system of either claim 6 or 14, wherein said image utilization means includes a programmable central processor unit having a preselected bus protocol for input/output ports;

and said system further comprises;

processor address means for addressing said random access memory means, including a processor address register and a processor selectable address port circuit for communicating address signals from said processor to said processor address register; and

processor read and write means for writing data into and reading data from said random access memory means, including write register means coupled to said random access memory means, read register means coupled to said random access memory means, a processor selectable read/write port circuit means for communicating data from said processor to said write register means and for transmitting data from said read register means to said processor, and read/write control circuit means responsive to signals from said processor via said read/write port circuit to initiate either a memory read or a memory write cycle.

19. The system of claim 18, wherein said function processor is a programmable function processor adapted to respond to at least one input control bit to process said video data stream and a modifier data stream in accordance with one of a plurality of selectable algorithms;

said memory means is adapted to store a plurality of modifier data planes; and

said system further comprises;

status register means for supplying at least one function processor select bit to said programmable function processor, at least one video mask address bit to said pixel address circuit means and at least one processor mask address bit to said processor address register, said status register means including a plurality of status registers and a processor selectable status register port circuit for communicating status register data from said processor to said status registers.

20. The system of claim 19 adapted to selectably write data to, and read data from, said random access memory in a word mode involving R bit transfers or a compatability mode involving r/2 bit transfers and to selectably autoincrement said processor address register during a processor controlled read or write cycle in either said word mode or said compatability mode, wherein said processor address register is a tristate counter;

said processor address means further includes control logic responsive to an increment address signal to automatically increment said process address register to the address of the next modifier data word, and a separate additional single address bit storage means and control logic coupled to said address port circuit for writing an address bit into said single address bit storage means;

said processor read and write means includes an r line data bus communicating between said read and write register means and said read/write port circuit means;

said write register means includes a write register high adapted to receive and store the high r/2 bits of a modifier memory word from the high r/2 lines of said r line data bus and to communicate said high r/2 bit to the high r/2 memory data lines of said random access memory in response to a high write out enable command, a write register low adapted to receive and store the lower r/2 bits of a modifier memory word from the low r/2 lines of said r line data bus and to communicate said low r/2 bits to the low r/2 memory data lines of said random access memory in response to a low write out enable command, and a write register swap adapted to receive and store the high r/2 bits of a modifier memory word from the low r/2 lines of said r line data bus and to communicate said high r/2 bits to the high r/2 memory data lines of said random access memory in response to a swap write out enable command, said read register means includes a read register high adapted to receive and store the high r/2 bits of a modifier memory word from the high r/2 memory data lines of said random access memory and to communicate said high r/2 bits to the high r/2 lines of said r line data bus in response to a high read out enable command, a read register low adapted to receive and store the lower r/2 bits of a modifier memory word from the low r/2 memory data lines of said random access memory means and to communicate said low r/2 bits to the low r/2 lines of said r line data BUS in response to a low read out enable command, and a read register swap adapted to receive and store the high r/2 bits of a modifier memory word from the high r/2 memory data lines of said random access memory means and to communicate said high r/2 bits to the low r/2 lines of said r line data bus in response to a swap write out enable command;

said read/write control circuit means includes logic circuit means responsive to a compatibility mode signal of one logic value to produce only low write out enable and high write out enable commands during a processor controlled write cycle and to produce only low read out enable and high read out enable signals during a processor controlled read cycle, said logic circuit means being further responsive to a compatibility mode signal of a second logic value to produce alternate low write out enable and swap write out enable signals depending on the logic value of said stored single address bit during a processor controlled write cycle and to produce alternative low read out enable and swap read out enable signals depending on said stored single address bit value during a processor controlled read cycle; and

said system further includes autoincrement logic circuit means responsive to an autoincrement read or autoincrement write signal during processor controlled read or write cycles to generate said increment address signal at the beginning of a read cycle or the end of a write cycle, and compatibility mode/autoincrement logic means including logic circuit means for deriving an address count enable signal for said processor address registers from a logic combination of the bit value of said compatibility mode signal and the bit value stored in said single address bit storage means to enable memory address count incrementing of said processor address register on receipt of each increment address signal when said system is in said word mode and to enable memory address circuit incrementing only on receipt of every other increment address signal when said system is in said compatibility mode.

21. In a method for real time processing of a serial image data stream comprising an array of pixels and having an accompanying separate format signal stream, the steps of:

separately delaying in multiple stages said image data stream and said format signal stream in a synchronized manner;

withdrawing at least a portion of said image data stream from at least one of said delay stages;

processing said withdrawn image data stream according to preselected algorithmic function to produce a modified image data stream; and

reinserting said modified image data stream into a data signal path in parallel with and synchronous to associated delayed format signals.

22. In a method for real time processing of a serial image data stream comprising an array of pixel data words or bits and having an accompanying separate format signal stream, the steps of:

delaying in separate synchronized digital delay stages said image data stream and said format signal stream;

withdrawing at least a portion of image data stream from at least one of said delay stages;

deriving a modifier data stream time synchronized with said image data stream;

processing said withdrawn image data stream and said modifier image data stream according to a preselected algorithmic function to produce a modified image data stream; and

reinserting said modified image data stream into a data signal path in parallel with and synchronized to associated delayed format signal in a delayed format signal stream.