USB video interface for ALPR cameras and associated method

• US 8,922,656 B2
• Filed: 06/14/2011
• Issued: 12/30/2014
• Est. Priority Date: 06/16/2010
• Status: Active Grant

• Current Assignee: NDI Technologies Incorporated
• Original Assignee: NDI Technologies Incorporated
• Inventors: Quinn, Alan, Goonetilleke, Tharaka
• Primary Examiner(s): Patel, Jay Au
• Assistant Examiner(s): GEROLEO, FRANCIS
• Application Number:
  US13/636,475
• Publication Number:
  US 20130076903A1
• Time in Patent Office: 1,295 Days
• Field of Search: 348/148
• US Class Current: 348/148
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A video processing system (10) for automatic license plate recognition (ALPR) provides an interface between video cameras (14, 16) and a host computer (20) through a single high speed USB connection (18). The system (10) supports four simultaneous composite video channels (28A, 28B, 28C, 28D) with two channels dedicated to an IR monochrome video and two channels dedicated to color video. Compression software incorporates a high speed memory for buffering and stripping unnecessary data from a digitized data stream. Monochrome channel data and color channel data are marked with an incrementing index to tag them with frame identification. As a result, processing software in the host computer can re-align the monochrome and color video information after processing a license plate image, regardless of the order of data arrival. Color overview and monochrome recognition images are provided at the same time, and thus show the same plate or vehicle.

Claims

1. A video signal processing system (10) for receiving video feeds (12A, 12B, 12C, 12D) from automatic license plate recognition (ALPR) cameras (14, 16) and for providing a single USB connection (18) to a host computer (20), the system (10) comprising:

video decoders (22A, 22B, 22C, 22D) adapted to accept video signal inputs (20) from the ALPR cameras (14, 16) and convert the video signal inputs (20) into digital component video formatted outputs (22) in video channels (28A, 28B, 28C, 28D);

sync processor logic modules (30A, 30B, 30C, 30D) communicating with the video channels (28A, 28B, 28C, 28D), wherein input data provided thereby is clocked-in via each video decoder (22A, 22B, 22C, 22D) into a corresponding sync processor logic module (30A, 30B, 30C, 30D), and wherein the sync processor logic modules (30A, 30B, 30C, 30D) extract active video data from a data stream and discard non-active data unnecessary for ALPR applications;

input FIFO modules (32A, 32B, 32C, 32D) for each of the video channels (32A, 32B, 32C, 32D), wherein each input FIFO module (32A, 32B, 32C, 32D) is controlled by one of the sync processor logic modules (30A, 30B, 30C, 30D) when the active video data is present;

a memory controller (34) operable for receiving data from the input FIFO modules (32A, 32B, 32C, 32D);

output FIFO modules (38A, 38B, 38C, 38D) operable for determining if there is sufficient available data storage and operable with the memory controller (34) for providing data for a particular video channel (28A, 28B, 28C, 28D);

a compression module (36) receiving data from the memory controller (34) for each of the video channels (32A, 32B, 32C, 32D) and adaptable for storing software instruction for compressing the data sent to the output FIFO modules (38A, 38B, 38C, 38D) for each video channel (32A, 32B, 32C, 32D), wherein the memory controller (34) serves the output FIFO modules (38A, 38B, 38C, 38D) through the compression module (36);

a FIFO control logic module (40) operable with each video channel (32A, 32B, 32C, 32D), the FIFO Control logic module (40) adapted to determine if the output FIFO modules (38A, 38B, 38C, 38D) have sufficient storage space for a preselected set of data entries, wherein the compression module (36) compresses the data and transmits resultant compressed data to the output FIFO modules (38A, 38B, 38C, 38D) via the FIFO control logic module (40);

a USB transceiver (42) comprising a microcontroller, wherein the USB transceiver (42) is operable in a first in, first out (FIFO) mode for an initial configuration and setup of the FIFO output modules (38A, 38B, 38C, 38d);

a USB endpoint controller (44) operable with the USB transceiver (42) for transferring data from the output FIFO modules (38A, 38B, 38C, 38D) to the USB Transceiver (42), wherein data from the Output FIFO modules (38A, 38B, 38C, 38D) is clocked in and clocked out using the USB transceiver (42) operable with the FIFO control logic module (40); and

a single high speed USB connector (52) connected to the USB transceiver (42), the single high speed USB connector (52) thus providing the single USB connection (18) to the host computer (20).

2. The system (10) according to claim 1, wherein first and third sync processor logic modules (30A, 30C) on first and third channels (32A, 32C) are dedicated to monochrome signal processing, and wherein second and fourth sync processor logic modules (30B, 30D) on second and fourth channels (32B, 32D) are dedicated to color signal processing.

3. The system (10) according to claim 1, wherein the sync processor logic modules (30A, 30B, 30C, 30D) preload the input FIFO modules (32A, 32B, 32C, 32D) with control codes useful to the host computer (20) for reassembling a video frame generated by the ALPR cameras (14, 16).

4. The system (10) according to claim 1, wherein the FIFO control logic module (40) is responsive to a programmable flag of the output FIFO modules (38A, 38B, 38C, 38D) to determine if the output FIFO modules (38A, 38B, 38C, 38D) have sufficient storage space for a preselected set of data entries, and wherein the FIFO control logic module (40) provides a READY signal to the memory controller (34) indicating it is ready to accept data for compression.

5. The system (10) according to claim 1, further comprising a USB Hub Controller (46) to allow the USB transceiver (42) and a 2 channel USB to Serial interface operable therewith to be on one USB port, wherein the hub controller (46) provides one USB uplink port (48) and two USB downlink ports (50).

6. The system (10) according to claim 1, wherein the compression module (36) is adapted to operate with compression software for operation with a 320 MHz clock, which is derived from an 80 MHz system clock using a digital clock manager module.

7. The system (10) according to claim 1, wherein the USB endpoint controller (44) uses a 48 MHz clock provided by the USB Transceiver (42) to perform all operations, and wherein the USB endpoint controller (44) operates in a round-robin fashion to serve each video channel (28A, 28B, 28C, 28D).

8. The system (10) according to claim 7, wherein the microcontroller of the USB transceiver (42) is in an idle mode after initialization, and wherein the USB Transceiver (42) is setup to have 4 USB Endpoints (54A, 54B, 54C, 54D), one endpoint for each video channel (28A, 28B, 28C, 28D), each endpoint configured as a bulk endpoint to ensure integrity of data and to maximize bandwidth, each endpoint having a 1 KB FIFO arranged in a double buffered configuration, wherein an interface (82) of the USB Transceiver (42) is a 16 bit data bus and control/status IO, wherein the interface for all USB endpoint FIFO modules (EP2, EP4, EP6 and EP8) is uni-directional to the host computer (20), wherein the USB Endpoint FIFO Logic Controller (44) writes data through the Endpoint FIFO Interface, and wherein a USB Serial Interface Engine (SIE) (58) of the USB Transceiver (42) transfers data between the Endpoint FIFO modules (EP2, EP4, EP6, EP8) and the host computer (20) for communications therewith.

9. The system (10) according to claim 8, wherein FIFO Full signals for all endpoints are configured to assert when there is room for one entry in the FIFO, and wherein when the FIFO Full signal resets, there is always room for two or more entries in the corresponding FIFO.

10. The system (10) according to claim 1, further comprising a DDR SDRAM module (60), wherein the memory controller (34) includes DDR memory service logic, and wherein the DDR memory service logic transfers data to and from the DDR SDRAM module (60).

11. The system (10) according to claim 10, wherein the memory controller (34) is operable for receiving data from the Input FIFO modules (32A, 32B, 32C, 32D) and transferring the data to and from the DDR SDRAM module.

12. The system (10) according to claim 1, wherein each input FIFO module (32A, 32B, 32C, 32D) contains a 16 Byte 8-bit FIFO (62), 8-bit to 32-bit converter (64) and a 32-bit FIFO (66) with 1032 32-bit entries, wherein if the 8-bit FIFO (62) is not full, 8-bit Data from each Sync Processor (30A, 30B, 30C, 30D) is clocked into the 8-bit FIFO (62) using a 27 MHz Clock derived from a CLK output of the video decoder (22A, 228, 22C, 22D), wherein a FIFO Enable for the 8-bit FIFO is controlled by the sync processor (30A, 308, 30C, 30D) and is asserted only when the active data is present on the input data bus, and wherein the sync processor preloads the 8-bit FIFO with control codes for a Frame Start, Line End and Frame End to support operation of the host computer (20) to re-assemble a Video Frame, wherein a FIFO Programmable Flag of the 8-bit FIFO is asserted when the 8-bit FIFO has at least 4 bytes, and wherein once the FIFO programmable flag is set, the 8 to 32 bit converter clocks out 4 bytes from the 8-bit FIFO and packs them in to a 32-bit register, and wherein if the 32-bit FIFO is not FULL, the 32 bit value is clocked in to the 32-bit FIFO, wherein the converter runs from an 80 MHz clock, which is 90°

phase shifted from a main clock to have the FIFO synchronized with a DDR memory controller of the memory service logic (34).

13. The system (10) according to claim 1, further comprising a serial interface (68) adapted to setup the video decoders (22A, 22B, 22C, 22D) and to adjust characteristics including at least one of hue, brightness, saturation, and sharpness.

14. The system (10) according to claim 1, further comprising an RS232 interface (70) communicating with the USB hub controller (46), the RS232 interface (70) having ports (70A, 70B) for connecting to two ALPR cameras (14, 16).

15. The system (10) according to claim 1, further comprising ALPR Camera Power Management electronics (72) controlling power delivered to the system from a power source (74).

16. The system (10) according to claim 15, further comprising a universal serial bus (76) for receiving power from the power source (74) providing a 12V DC supply used to power the ALPR Camera Power Management electronics (72).

17. The system (10) according to claim 16, wherein the 12V DC supply for the cameras is sent through a protection block (78) to protect from at least one of an over voltage, reverse polarity and transient voltages.

18. The system (10) according to claim 1, wherein the compression module (36) comprises a processor adapted to compress video data with a minimum resolution loss, wherein the processor looks ahead for a pre-determined amount of data in a data stream to recognize a repetition of consequent data, and wherein if a repetition is not recognized, the processor modifies the data stream to create a repetition such that de-compression may be completed in the host computer (20) through a decoding of the data.

19. The system (10) according to claim 18, wherein the compression of the video data provided by the processor comprises;

memory service logic for reading 512 bytes of data from the memory controller (34) into the compression module (36) when the compression module (36) is in a ready state;

comparators adapted to compare a first 16 bytes of data to determine if a repetition of the value of first byte is recognized (referred as Rv hereafter), wherein the comparators compare in parallel the same data with Rv+1, and Rv−

1, if Rv>

1 or Rv+2 and Rv+1 if Rv=1 (video data values being non zero), wherein three results are combined using OR Logic to see if there is a continuous repetition any combination, wherein if a repetition is not found or the number of repeated characters is less than 4, the first byte is clocked out and the operation jumps to a next step, wherein the most common value within the repetition (Rv, Rv+1, Rv−

1 or Rv+2) is treated as a value for the repetition (referred as Vr hereafter), and wherein the entire portion of the repetition is omitted from the data stream and a three byte sequence is sent to the Output FIFO modules (38A, 38B, 38C, 38D) as appropriate, and wherein the memory controller (34) is notified to receive a next block of data.

20. The system (10) according to claim 1, wherein the compression module (36) comprises an adaptive frame rate controller operable with the memory controller (34).

21. The system (10) according to claim 1, wherein the video signal inputs (24) from the ALPR cameras (14, 16) comprise at least one of Composite Video (CVBS) NTSC (J, M, 4.43), PAL (B, D, G, H, I, M, N, Nc), and SECAM (B, D, G, K, K1, L) Composite Video.

22. The system (10) according to claim 1, wherein each video decoder (22A, 22B, 22C, 22D) comprises one 9-bit analog to digital converter (ADC) with 2×

sampling, and wherein data sampling is done in compliance with ITU-R BT.601, 27 MHz (generated from the 14.32222 MHz crystal oscillator input, and wherein the output format is 8-bit ITU-R BT.656 with embedded synchronization, the out interface (80) has an 8-bit data bus and a clock, wherein every rising edge of a clock signal indicates valid data on the data bus, the clock output being used to clock-in the data into the sync processor modules (30A, 30B, 30C, 30D).

23. The system (10) according to claim 22, wherein a signal-in signal timing comprises a data stream from the video decoders (22A, 22B, 22C, 22D) having embedded sync/timing codes used to indicate a start active video (SAV), end active video (EAV), and vertical/horizontal blanking, wherein a sync processor extracts the active video, wherein the video decoder (22A, 22B, 22C) automatically detects a color system being used, and wherein data from the video decoders are fed into a sync processor module (30A, 30B, 30C, 30D), wherein valid data is available and clocked into the sync processor module on every rising edge of a clock signal.

24. The system (10) according to claim 23, wherein each sync processor module (30A, 30B, 30C) employs a PAL standard, wherein each Line in the PAL video frame consists of 1716 bytes of data, of which, 1440 bytes are pixel data (from 0 to 1440 bytes after SAV) and 275 bytes of blanking data, wherein 275 bytes of blanking data is removed from the data stream, which thus equates to 158400 bytes (275*576), wherein each Frame includes two video fields (odd field and even field), each odd field contains 22 blanking lines before and 2 blanking lines after active video, each even field contains 23 blanking lines before and 3 blanking lines after active video, wherein the blanking lines do not represent valid data in a digital domain unless special features are used, wherein a total of 49 Lines out of 625 lines present in the data stream is removed by the sync processor module (30A, 30B, 30C, 30D), thus equating to a reduction of 84084 bytes (1716*49) per frame, wherein after eliminating the blanking data from the frame, each frame contains 829440 bytes (1440*576), and wherein a raw frame contains 1072500 bytes (625*1716), wherein the reduction of data per frame equates to 22.6% (829440/1072500*100%), thus with 25 Frames per second on each video channel produces an effective data rate of 20,736,000 bytes/sec per video channel, and wherein the sync processor module removes color information from the dedicated monochrome channels, reducing the data rate by half, to about 10 MB/S for monochrome channels, making an effective data rate from all video decoders approximately 60 MB/S.

25. The system (10) according to claim 1, wherein the memory controller (34) operates such that data from each video channel (28A, 28B, 28c, 28D) goes into a separate bank of memory, and wherein the memory acts as a circular buffer providing a 3 second buffer for a monochrome channel and a 2 second buffer for a color channel, wherein data from IN FIFO modules (32A, 32B, 32C, 32D) are transferred into DDR memory once a word count flag of the IN FIFO modules is equal to or greater than 132, wherein data are transferred from the DDR memory into the OUT FIFO modules (38A, 38B, 38C, 38D) until the word count flag is equal to or greater than 1536, wherein the DDR Memory Interface transfers data at a rate of 320 MB/s, transferring 2 bytes for both rising and falling edge of 80 MHz Clock.

26. The system (10) according to claim 1, wherein each output FIFO module (38A, 38B, 38C, 38D) is 32-bit FIFO with 2048 32-bit entries, the output FIFO modules having a programmable flag, which is asserted when there is room for at least 132 entries (512 Bytes), wherein when the memory service logic (34) gets its turn for a particular video channel to output data, if the programmable flag is set on the Output FIFO module for that video channel, the memory service logic pulls 512 bytes of data from the DDR memory, runs it through the compression module (36) and writes the resultant data to the Output FIFO modules, wherein the output FIFO programmable flag is set to 132 entries (512 Bytes) for providing room to fit all data pulled from the memory (512 Bytes), wherein the Output FIFO data is clocked in with 80 MHz, Ø

90°

clock and the data is clocked out using a 48 MHz Clock from the USB transceiver (42).

27. The system (10) according to claim 1, wherein the a USB transceiver (42) operates with the USB endpoint controller (44), the controller having an 8051 based microcontroller and up to 4 endpoint FIFOs, wherein the USB Transceiver (42) is used in a FIFO mode, wherein the 8051 based microcontroller is used for initial configuration, wherein after the initialization, the microcontroller is in idle mode, wherein the USB transceiver (42) is setup to have 4 USB endpoints, one for each video channel (28A, 28B, 28C, 28D), each endpoint is configured as a bulk endpoint to ensure integrity of data and to maximize bandwidth, each endpoint has a 1 KB FIFO arranged in a double buffered configuration, wherein an interface (82) to the USB Transceiver (42) is a 16 bit data bus and control/status 10, wherein all USB endpoints (54A, 54B, 54C, 54D) are uni-directional to the host computer, wherein the USB endpoint controller writes data through the Endpoint FIFO Interface, wherein a serial interface engine (SIE) (58) of the USB Transceiver (42) transfers data between the output FIFO modules (38A, 38B, 38C, 38D) and the host computer (20).

28. The system (10) according to claim 1, wherein data transfer from the output FIFO modules (38A, 38B, 38C, 38D) to the endpoint FIFO logic (44) comprises processor means for providing at least one of at least setting FIFOADR signals to select the Endpoint FIFO for a particular video channel, checking the an Endpoint FIFO Full signal (EPxFF) of the selected Endpoint FIFO, checking FIFO Empty signal from the respective Output FIFO, setting SLWR Low to indicate a Write Cycle, pulling one 32-bit entry from the Output FIFO, placing a Least Significant 16 bits of the 32 bit entry on the Endpoint FIFO data bus on the falling edge of the CLK signal, and transferring the data into the FIFO on the next rising edge of the CLK signal, and placing a Most Significant 16 bits of the 32 bit entry on the Endpoint FIFO data bus on the next falling edge of the CLK signal, the data transferred into the FIFO on the next rising edge of the CLK signal.

29. The system (10) according to claim 1, wherein four video decoders (22A, 22B, 22C, 22D) communicate with four sync processors (30A, 30B, 30C, 30D) operable with four input FIFO modules (32A, 32B, 32C, 32D) through four video channels (28A, 28B, 28C, 28D), and wherein the compression module (36) receives data from the memory controller (34) for the four video channels and provides compressed data to four output FIFO modules (38A, 38B, 38C, 38D).