High bandwidth image processing with run time library function offload via task distribution to special purpose engines

• US 8,675,219 B2
• Filed: 10/24/2007
• Issued: 03/18/2014
• Est. Priority Date: 10/24/2007
• Status: Expired due to Fees

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Kim, Moon J., Nakano, Hiroki, Mori, Yumi
• Primary Examiner(s): Yang, Qian
• Application Number:
  US11/877,926
• Publication Number:
  US 20090110326A1
• Time in Patent Office: 2,337 Days
• Field of Search: 358/1.15
• US Class Current: 358/1.15
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding

Abstract

The present invention provides a high bandwidth image processing system, which generally includes an image processing unit having a set of servers that each have a universal operating system for receiving image data corresponding to a set of images, generating commands for processing the image data, and sending the images and the commands to an image processing unit (also having a universal operating system(s)) of the high bandwidth image processing system. Upon receipt, the image processing unit will recognize and interpret the commands, assign and/or schedule tasks for processing the image data to a set of (e.g., special) processing engines based on the commands, and return results and/or processed image data to the image interface unit.

Claims

1. A high bandwidth image processing system, comprising:

an image interface unit being configured to receive image data from a set of image recordation mechanisms for a single field of view of a same object, the set of image recordation mechanisms including at least two cameras, the image data including at least one image from each camera of a different area of the same object at substantially a same time, and to generate commands selected from an image command processing library to process the image data, and further to send the image data and the commands, the image interface unit comprising a set of servers each comprising a universal operating system; and

a plurality of special purpose processing engines (SPEs) configured to receive the image data and the generated commands from the image interface unit, to process the image data based on and by executing the commands, including assigning and distributing tasks and the image data among the plurality of SPEs based on the commands, and to send processed image data to the image interface unit, wherein each SPE includes at least one processing engine and the commands include at least one library function that is written for the at least one SPE that supports the processing of the image data, and wherein the at least one library function is one of a plurality of algorithms, an optimization for a processor architecture, and a wrapper for a scripting language, the at least one SPE thereby being configured to receive an off-load of the at least one library function at a run time, wherein the field of view has a plurality of inspection threads each running an algorithm that is different from the algorithm of any other inspection thread.

2. The high bandwidth image processing system of claim 1, wherein the image interface unit and the SPEs are coupled via a communications switch.

3. The high bandwidth image processing system of claim 1, further comprising a client interface unit for accessing processed image data from at least one of the SPEs and the image interface unit.

4. The high bandwidth image processing system of claim 1, each of the set of servers further comprising:

a set of image grabbers being configured to receive image data from the set of image recordation mechanisms;

a set of communications cards; and

a set of high density drives (HDDs) for storing the image data collected from the image recordation mechanisms.

5. The high bandwidth image processing system of claim 4, the SPEs comprising a set of processors, each of the set of processors comprising a communications card.

6. The high bandwidth image processing system of claim 5, further comprising:

a Gigabit Ethernet (GbE) switch coupled to the SPEs; and

a network file system coupled to the GbE switch.

7. The high bandwidth image processing system of claim 6, the network file system comprising a server having a HDD.

8. The high bandwidth image processing system of claim 1, the image interface unit and the SPEs being embodied within an image co-processor, the image co-processor communicating with a control processor.

9. The high bandwidth image processing system of claim 1, the universal operating system of each of the set of servers being Linux.

10. A high bandwidth image processing system, comprising:

an image interface unit configured for;

receiving image data from a plurality of image recordation mechanisms for a single field of view of a same object, the plurality of image recordation mechanisms including at least two cameras, image data including at least one image from each of the at least two cameras, each image being of a different area of the same object at substantially a same time,

generating commands selected from an image command processing library,

sending the image data and the commands, and

receiving processed image data, the image interface unit comprising a universal operating system, client application, an image grabber library, a first communications library, and an image processing command library; and

a plurality of special purpose processing engines (SPEs) configured for;

receiving the image data and the generated commands from the image interface unit,

assigning and distributing tasks among the plurality of SPEs based on the generated commands such that at least one SPE is configured to receive an off-load of at least one library function at a run time, the at least one library function being written for the at least one SPE, supporting the processing of the image data, and is one of a plurality of algorithms, an optimization for a processor architecture, and a wrapper for a scripting language,

processing the image data by executing the commands, and

returning processed image data to the image interface unit,

wherein the plurality of processing engines includes a universal operating system, a cell application, a command dispatcher, an image processing library, a software developer kit library, and a second communications library, wherein the commands include the at least one library function.

11. The high bandwidth image processing system of claim 10, the cell application processing the image data based on the commands received from the client application, the commands being generated using the image processing command library.

12. The high bandwidth image processing system of claim 10, the image interface unit further comprising:

a set of offload stubs;

a communications card, the set of offload stubs and the communications card being used by the client application to send the image data to the cell application and to receive the processed image data from the cell application; and

an image grabber card, the image grabber card being used by the client application to receive the image data from a set of image recordation mechanisms.

13. The high bandwidth image processing system of claim 10, wherein the command dispatcher is configured for recognizing the commands, and for distributing tasks for processing the image data to special purpose engines.

14. The high bandwidth image processing system of claim 10, the commands and image data being sent from the client application to the cell application using the first communication library, the second communications library and a set of communications cards.

15. The high bandwidth image processing system of claim 10, the image interface unit and the SPEs each further comprising firmware and a processor.

16. The system of claim 10, the image interface unit and the SPEs being implemented as a co-processor.

17. A method for processing images, comprising:

receiving image data on an image interface unit from a set of image recordation mechanisms for a single field of view of a same object, the set of image recordation mechanisms including at least two cameras, the image data including at least one image from each camera of a different area of the same object at substantially a same time, and generating commands selected from an image command processing library, the image interface unit comprising a set of servers each comprising a universal operating system;

sending the image data and the generated commands to a plurality of special purpose processing engines (SPEs);

interpreting the generated commands;

assigning tasks to and distributing tasks among the plurality of SPEs;

processing the image data on the plurality of SPEs by executing the commands and in accordance with assigned and distributed tasks, wherein the commands include at least one library function that is written for the plurality of SPEs, wherein the at least one library function supports the processing of the image data, and wherein the at least one library function is one of a plurality of algorithms, an optimization for a processor architecture, and a wrapper for a scripting language, the processing of the image data including receiving with at least one SPE an off-load of at least one respective library function at a run time; and

providing access to processed image data from a client application unit.

18. The method of claim 17, further comprising generating the commands on the image interface unit using an image processing command library.

19. The method of claim 17, further comprising:

interpreting the commands on the SPEs; and

scheduling tasks for processing the image data on the SPEs.

20. The method of claim 17, the sending and the returning occurring via communication libraries accessed by the image interface unit and the SPEs.

21. The method of claim 17, the receiving occurring via a set of image grabbers of the image interface unit.

22. A program product stored on at least one computer readable storage device for processing images, the at least one computer readable storage device comprising program code for causing at least one computer system to:

receive image data from a set of image recordation mechanisms for a single field of view of a same object on an image interface unit comprising a set of servers each comprising a universal operating system, the set of image recordation mechanisms including at least two cameras and the image data including at least one image from each camera of a different area of the same object at substantially a same time;

generate commands selected from an image command processing library;

interpret the commands;

assign at least one task to a plurality of special purpose processing engines (SPEs) and, responsive to the at least one task including at least two tasks, distributing the at least two tasks among the plurality of SPEs, to process the image data by executing the generated commands corresponding the image data including off-loading at least one library function to a respective one of the plurality of SPEs at a run time;

process the image data in response to the tasks,

wherein the commands include the at least one library function that is written for the plurality of SPEs, wherein the at least one library function supports the processing of the image data, and wherein the at least one library function is one of a plurality of algorithms, an optimization for a processor architecture, and a wrapper for a scripting language.

23. The program product of claim 22, the at least one computer readable medium further comprising program code for causing at least one computer system to send processed image data.

24. The program product of claim 22, the commands being generated using an image processing command library.

25. The program product of claim 22, the image data being received by a set of image grabbers.

26. A method for deploying a system for processing images, comprising:

providing a computer infrastructure being operable to;

receive image data for a single view of a same object from a set of image recordation mechanisms on an image interface unit comprising a set of servers each comprising a universal operating system, the set of image recordation mechanisms including at least two cameras and the image data including at least one image from each camera of a different view of the same object at substantially a same time;

generate commands selected from an image command processing library;

interpret the commands;

assign at least one task to a plurality of special purpose processing engines (SPEs) to process the image data by executing the generated commands corresponding the image data, including off-loading at least one library function to a respective one of the plurality of SPEs at a run time; and

process the image data in response to the tasks,

wherein the commands include the at least one library function that is written for the plurality of SPEs, wherein the at least one library function supports the processing of the image data, and wherein the at least one library function is one of a plurality of algorithms, an optimization for a processor architecture, and a wrapper for a scripting language.