Prototyping an image processing algorithm and emulating or simulating execution on a hardware accelerator to estimate resource usage or performance

• US 10,235,477 B2
• Filed: 07/24/2015
• Issued: 03/19/2019
• Est. Priority Date: 07/31/2014
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Caltagirone, Christophe R. J., Vazquez, Nicolas F., Nair, Dinesh, Buchanan, Bradley D., Jenson, Jared W., Schultz, Kevin L.
• Primary Examiner(s): Gebresilassie, Kibrom K
• Application Number:
  US14/808,116
• Publication Number:
  US 20160034617A1
• Time in Patent Office: 1,334 Days
• Field of Search: None
• CPC Class Codes: G06F 11/323 Visualisation of programs or trace data
• CPC Class Codes: G06F 11/324 Display of status information
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06F 30/331 with hardware acceleration, e.g. by using field programmable gate array [FPGA] or emulation
• CPC Class Codes: G06F 8/20 Software design
• CPC Class Codes: G06F 8/314 Parallel programming languages G06F8/313 takes precedence
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06F 8/456 Parallelism detection
• CPC Class Codes: Y10S 715/967 Visual or iconic programming

Abstract

System and method for creating a machine vision application. A machine vision prototype comprising a plurality of machine vision steps specifying a machine vision image processing algorithm and associated parameters may be stored. The steps may be interpretable by an emulator to perform the specified image processing on an image by emulating or simulating execution of the steps on a hardware accelerator, e.g., a programmable hardware element or graphics processing unit. An emulator may emulate or simulate execution of the steps on the hardware accelerator, thereby generating image processing results, which may be displayed for validation of the emulating or simulating by a user. The prototype may be analyzed, and based on the analyzing, an estimate of resource usage or performance of the image processing algorithm for the hardware accelerator may be determined and displayed, and may be usable for target platform selection or modification of the image processing algorithm.

Claims

1. A non-transitory computer accessible memory medium that stores program instructions executable by a processor to implement:

storing a machine vision prototype, the machine vision prototype comprising;

a plurality of machine vision steps, wherein the plurality of machine vision steps specifies a machine vision image processing algorithm and associated parameters, and wherein the plurality of machine vision steps is interpretable by an emulator to perform the specified image processing on an image by emulating or simulating execution of the plurality of machine vision steps by a hardware accelerator;

analyzing the machine vision prototype;

automatically generating, based on said analyzing, a graphical program implementing the specified image processing algorithm, wherein the graphical program parallelizes at least a portion of the image processing, and wherein the graphical program is targeted for deployment to the hardware accelerator;

estimating resource usage or performance of execution of the graphical program by the hardware accelerator;

based on the estimated resource usage or performance, determining that the hardware accelerator is capable of executing the graphical program; and

based on the determination that the hardware accelerator is capable of executing the graphical program, deploying the graphical program to the hardware accelerator.

2. The non-transitory computer accessible memory medium of claim 1, wherein said analyzing comprises:

applying one or more heuristics to determine maximum parallelism of the machine vision image processing algorithm that can be implemented on the hardware accelerator.

3. The non-transitory computer accessible memory medium of claim 1, wherein said generating a graphical program comprises:

determining synchronization signals for the determined parallelism; and

generating graphical program code implementing the synchronization signals, wherein during operation on the hardware accelerator, the synchronization signals are used to synchronize the parallelized at least a portion of the image processing.

4. The non-transitory computer accessible memory medium of claim 1, wherein said storing, said emulating or simulating, said displaying the image processing results, said analyzing, said estimating, and said displaying the estimated resource usage or performance, are performed prior to said generating the graphical program code.

5. The non-transitory computer accessible memory medium of claim 1, wherein the hardware accelerator comprises a programmable hardware element, and wherein the program instructions are further executable by a processor to implement:

generating a hardware configuration program based on the graphical program code, wherein the hardware configuration program is deployable to the programmable hardware element for hardware implementation of the machine vision image processing algorithm.

6. The non-transitory computer accessible memory medium of claim 1, wherein the hardware accelerator comprises a graphics processing unit (GPU), and wherein the program instructions are further executable by a processor to implement:

compiling the graphical program code for execution on the GPU, thereby generating an executable program that is deployable to the GPU for hardware implementation of the machine vision image processing algorithm.

7. The non-transitory computer accessible memory medium of claim 1, wherein the plurality of machine vision steps comprises non-executable machine vision steps.

8. The non-transitory computer accessible memory medium of claim 1, wherein the plurality of machine vision steps comprises a sequence of machine vision steps.

9. The non-transitory computer accessible memory medium of claim 1, wherein the program instructions are further executable by the processor to implement:

emulating or simulating, via the emulator, execution of the plurality of machine vision steps on the hardware accelerator, thereby generating image processing results; and

displaying, on a display, the image processing results for validation of said emulating or simulating by a user.

10. The non-transitory computer accessible memory medium of claim 1, wherein the program instructions are further executable by the processor to implement:

displaying, on a display, the estimated resource usage or performance; and

modifying, by a user, the image processing algorithm based on the displayed resource usage or performance.

11. A computer implemented method, comprising:

utilizing a computer to perform;

storing a machine vision prototype in a memory medium, the machine vision prototype comprising;

a plurality of machine vision steps, wherein the plurality of machine vision steps specifies a machine vision image processing algorithm and associated parameters, and wherein the plurality of machine vision steps is interpretable by an emulator to perform the specified image processing on an image by emulating or simulating execution of the plurality of machine vision steps on a hardware accelerator;

analyzing the machine vision prototype;

automatically generating, based on said analyzing, a graphical program implementing the specified image processing algorithm, wherein the graphical program parallelizes at least a portion of the image processing, wherein the graphical program comprises a plurality of interconnected nodes that visually indicate functionality of the graphical program, and wherein the graphical program is targeted for deployment to the hardware accelerator;

estimating resource usage or performance of execution of the graphical program by the hardware accelerator;

based on the estimated resource usage or performance, determining that the hardware accelerator is capable of executing the graphical program; and

based on the determination that the hardware accelerator is capable of executing the graphical program, deploying the graphical program to the hardware accelerator.

12. The method of claim 11, wherein said analyzing comprises:

applying one or more heuristics to determine maximum parallelism of the machine vision image processing algorithm that can be implemented on the hardware accelerator.

13. The method of claim 11, wherein said generating a graphical program comprises:

determining synchronization signals for the determined parallelism; and

generating graphical program code implementing the synchronization signals, wherein during operation on the hardware accelerator, the synchronization signals are used to synchronize the parallelized at least a portion of the image processing.

14. The method of claim 11, wherein said storing, said emulating or simulating, said displaying the image processing results, said analyzing, said estimating, and said displaying the estimated resource usage or performance, are performed prior to said generating the graphical program code.

15. The method of claim 11, wherein the hardware accelerator comprises a programmable hardware element, and wherein the program instructions are further executable by a processor to implement:

generating a hardware configuration program based on the graphical program code, wherein the hardware configuration program is deployable to the hardware accelerator for hardware implementation of the machine vision image processing algorithm.

16. The method of claim 11, wherein the hardware accelerator comprises a graphics processing unit (GPU), and wherein the program instructions are further executable by a processor to implement:

compiling the graphical program code for execution on the GPU, thereby generating an executable program that is deployable to the GPU for hardware implementation of the machine vision image processing algorithm.

17. The method of claim 11, wherein the plurality of machine vision steps comprises non-executable machine vision steps.

18. The method of claim 11, wherein the plurality of machine vision steps comprises a sequence of machine vision steps.

19. The method of claim 11, the method further comprising:

emulating or simulating, via the emulator, execution of the plurality of machine vision steps on the hardware accelerator, thereby generating image processing results; and

displaying, on a display, the image processing results for validation of said emulating or simulating by a user.

20. The method of claim 11, the method further comprising:

displaying, on a display, the estimated resource usage or performance; and

modifying, by a user, the image processing algorithm based on the displayed resource usage or performance.

21. A system, comprising:

a processor;

a memory, coupled to the processor; and

a display, coupled to the processor and memory;

wherein the memory stores program instructions executable by the processor to;

store a machine vision prototype, the machine vision prototype comprising;

a plurality of machine vision steps, wherein the plurality of machine vision steps specifies a machine vision image processing algorithm, and wherein the plurality of machine vision steps is interpretable by an emulator to perform the specified image processing on an image by emulating or simulating execution of the plurality of machine vision steps on a hardware accelerator;

analyze the machine vision prototype;

automatically generate, based on said analyzing, a graphical program implementing the specified image processing algorithm, wherein the graphical program parallelizes at least a portion of the image processing, and wherein the graphical program is targeted for deployment to the hardware accelerator;

estimate resource usage or performance of execution of the graphical program by the hardware accelerator;

based on the estimated resource usage or performance, determine that the hardware accelerator is capable of executing the graphical program; and

based on the determination that the hardware accelerator is capable of executing the graphical program, deploy the graphical program to the hardware accelerator.

22. The system of claim 21, wherein the program instructions are further executable by the processor to:

emulate or simulate, via the emulator, execution of the plurality of machine vision steps on the hardware accelerator, thereby generating image processing results; and

display, on a display, the image processing results for validation of said emulating or simulating by a user.

23. The system of claim 21, wherein the program instructions are further executable by the processor to:

display, on a display, the estimated resource usage or performance; and

modify, by a user, the image processing algorithm based on the displayed resource usage or performance.