Characterizing vision systems

• US 8,068,660 B2
• Filed: 05/27/2008
• Issued: 11/29/2011
• Est. Priority Date: 05/27/2008
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Giesen, Robert J. B., Smyth, Bruce, Keeler, Joshua B., Nair, Dinesh R., Vazquez, Nicolas
• Primary Examiner(s): Stafira; Michael P
• Application Number:
  US12/127,231
• Publication Number:
  US 20090297042A1
• Time in Patent Office: 1,281 Days
• Field of Search: 382141-154
• US Class Current: 382/141
• CPC Class Codes: G01N 21/8806 Specially adapted optical and illumination features

Abstract

System and method for characterizing vision systems. A multi-dimensional condition space is provided, each dimension representing a respective condition axis, where each point in the condition space specifies a set of conditions under which a vision system may operate. An image is provided. The condition space is sampled according to a pseudo-random sequence, e.g., a low-discrepancy sequence, to determine a plurality of test conditions usable to characterize the vision system, where each test condition corresponds to a respective set of conditions. A plurality of test images corresponding to the plurality of test conditions are generated based on the image, e.g., by applying image processing functions to the image that simulate the test conditions. A vision inspection is performed on each of the plurality of test images to generate respective test results, and the test results are analyzed to determine conditions under which the vision system operates correctly.

Claims

1. A computer-implemented method for characterizing a vision system, the method comprising:

providing a condition space comprising a plurality of dimensions, each dimension representing a respective condition axis, wherein each point in the condition space specifies a set of conditions under which a vision system may operate;

providing an image;

sampling the condition space according to a pseudo-random sequence to determine a plurality of test conditions usable to characterize the vision system, wherein each test condition corresponds to a respective set of conditions;

generating a plurality of test images corresponding to the plurality of test conditions based on the image;

performing a vision inspection on each of the plurality of test images to generate respective test results;

analyzing the test results to determine conditions under which the vision system operates correctly; and

storing the test results.

2. The method of claim 1, wherein the condition axes of the condition space comprise axes related to one or more of:

image object geometry;

camera geometry;

lighting;

focus blur;

motion blur;

or

noise.

3. The method of claim 2, wherein the condition axes related to image object geometry comprise axes for one or more of:

object scale;

object rotation;

or

object offset.

4. The method of claim 2, wherein the condition axes related to camera geometry comprise axes for one or more of:

camera position;

look at position;

camera pan;

or

camera rotation.

5. The method of claim 2, wherein the condition axes related to lighting comprise axes for one or more of:

brightness;

contrast;

or

gamma.

6. The method of claim 2, wherein the condition axes related to motion blur comprise axes for one or more of:

length of blur;

or

angle of blur.

7. The method of claim 2, wherein the condition axes related to noise comprise axes for one or more of:

noise type, comprising;

uniform, Gaussian, and/or impulse;

standard deviation;

or

amount of noise.

8. The method of claim 1, wherein said generating a plurality of test images corresponding to the plurality of test conditions based on the image comprises:

generating the plurality of test images by applying image processing functions to the image that simulate the plurality of test conditions.

9. The method of claim 8, wherein said generating a plurality of test images corresponding to the plurality of test conditions based on the image comprises:

receiving user input to a graphical user interface specifying one or more of;

test conditions to simulate;

number of samples of the condition space to use;

or

range values for one or more of the condition axes.

10. The method of claim 1, wherein said sampling the condition space, said generating the plurality of test images, and said performing a vision inspection on each of the plurality of test images comprises:

for each of a plurality of iterations;

a) sampling the condition space to generate a condition point;

b) generating a test image based on the condition point;

c) performing a vision inspection on the test image;

d) if the vision inspection fails, performing a) - c) for one or more condition points in a specified neighborhood of the condition point to determine a point for which the vision inspection passes.

11. The method of claim 1, wherein said sampling the condition space, said generating the plurality of test images, and said performing a vision inspection on each of the plurality of test images comprises:

for each of a plurality of iterations;

a) sampling the condition space to generate a condition point;

b) generating a test image based on the condition point;

c) performing a vision inspection on the test image;

d) if the vision inspection fails,

e) modifying parameters of the vision inspection;

f) performing a vision inspection on the test image using the modified parameters; and

g) performing e)-f) in an iterative manner until the vision inspection passes.

12. The method of claim 1, wherein said analyzing comprises determining one or more of:

fraction of passes of the vision inspection within the condition space;

or

a largest convex hull in the condition space specifying conditions under which vision inspections by the vision system will pass within a specified tolerance.

13. The method of claim 1, wherein the pseudo-random sequence comprises a low-discrepancy sequence.

23. The method of claim 8, wherein said generating a plurality of test images corresponding to the plurality of test conditions based on the image comprises:

receiving user input to a graphical user interface specifying one or more of;

test conditions to simulate;

number of samples of the condition space to use;

or range values for one or more of the condition axes.

14. A non-transitory computer accessible memory medium that stores program instructions for characterizing a vision system, wherein the program instructions are executable by a processor to:

provide a condition space comprising a plurality of dimensions, each dimension representing a respective condition axis, wherein each point in the condition space specifies a set of conditions

under which a vision system may operate;

provide an image;

sample the condition space according to a pseudo-random sequence to determine a plurality of test conditions usable to characterize the vision system, wherein each test condition corresponds to a respective set of conditions;

generate a plurality of test images corresponding to the plurality of test conditions based on the image;

perform a vision inspection on each of the plurality of test images to generate respective test results;

analyze the test results to determine conditions under which the vision system operates correctly.

15. A computer accessible memory medium that stores program instructions for characterizing a vision system, wherein the program instructions are executable by a processor to:

provide a condition space comprising a plurality of dimensions, each dimension representing a respective condition axis, wherein each point in the condition space specifies a set of conditions under which a vision system may operate;

provide an image;

sample the condition space according to a pseudo-random sequence to determine a plurality of test conditions usable to characterize the vision system, wherein each test condition corresponds to a respective set of conditions;

generate a plurality of test images corresponding to the plurality of test conditions based on the image;

perform a vision inspection on each of the plurality of test images to generate respective test results;

analyze the test results to determine conditions under which the vision system operates correctly.

16. The method of claim 15, wherein the condition axes of the condition space comprise axes related to one or more of:

image object geometry;

camera geometry;

lighting;

focus blur;

motion blur;

or

noise.

17. The method of claim 16, wherein the condition axes related to image object geometry comprise axes for one or more of:

object scale;

object rotation;

or

object offset.

18. The method of claim 17, wherein the condition axes related to camera geometry comprise axes for one or more of:

camera position;

look at position;

camera pan;

or

camera rotation.

19. The method of claim 16, wherein the condition axes related to lighting comprise axes for one or more of:

brightness;

contrast;

or

gamma.

20. The method of claim 16, wherein the condition axes related to motion blur comprise axes for one or more of:

length of blur;

or

angle of blur.

21. The method of claim 16, wherein the condition axes related to noise comprise axes for one or more of:

noise type, comprising;

uniform, Gaussian, and/or impulse;

standard deviation;

or

amount of noise.

22. The method of claim 15, wherein said generating a plurality of test images corresponding to the plurality of test conditions based on the image comprises:

generating the plurality of test images by applying image processing functions to the image that simulate the plurality of test conditions.

24. The method of claim 15, wherein said sampling the condition space, said generating the plurality of test images, and said performing a vision inspection on each of the plurality of test images comprises:

for each of a plurality of iterations;

a) sampling the condition space to generate a condition point;

b) generating a test image based on the condition point;

c) performing a vision inspection on the test image;

d) if the vision inspection fails, performing a) - c) for one or more condition points in a specified neighborhood of the condition point to determine a point for which the vision inspection passes.

25. The method of claim 15, wherein said sampling the condition space, said generating the plurality of test images, and said performing a vision inspection on each of the plurality of test images comprises:

for each of a plurality of iterations;

a) sampling the condition space to generate a condition point;

b) generating a test image based on the condition point;

c) performing a vision inspection on the test image;

d) if the vision inspection fails,

e) modifying parameters of the vision inspection;

f) performing a vision inspection on the test image using the modified parameters; and

g) performing e)-f) in an iterative manner until the vision inspection passes.

26. The method of claim 15, wherein said analyzing comprises determining one or more of:

fraction of passes of the vision inspection within the condition space;

or

a largest convex hull in the condition space specifying conditions under which vision inspections by the vision system will pass within a specified tolerance.

27. The method of claim 15, wherein said sampling comprises:

sampling the condition space according to a pseudo-random sequence.

28. The method of claim 27, wherein the pseudo-random sequence comprises a low-discrepancy sequence.

29. A non-transitory computer accessible memory medium that stores program instructions for characterizing a vision system, wherein the program instructions are executable by a processor to:

provide a condition space comprising three or more dimensions, each dimension representing a respective condition axis, wherein each point in the condition space specifies a set of conditions under which a vision system may operate;

provide an image;

sample the condition space to determine a plurality of test conditions usable to characterize the vision system, wherein each test condition corresponds to a respective set of conditions;

generate a plurality of test images corresponding to the plurality of test conditions based on the image;

perform a vision inspection on each of the plurality of test images to generate respective test results;

analyze the test results to determine conditions under which the vision system operates correctly; and

store the test results.

30. The non-transitory computer accessible memory medium of claim 29, wherein said sampling comprises:

sampling the condition space according to a pseudo-random sequence.

31. The non-transitory computer accessible memory medium of claim 30, wherein the pseudo-random sequence comprises a low-discrepancy sequence.

32. The non-transitory computer accessible memory medium of claim 29, wherein the condition axes of the condition space comprise axes related to one or more of:

image object geometry;

camera geometry;

lighting;

focus blur;

motion blur;

or

noise.

33. The non-transitory computer accessible memory medium of claim 29, wherein said generating a plurality of test images corresponding to the plurality of test conditions based on the image comprises:

generating the plurality of test images by applying image processing functions to the image that simulate the plurality of test conditions.

34. The non-transitory computer accessible memory medium of claim 33, wherein said generating a plurality of test images corresponding to the plurality of test conditions based on the image comprises:

receiving user input to a graphical user interface specifying one or more of;

test conditions to simulate;

number of samples of the condition space to use;

or

range values for one or more of the condition taxes.

35. The non-transitory computer accessible memory medium of claim 29, wherein said sampling the condition space, said generating the plurality of test images, and said performing a vision inspection on each of the plurality of test images comprises:

for each of a plurality of iterations;

a) sampling the condition space to generate a condition point;

b) generating a test image based on the condition point;

c) performing a vision inspection on the test image;

d) if the vision inspection fails, performing a)-c) for one or more condition points in a specified neighborhood of the condition point to determine a point for which the vision inspection passes.

36. The non-transitory computer accessible memory medium of claim 29, wherein said sampling the condition space, said generating the plurality of test images, and said performing a vision inspection on each of the plurality of test images comprises:

for each of a plurality of iterations;

a) sampling the condition space to generate a condition point;

b) generating a test image based on the condition point;

c) performing a vision inspection on the test image;

d) if the vision inspection fails,

e) modifying parameters of the vision inspection;

f) performing a vision inspection on the test image using the modified parameters; and

g) performing e)-f) in an iterative manner until the vision inspection passes.

37. The non-transitory computer accessible memory medium of claim 29, wherein said analyzing comprises determining one or more of:

fraction of passes of the vision inspection within the condition space;

or

a largest convex hull in the condition space specifying conditions under which vision inspections by the vision system will pass within a specified tolerance.