Displaying operations in an application using a graphical programming representation

• US 20030227483A1
• Filed: 06/10/2002
• Published: 12/11/2003
• Est. Priority Date: 06/10/2002
• Status: Active Grant

• Current Assignee: Nationalinstruments Corporation
• Original Assignee: Nationalinstruments Corporation
• Inventors: Vazquez, Nicolas, Schultz, Kevin L.
• Granted Patent: US 7,139,979 B2
• Time in Patent Office: Days
• US Class Current: 715/763
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Operations in an application may be displayed using a graphical programming representation. A plurality of interconnected icons may be displayed, where each icon corresponds to an operation included in the application. The plurality of interconnected icons may visually indicate a function implemented by operations in the application. Each displayed icon may correspond to a node in a graphical programming development environment.

Claims

1. A computer-implemented method for displaying operations of an application, the method comprising:

including a plurality of operations in the application in response to user input, wherein the plurality of operations implement a first function;

displaying a plurality of interconnected icons in response to said including the plurality of operations in the application, wherein each icon corresponds to an operation in the application, wherein the plurality of interconnected icons visually indicate the first function; and

storing information representing the application in a data structure, wherein said storing the information comprises storing information specifying the plurality of operations in the application.

2. The method of claim 1,

wherein said including the plurality of operations in the application comprises receiving user input individually selecting each operation for inclusion in the application;

wherein said displaying a plurality of interconnected icons comprises displaying at least one icon after each instance of said user input including one of said operations in the application.

3. The method of claim 1, wherein said including and said displaying comprise:

including a first operation in the application in response to user input;

displaying a first icon corresponding to the first operation in response to said including the first operation in the application;

including a second operation in the application in response to user input; and

displaying a second icon corresponding to the second operation in response to said including the second operation in the application; and

displaying a connection between the first icon and the second icon.

4. The method of claim 1,

wherein said displaying the plurality of interconnected icons comprises displaying a plurality of lines interconnecting the icons, wherein each line connects at least two icons;

wherein the plurality of lines indicate a relationship of the icons.

5. The method of claim 4,

wherein the plurality of lines includes a first line that connects a first icon to a second icon;

wherein the first line indicates that the operation corresponding to the first icon is performed before the operation corresponding to the second icon.

6. The method of claim 4,

wherein the plurality of lines include a first line that connects a first icon to a second icon;

wherein the first line indicates that data produced by the operation corresponding to the first icon is used by the operation corresponding to the second icon.

7. The method of claim 1,

wherein said displaying the plurality of interconnected icons comprises displaying a graphical programming representation of the plurality of operations in the application.

8. The method of claim 1,

wherein each icon corresponds to a node in a graphical programming development environment.

9. The method of claim 1,

wherein the plurality of interconnected icons comprise a graphical program, wherein the graphical program is executable to perform the first function.

10. The method of claim 9, further comprising:

including one or more additional nodes in the graphical program in response to user input, wherein the one or more additional nodes do not correspond to operations from the plurality of operations in the application.

11. The method of claim 1, further comprising:

programmatically generating a graphical program based on the application;

wherein said programmatically generating the graphical program comprises programmatically including a plurality of interconnected nodes in the graphical program, wherein each node corresponds to an operation from the plurality of operations in the application;

wherein the plurality of interconnected nodes implement the first function.

12. The method of claim 11,

wherein each of the nodes has an iconic appearance that is substantially identical to the icon corresponding to the operation to which the node corresponds.

13. The method of claim 11, further comprising:

displaying the plurality of interconnected nodes in the graphical program.

wherein the plurality of interconnected nodes in the graphical program appear substantially identical to the plurality of interconnected icons.

14. The method of claim 13,

wherein said displaying the plurality of interconnected icons comprises displaying the plurality of interconnected icons in a first window;

wherein said displaying the plurality of interconnected nodes in the graphical program comprises displaying the graphical program in a second window.

15. The method of claim 11,

wherein said programmatically generating the graphical program further comprises programmatically including one or more additional nodes in the graphical program that do not correspond to operations from the plurality of operations in the application.

16. The method of claim 11, further comprising:

executing the graphical program;

wherein said executing the graphical program comprises executing the plurality of interconnected nodes to perform the first function.

17. The method of claim 1,

wherein said including the plurality of operations in the application comprises receiving user input individually selecting each operation for inclusion in the application;

wherein said displaying the plurality of interconnected icons comprises programmatically including a node in a graphical program in response to each operation selected for inclusion in the application, wherein each node is operable to perform the respective operation when the graphical program is executed.

18. The method of claim 1,

wherein said including the plurality of operations in the application comprises receiving user input individually selecting each operation for inclusion in the application;

wherein the method further comprises programmatically including a node in a graphical program in response to each operation selected for inclusion in the application, wherein each node is operable to perform the respective operation when the graphical program is executed.

19. The method of claim 18,

wherein each node included in the graphical program in response to selecting an operation has an iconic appearance that is substantially identical to the icon corresponding to the operation.

20. The method of claim 18, further comprising:

removing a first operation from the application in response to user input; and

programmatically removing the node operable to perform the first operation from the graphical program in response to said removing the first operation from the application.

21. The method of claim 1, further comprising:

removing a first operation from the application, wherein the first operation corresponds to a first icon; and

discontinuing display of the first icon in response to said removing the first operation from the application.

22. The method of claim 1,

wherein said including the plurality of operations in the application comprises receiving user input individually selecting each operation for inclusion in the application;

wherein the method further comprises executing each operation as the operation is selected for inclusion in the application.

23. The method of claim 22, further comprising:

for each operation, displaying results of said executing the operation, wherein said displaying the results of said executing the operation comprises displaying the results together with the icon that corresponds to the operation.

24. The method of claim 23,

wherein said displaying the results together with the icon that corresponds to the operation comprises displaying a pass/fail result together with the icon that corresponds to the operation.

25. The method of claim 24,

wherein the plurality of operations includes a first operation;

wherein executing the first operation produces a fail result;

wherein displaying the pass/fail result together with the icon that corresponds to the first operation comprises displaying information indicating the fail result;

wherein the method further comprises displaying information indicating a reason for the fail result.

26. The method of claim 23,

wherein one or more of the plurality of operations in the application are operable to perform a test of a unit under test (UUT);

wherein, for each operation operable to perform a test of the UUT, said displaying the results together with the icon that corresponds to the operation comprises displaying a pass/fail result together with the icon that corresponds to the operation.

27. The method of claim 1, further comprising:

for each operation, displaying a name of the operation together with the icon that corresponds to the operation.

28. The method of claim 1, further comprising:

displaying a graphical user interface that provides access to a set of operations; and

receiving user input to the graphical user interface requesting inclusion of the plurality of operations in the application, wherein the plurality of operations are selected from the set of operations.

29. The method of claim 28,

wherein said receiving user input to the graphical user interface requesting inclusion of the plurality of operations in the application does not include receiving user input specifying programming language code to implement the plurality of operations.

30. The method of claim 1, further comprising:

executing the application;

wherein said executing the application comprises executing the plurality of operations in the application to perform the first function.

31. The method of claim 30, further comprising:

highlighting execution of the interconnected icons on the display to visually indicate progression of execution of the operations in the application.

32. The method of claim 30,

wherein the plurality of operations comprises a plurality of machine vision operations, wherein the plurality of machine vision operations implement a machine vision function;

wherein the plurality of interconnected icons visually indicate the machine vision function;

wherein said executing the plurality of operations in the application comprises performing the machine vision function.

33. The method of claim 32,

wherein said performing the machine vision function includes one or more of:

acquiring one or more images; and

analyzing acquired images.

34. The method of claim 30,

wherein the plurality of operations comprises a plurality of motion control operations, wherein the plurality of motion control operations implement a motion control function;

wherein the plurality of interconnected icons visually indicate the motion control function;

wherein said executing the plurality of operations in the application comprises performing the motion control function.

35. The method of claim 34,

wherein said performing the motion control function includes one or more of:

controlling motion of a device; and

controlling a device to move an object.

36. The method of claim 30,

wherein the plurality of operations comprises a plurality of data acquisition (DAQ) operations, wherein the plurality of DAQ operations implement a DAQ function;

wherein the plurality of interconnected icons visually indicate the DAQ function;

wherein said executing the plurality of operations in the application comprises performing the DAQ function.

37. The method of claim 36,

wherein said performing the DAQ function includes acquiring measurement data from a device.

38. The method of claim 1, further comprising:

receiving user input for configuring one or more of the plurality of operations in the application;

wherein said receiving user input for configuring one or more of the plurality of operations in the application does not include receiving user input specifying programming language code to configure the operations;

wherein, for each operation, said configuring the operation affects functionality of the operation.

39. The method of claim 38, further comprising:

for each operation to be configured,

displaying a graphical panel including graphical user interface elements for setting properties of the operation; and

receiving user input to the graphical panel to set one or more properties of the operation.

40. The method of claim 1, further comprising:

displaying a text view of the plurality of operations in the application.

41. The method of claim 40,

wherein said displaying the text view of the plurality of operations in the application comprises displaying a table including text that specifies the plurality of operations in the application.

42. A computer-implemented method for displaying operations of an application, the method comprising:

including a first operation in the application in response to first user input;

displaying a first icon corresponding to the first operation in response to said including the first operation in the application;

including a second operation in the application in response to second user input; and

displaying a second icon corresponding to the second operation in response to said including the second operation in the application;

displaying a connection between the first icon and the second icon;

wherein the interconnected first icon and second icon form a block diagram visual representation of the application comprising the first and second operations.

43. The method of claim 42, further comprising:

wherein said first user input comprises applying the first operation to an object, wherein the first operation is included in the application in response to said applying the first operation to the object; and

wherein said second user input comprises applying the second operation to the object, wherein the second operation is included in the application in response to said applying the second operation to the object.

44. The method of claim 42, further comprising:

applying the first operation to an object in response to said including the first operation in the application; and

applying the second operation to the object in response to said including the second operation in the application.

45. The method of claim 42,

wherein the first icon corresponds to a first node in a graphical programming development environment;

and

wherein the second icon corresponds to a second node in the graphical programming development environment.

46. The method of claim 42, further comprising:

displaying a graphical user interface which is useable for selecting the operations;

receiving said first user input to the graphical user interface selecting the first operation, wherein said including the first operation in the application is performed in response to said first user input;

receiving said second user input to the graphical user interface selecting the second operation, wherein said including the second operation in the application is performed in response to said second user input.

47. The method of claim 46,

wherein the graphical user interface displays an object;

wherein said first user input comprises applying the first operation to the object,

wherein the first operation is included in the application in response to said applying the first operation to the object; and

wherein said second user input comprises applying the second operation to the object, wherein the second operation is included in the application in response to said applying the second operation to the object.

48. The method of claim 47, further comprising:

modifying the object on the display after each of said applying the first operation to the object and said applying the second operation to the object.

49. The method of claim 46,

wherein the graphical user interface is displayed in a first window;

wherein the first icon and the second icon are displayed in a second window.

50. The method of claim 42, further comprising:

executing a prototyping environment, wherein said executing includes displaying a graphical user interface which is useable for selecting the operations;

wherein said first user input and said second user input are received to the graphical user interface of the prototyping environment.

51. A computer-implemented method for visually depicting an application which performs a function, the method comprising:

displaying an object on a display;

performing a first operation on the object in response to user input;

displaying a first icon corresponding to the first operation, wherein said displaying the first icon is performed in response to said performing the first operation on the object;

performing a second operation on the object in response to user input;

displaying a second icon corresponding to the second operation, wherein said displaying the second icon is performed in response to said performing the second operation on the object; and

displaying a connection between the first icon and the second icon.

52. The method of claim 51,

wherein the connection visually indicates an ordering of execution of the first operation and the second operation.

53. The method of claim 51,

wherein the first icon, the second icon, and the connection have the appearance of a graphical program.

54. The method of claim 51,

wherein the first icon has an appearance corresponding to the first operation;

wherein the second icon has an appearance corresponding to the second operation.

55. The method of claim 51,

wherein the first icon, the second icon, and the connection displayed on the display operate to visually depict the first and second operations performed by the user.

56. The method of claim 51, further comprising:

including the first operation in the application in response to said performing the first operation on the object; and

including the second operation in the application in response to said performing the second operation on the object.

57. The method of claim 51,

wherein the first icon and the second icon originate from a graphical programming development environment.

58. The method of claim 51,

wherein the first icon, the second icon, and the connection displayed between the first icon and the second icon comprise a graphical program, wherein the graphical program is executable to implement the first and second operations.

59. The method of claim 51,

wherein the first icon and the second icon are displayed in an order from left to right to visually indicate an ordering of the first operation and the second operation.

60. The method of claim 51, further comprising:

performing a third operation on the object in response to user input;

displaying a third icon corresponding to the third operation, wherein said displaying the third icon is performed in response to said performing the third operation on the object;

displaying a second connection between the second icon and the third icon.

61. The method of claim 51,

wherein the object is an image;

wherein the first operation and the second operation are image processing operations applied to the image

62. A computer-implemented method for creating an application which performs a function, the method comprising:

displaying an object on a display;

performing a first operation on the object in response to user input;

displaying a first icon corresponding to the first operation, wherein said displaying the first icon is performed in response to said performing the first operation on the object;

performing a second operation on the object in response to user input; and

displaying a second icon corresponding to the second operation, wherein said displaying the second icon is performed in response to said performing the second operation on the object.

63. The method of claim 62,

wherein the first icon has an appearance corresponding to the first operation;

and

wherein the second icon has an appearance corresponding to the second operation.

64. The method of claim 62, further comprising:

storing information regarding the first operation in a data structure in response to said performing the first operation; and

storing information regarding the second operation in the data structure in response to said performing the second operation.

65. A computer-implemented method for displaying operations of a machine vision application, the method comprising:

including a plurality of machine vision operations in the application in response to user input, wherein the plurality of machine vision operations implement a machine vision function;

displaying a plurality of interconnected icons in response to said including the plurality of machine vision operations in the application, wherein each icon corresponds to a machine vision operation in the application, wherein the plurality of interconnected icons visually indicate the machine vision function; and

storing information representing the application in a data structure, wherein said storing the information comprises storing information specifying the plurality of machine vision operations in the application.

66. The method of claim 65,

wherein said including the plurality of machine vision operations in the application comprises receiving user input individually selecting each machine vision operation for inclusion in the application;

wherein said displaying a plurality of interconnected icons comprises displaying at least one icon after each instance of said user input including one of said machine vision operations in the application.

67. The method of claim 65, wherein said including and said displaying comprise:

including a first machine vision operation in the application in response to user input;

displaying a first icon corresponding to the first machine vision operation in response to said including the first machine vision operation in the application;

including a second machine vision operation in the application in response to user input; and

displaying a second icon corresponding to the second machine vision operation in response to said including the second machine vision operation in the application; and

displaying a connection between the first icon and the second icon.

68. The method of claim 65,

wherein said displaying the plurality of interconnected icons comprises displaying a plurality of lines interconnecting the icons, wherein each line connects two icons.

69. The method of claim 68,

wherein the plurality of lines includes a first line that connects a first icon to a second icon;

wherein the first line indicates that performing the machine vision function includes performing the machine vision operation corresponding to the first icon before performing the machine vision operation corresponding to the second icon.

70. The method of claim 68,

wherein the plurality of lines includes a first line that connects a first icon to a second icon;

wherein the first line indicates that data produced by the machine vision operation corresponding to the first icon is used by the machine vision operation corresponding to the second icon.

71. The method of claim 65,

wherein said displaying the plurality of interconnected icons comprises displaying a graphical programming representation of the plurality of machine vision operations in the application.

72. The method of claim 65,

wherein each icon corresponds to a node in a graphical programming development environment.

73. The method of claim 65, further comprising:

programmatically generating a graphical program based on the application;

wherein said programmatically generating the graphical program comprises programmatically including a plurality of interconnected nodes in the graphical program,

wherein each node corresponds to a machine vision operation from the plurality of machine vision operations in the application;

wherein the plurality of interconnected nodes implement the machine vision function.

74. The method of claim 73,

wherein said programmatically generating the graphical program comprises automatically generating the graphical program.

75. The method of claim 73,

wherein, for each node, the node has an iconic appearance that is identical to the icon corresponding to the machine vision operation to which the node corresponds.

76. The method of claim 73,

wherein, for each node, the node has an iconic appearance that is substantially the same as the icon corresponding to the machine vision operation to which the node corresponds.

77. The method of claim 73, further comprising:

displaying the plurality of interconnected nodes in the graphical program.

78. The method of claim 77,

wherein the plurality of interconnected nodes in the graphical program appears identical to the plurality of interconnected icons.

79. The method of claim 77,

wherein the plurality of interconnected nodes in the graphical program appears substantially the same as the plurality of interconnected icons.

80. The method of claim 77,

wherein said displaying the plurality of interconnected icons comprises displaying the plurality of interconnected icons in a first window;

wherein said displaying the plurality of interconnected nodes in the graphical program comprises displaying the graphical program in a second window.

81. The method of claim 73, further comprising:

executing the graphical program;

wherein said executing the graphical program comprises executing the plurality of interconnected nodes to perform the machine vision function.

82. The method of claim 65,

wherein said including the plurality of machine vision operations in the application comprises receiving user input individually selecting each machine vision operation for inclusion in the application;

wherein the method further comprises programmatically including a node in a graphical program in response to each machine vision operation selected for inclusion in the application, wherein each node is operable to perform the respective machine vision operation when the graphical program is executed.

83. The method of claim 82,

wherein each node included in the graphical program in response to selecting a machine vision operation has an iconic appearance that is identical to the icon corresponding to the machine vision operation.

84. The method of claim 82,

wherein each node included in the graphical program in response to selecting a machine vision operation has an iconic appearance that is substantially the same as the icon corresponding to the machine vision operation.

85. The method of claim 82, further comprising:

removing a first machine vision operation from the application; and

programmatically removing the node operable to perform the first machine vision operation from the graphical program in response to said removing the first machine vision operation from the application.

86. The method of claim 65,

wherein said including the plurality of machine vision operations in the application comprises receiving user input individually selecting each machine vision operation for inclusion in the application;

wherein the method further comprises executing each machine vision operation as the machine vision operation is selected for inclusion in the application.

87. The method of claim 86, further comprising:

for each machine vision operation, displaying results of said executing the machine vision operation, wherein said displaying the results of said executing the machine vision operation comprises displaying the results together with the icon that corresponds to the machine vision operation.

88. The method of claim 87,

wherein said displaying the results together with the icon that corresponds to the machine vision operation comprises displaying a pass/fail result together with the icon that corresponds to the machine vision operation.

89. The method of claim 88,

wherein the plurality of machine vision operations includes a first machine vision operation;

wherein executing the first machine vision operation produces a fail result;

wherein displaying the pass/fail result together with the icon that corresponds to the first machine vision operation comprises displaying information indicating the fail result;

wherein the method further comprises displaying information indicating a reason for the fail result.

90. The method of claim 87,

wherein one or more of the plurality of machine vision operations in the application are operable to analyze an image of a unit under test (UUT);

wherein, for each machine vision operation operable to analyze an image of the UUT, said displaying the results together with the icon that corresponds to the machine vision operation comprises displaying a pass/fail result of the analysis together with the icon that corresponds to the machine vision operation.

91. The method of claim 86, further comprising:

displaying an image;

wherein said executing each machine vision operation comprises applying the machine vision operation to the displayed image;

wherein the method further comprises re-displaying the image after applying each machine vision operation to the image.

92. The method of claim 65, further comprising:

for each machine vision operation, displaying a name of the machine vision operation together with the icon that corresponds to the machine vision operation.

93. The method of claim 65, further comprising:

executing the application;

wherein said executing the application comprises executing the plurality of machine vision operations in the application to perform the machine vision function.

94. The method of claim 93,

wherein said performing the machine vision function includes analyzing one or more images.

95. The method of claim 93,

wherein said performing the machine vision function includes acquiring one or more images for analysis.

96. A computer-implemented method for displaying operations of a machine vision application, the method comprising:

including a first machine vision operation in the machine vision application;

displaying a first icon corresponding to the first machine vision operation in response to said including the first machine vision operation in the machine vision application;

including a second machine vision operation in the machine vision application; and

displaying a second icon corresponding to the second machine vision operation in response to said including the second machine vision operation in the machine vision application.

97. The method of claim 96, further comprising:

displaying a connection between the first icon and the second icon.

98. The method of claim 96, further comprising:

displaying an image;

applying the first machine vision operation to the image in response to said including the first machine vision operation in the machine vision application; and

applying the second machine vision operation to the image in response to said including the second machine vision operation in the machine vision application.

99. The method of claim 96, further comprising:

displaying an image;

applying the first machine vision operation to the image in response to user input, wherein the first machine vision operation is included in the machine vision application in response to said applying the first machine vision operation to the image; and

applying the second machine vision operation to the image in response to user input, wherein the second machine vision operation is included in the machine vision application in response to said applying the second machine vision operation to the image.

100. The method of claim 96,

wherein the first icon corresponds to a first node in a graphical programming development environment;

and

wherein the second icon corresponds to a second node in the graphical programming development environment.

101. The method of claim 96, further comprising:

displaying first results of executing the first machine vision operation, wherein said displaying first results of executing the first machine vision operation comprises displaying the first results together with the first icon; and

displaying second results of executing the second machine vision operation, wherein said displaying second results of executing the second machine vision operation comprises displaying the second results together with the second icon.

102. The method of claim 101,

wherein said displaying the first results together with the first icon comprises displaying a first pass/fail result together with the first icon;

and

wherein said displaying the second results together with the second icon comprises displaying a second pass/fail result together with the second icon.

103. The method of claim 96, further comprising:

displaying a name of the first machine vision operation together with the first icon; and

displaying a name of the second machine vision operation together with the second icon.

104. A computer-implemented method for creating an application which performs a machine vision function, the method comprising:

displaying an image on a display;

performing a first machine vision operation on the image in response to user input;

displaying a first icon corresponding to the first machine vision operation, wherein said displaying the first icon is performed in response to said performing the first machine vision operation on the image;

performing a second machine vision operation on the image in response to user input;

displaying a second icon corresponding to the second machine vision operation, wherein said displaying the second icon is performed in response to said performing the second machine vision operation on the image; and

displaying a connection between the first icon and the second icon.

105. The method of claim 104,

wherein the first icon, the second icon, and the connection have the appearance of a graphical program.

106. The method of claim 104,

wherein the first icon, the second icon, and the connection have the appearance of a graphical data flow program.

107. The method of claim 104,

wherein the first icon has an appearance corresponding to the first machine vision operation;

wherein the second icon has an appearance corresponding to the second machine vision operation.

108. The method of claim 104,

wherein the first icon, the second icon, and the connection displayed on the display operate to visually depict the first E nd second machine vision operations performed by the user.

109. The method of claim 104, further comprising:

including the first machine vision operation in the application in response to said performing the first machine vision operation on the image; and

including the second machine vision operation in the application in response to said performing the second machine vision operation on the image.

110. The method of claim 104,

wherein the first icon and the second icon originate from a graphical programming development environment.

111. The method of claim 104,

wherein the first icon, the second icon, and the connection displayed between the first icon and the second icon comprise a graphical program, wherein the graphical program is executable to implement the first and second machine vision operations.

112. The method of claim 104,

wherein the first icon and the second icon are displayed in an order from left to right to visually indicate an ordering of the first machine vision operation and the second machine vision operation.

113. The method of claim 104, further comprising:

performing a third machine vision operation on the image in response to user input;

displaying a third icon corresponding to the third machine vision operation, wherein said displaying the third icon is performed in response to said performing the third machine vision operation on the image; and

displaying a second connection between the second icon and the third icon.

114. A computer-implemented method for creating an application which performs a machine vision function, the method comprising:

displaying an image on a display;

performing a first machine vision operation on the image in response to user input;

displaying a first icon corresponding to the first machine vision operation, wherein said displaying the first icon is performed in response to said performing the first machine vision operation on the image;

performing a second machine vision operation on the image in response to user input; and

displaying a second icon corresponding to the second machine vision operation, wherein said displaying the second icon is performed in response to said performing the second machine vision operation on the image.

115. The method of claim 114,

wherein the first icon has an appearance corresponding to the first machine vision operation;

and

wherein the second icon has an appearance corresponding to the second machine vision operation.