VISUAL PROGRAMMING SYSTEM
Methods, systems, and system for generating code using visual blocks. A selection is made from a list of one or more programming blocks. One or more of the programming blocks are selected and used to display a visual programming block that corresponds to the selected one or more programming blocks. This visual programming block is then used to generate or produce output code.
- 1-20. -20. (canceled)
- 21. A method for defining a process on a visual programming canvas, the method comprising:
causing display of a visual programming canvas on an electronic display, wherein the visual programming canvas is configured to enable design of the process; causing display of a palette of a plurality of programming elements addable to the visual programming canvas, wherein the display of the plurality of programming elements is adjacent to the visual programming canvas and the plurality of programming elements correspond to actions to be performed within the process; receiving a selection of a first programming element of the plurality of programming elements to be executed within the process, wherein the first programming element corresponds to a first action of the actions; causing a display of a first visual element representing the first programming element on the visual programming canvas, wherein the first visual element provides access to constituent components of the first action, and wherein the constituent components include an input parameter; receiving a selection of a second programming element of the plurality of programming elements to be executed subsequent to the first programming element in the process; causing the display of a second visual element representing the second programming element on the visual programming canvas, wherein the second visual element provides access to constituent components of the second action; causing the display of a combination visual element that corresponds to two or more actions corresponding to two or more of the plurality of programming elements, wherein the combination visual element corresponds to sequential execution of the two or more actions; and executing actions corresponding to the first visual element, the second visual element, and the combination visual element.
- View Dependent Claims (22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32)
- 33. A system comprising:
one or more processors; and memory having instructions stored thereon that, when executed by the one or more processors, are configured to cause the one or more processors to; cause display of a visual programming canvas on a display; cause display of palette of a plurality of programming elements configured to be added to the visual programming canvas, wherein the plurality of programming elements correspond to one or more actions to be performed in a cloud computing system; causing the display of a combination visual element that corresponds to two or more actions corresponding to two or more of the plurality of programming elements in a single combination visual element, wherein the combination visual element corresponds to sequential execution of the two or more actions; and executing the two or more actions.
- View Dependent Claims (34, 35, 36, 37, 38, 39)
- 40. A system comprising:
one or more processors; and memory having instructions stored thereon that, when executed by the one or more processors, are configured to cause the one or more processors to; causing display of a visual programming canvas on an electronic display, wherein the visual programming canvas is configured to enable design of a process; causing the display of a first visual element representing the first programming element on the visual programming canvas, wherein the first visual element provides access to constituent components of the action; receiving a selection of a second programming element of the plurality of programming elements to be executed subsequent to the first programming element in the process, wherein the second programming element refers to an action that returns a return value; causing the display of a second visual element representing the second programming element on visual programming canvas and the return value on the visual programming canvas; causing the display of a combination visual element that corresponds to two or more actions corresponding to two or more of the plurality of programming elements in a single combination visual element, wherein the combination visual element corresponds to sequential execution of the two or more actions; determine whether a required element is missing from the first visual element, the second visual element, or the combination visual element; responsive to determining that the required element is missing, generating a visualization of a hole corresponding to the required element; and providing assistance in the visual programming canvas to fill the hole with a proper type for the required element.
This application is a continuation of U.S. patent application Ser. No. 15/939,734, filed on Mar. 29, 2018, which claims priority to U.S. patent application Ser. No. 15/595,213, filed on May 15, 2017, which claims the benefit of Provisional Application Ser. No. 62/336,681, filed May 15, 2016, entitled “Visual Programming System,” both of which are incorporated by reference herein in their entirety.
The present disclosure relates in general to techniques and devices for a visual programming system.
A computer-based visual programming system can be used to create applications and scripts by arranging blocks representing activities and functions on a canvas. Individual blocks can represent certain functionality and can be connected with other blocks.
Disclosed herein are aspects of a visual programming system and associated methods.
According to an implementation, a visual programming system is provided. The system includes a processor and a memory. The memory includes instructions executable by the processor to automatically generate display code for rendering a first visual programming block and generation code for producing output code from a definition of an interface to an application programming interface, render a plurality of visual programming blocks including the first visual programming block on a visual programming canvas responsive to user input using at least the display code of the first visual programming block, and produce output code using at least the generation code of the first visual programming block.
These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying figures.
The description herein makes reference to the accompanying drawings wherein like reference numerals refer to like parts throughout the several views.
A visual programming system may be utilized for creating applications or scripts in which constituent components are displayed as visual programming blocks (also referred to as blocks) on a visual programming canvas (also referred to as a canvas) area of a screen.
Certain of the blocks may be automatically created from class definitions on the system. Compound blocks can be produced by aggregating individual blocks and provided parameters, and then the compound blocks can, in turn, be combined with other compound blocks. Compound blocks can simplify the visual blocks displayed to a user while maintaining the functionality of the combined blocks.
Display code used to render the blocks and generation code used to produce executable code from the block can be produced automatically for an interface (e.g., class or method) provided by application programming interface (API), such as through a class definition, class reflection, static analysis of source code, run-time instrumentation of execution environments or other technique available for the determining structure and/or function of the API.
To describe some implementations in greater detail, reference is first made to examples of hardware structures and interconnections usable in implementations of the present disclosure.
Cloud computing system 100 can include any number of datacenters, including datacenter 120. Each datacenter 120 may have servers, such as servers 122. Each datacenter 120 may represent a facility in a different geographic location where servers are located. Each of servers 122 can be in the form of a computing system including multiple computing devices, or in the form of a single computing device, for example, a desktop computer, a server computer and the like. The datacenter 120 and servers 122 are examples only, and a cloud computing system may have a different number of datacenters and servers or may have a different configuration of datacenters and servers. For example, there may be tens of datacenters and each datacenter may have hundreds or any number of servers.
Clients 112 and servers 122 may be configured to connect to network 130. The clients for a particular customer may connect to network 130 via a common connection point 116 or different connection points (e.g. a wireless connection point 118 and a wired connection point 119). Any combination of common or different connections points may be present, and any combination of wired and wireless connection points may be present as well. Network 130 can be, for example, the Internet. Network 130 can also be or include a local area network (LAN), wide area network (WAN), virtual private network (VPN), or any other means of transferring data between any of clients 112 and servers 122. Network 130, datacenter 120 and/or blocks not shown may include network hardware such as routers, switches, load balancers and/or other network devices.
Other implementations of the cloud computing system 100 are also possible. For example, devices other than the clients and servers shown may be included in system 100. In an implementation, one or more additional servers may operate as a cloud infrastructure control, from which servers and/or clients of the cloud infrastructure are monitored, controlled and/or configured. For example, some or all of the techniques described herein may operate on said cloud infrastructure control servers. Alternatively, or in addition, some or all of the techniques described herein may operate on servers such as servers 122.
The computing device 200 can include a number of components, as illustrated in
Random Access Memory (RAM) 204 can be any suitable non-permanent storage device that is used as memory. RAM 204 can include executable instructions and data for access by CPU 202. RAM 204 typically comprises one or more DRAM modules such as DDR SDRAM. Alternatively, RAM 204 can include another type of device, or multiple devices, capable of storing data for processing by CPU 202 now-existing or hereafter developed. CPU 202 can access and manipulate data in RAM 204 via bus 212. The CPU 202 may utilize a cache 220 as a form of localized fast memory for operating on data and instructions.
Storage 206 can be in the form of read only memory (ROM), a disk drive, a solid-state drive, flash memory, Phase-Change Memory (PCM), or any form of non-volatile memory designed to maintain data for some duration of time, and preferably in the event of a power loss. Storage 206 can include executable instructions 206A and application files/data 206B along with other data. The executable instructions 206A can include, for example, an operating system and one or more application programs for loading in whole or part into RAM 204 (with RAM-based executable instructions 204A and application files/data 204B) and to be executed by CPU 202. The executable instructions 206A may be organized into programmable modules or algorithms, functional programs, codes, and code segments designed to perform various functions described herein. The operating system can be, for example, a Microsoft Windows®, Mac OS X®, or Linux® operating system, or can be an operating system for a small device, such as a smart phone or tablet device, or a large device, such as a mainframe computer. The application program can include, for example, a web browser, web server and/or database server. Application files 206B can, for example, include user files, database catalogs and configuration information. In an implementation, storage 206 includes instructions to perform the discovery techniques described herein. Storage 206 may comprise one or multiple devices and may utilize one or more types of storage, such as solid state or magnetic.
The computing device 200 can also include one or more input/output devices, such as a network communication unit 208 and interface 230 that may have a wired communication component or a wireless communications component 290, which can be coupled to CPU 202 via bus 212. The network communication unit 208 can utilize any of a variety of standardized network protocols, such as Ethernet, TCP/IP, or the like to effect communications between devices. The interface 230 can comprise one or more transceiver(s) that utilize the Ethernet, power line communication (PLC), WiFi, infrared, GPRS/GSM, CDMA, etc.
A user interface 210 can include a display, positional input device (such as a mouse, touchpad, touchscreen, or the like), keyboard, or other forms of user input and output devices. The user interface 210 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the client or server can be provided in addition to or as an alternative to display 210. When the output device is or includes a display, the display can be implemented in various ways, including by a liquid crystal display (LCD) or a cathode-ray tube (CRT) or light emitting diode (LED) display, such as an OLED display.
Other implementations of the internal configuration or architecture of clients and servers 200 are also possible. For example, servers may omit display 210. RAM 204 or storage 206 can be distributed across multiple machines such as network-based memory or memory in multiple machines performing the operations of clients or servers. Although depicted here as a single bus, bus 212 can be composed of multiple buses, that may be connected to each other through various bridges, controllers, and/or adapters. Computing devices 200 may contain any number of sensors and detectors that monitor the device 200 itself or the environment around the device 200, or it may contain a location identification unit 260, such as a GPS or other type of location device. The computing device 200 may also contain a power source 270, such as a battery, so that the unit can operate in a self-contained manner. These may communicate with the CPU/processor 202 via the bus 212.
The datacenter 120A includes a primary database 310A, and the datacenter 120B includes a secondary database 310B. The datacenters 120A, 120B operate in such a manner that the secondary database 310B can provide an exact or substantially exact mirror of the primary database 310A. A line 320 is used to graphically emphasize the logical boundary between datacenters 120A and 120B. Depending upon the intended application, the databases 310A, 310B may be implemented using, for example, a relational database management system (RDBMS), an object database, an XML database, flat files, or the like.
Each datacenter can include two application nodes 122A1, 122A2, 122B1, 122B2 (collectively or individually by way of example 122), although a greater or lesser number can be used depending on the implementation. The application nodes can be implemented using processing threads, virtual machine instantiations, or other computing features of the datacenters that run programs on behalf of remotely sited clients, and exchange related data with such clients via the network 130. In connection with running these programs, occasions arise for the application nodes to store and retrieve data, with the databases 310A and 310B filling this role. In an implementation, each of the application nodes connects to a single primary database, regardless of whether said database is located in the same datacenter as said application node. For example, a primary database may be read/write and a secondary database may be configured to be read-only such that it mirrors changes from the primary database. Requests to the system 300 may be routed to the application nodes in the datacenter of the primary database first, followed by the other datacenter. In a failover situation, the secondary database may become read/write with the formerly primary database switched to mirror the secondary database (which becomes the primary database). In this situation, each application node can be reconfigured to point to the secondary database (now the primary database) as shown by the dashed lines.
As mentioned above, each datacenter 120A, 120B may have its own load balancer 304A, 304B. Each load balancer may be configured to direct traffic to respective servers and processing nodes located within its datacenter. In regard to proxy services, in one example the load balancers 304A, 304B are configured to provide a single Internet-delivered service to remote clients via the network 130, where this service is actually provided by a server farm composed of the computerized servers of the datacenters 120A, 120B. The components 304A, 304B also coordinate requests from remote clients to the datacenters 120A, 120B, simplifying client access by masking the internal configuration of the datacenters. The components 304A, 304B may serve these functions by directing clients to processing nodes as configured directly or via DNS. Load balancer 304A, 304B can be configured for sticky sessions. With sticky sessions, the load balancer can attempt to forward all requests from a client to the same application node 122A1, 122A2. Different sticky session implementations are available. For example, in an implementation, a load balancer can be configured to direct all requests associated with a particular session to a same application node, so long as that node is available.
In regard to load balancing, the components 304A, 304B can be configured to direct traffic to the secondary datacenter in the event the primary datacenter 120A experiences one of many enumerated conditions predefined as failure. The load balancing functionality of the components 304A, 304B can be provided as separate components or as a single component.
A visual programming system is a tool for creating applications or scripts using graphical blocks. A requirement that blocks must be manually created for use with a visual programming system can reduce the effectiveness of these tools for larger scale development, such as in an enterprise or platform environment.
The prototype compound block 920 can be saved, and the display code and generation code of the resulting compound block can be automatically produced without having to write any further code.
As shown in
When a function or construct representation has parameters, in an implementation, in order to assist the user in getting at the arguments, the user can drag the parameters from the representation so that they can be filled in. For example, a user can drag an indicator of the loop variable to create a do block.
To generate a compound block, the system can take the code from constituent blocks and can automatically generate the compound block code. In an implementation, the compound block is first traversed to determine what elements are missing from it. Making this determination allows the system to allow the user to plug in the missing elements. The compound block is generated by putting together a number of blocks, and then determining what the inputs are and what the outputs are to that group of blocks to create the compound block.
If the compound block is assigned a return value, that can become its output. If there is no return value, then the compound block may not have an output. Any time, however, that a hole is left in a constituent block, a hole is created for the compound block that will require a user to fill it in. The system also allows renaming of the compound block, since the original name might not be meaningful to an external user who cannot see the internal details of the compound block. Thus, in addition to the automatic generation of code based on class definitions to create API points, the system, according to an implementation, can perform the automatic generation of code for compound blocks.
As previously described, there can be two stages for creating the block—producing display code and producing generation code. The first stage of producing display code permits the rendering of the physical representation of the block that can be placed on a canvas and that informs a user where the holes are that require something else to be plugged in. If there is a type to those holes, assistance can be provided so that the wrong type of data is not used in a particular hole.
In one implementation, a basic block may be generated that can expose the entire functionality of a given API and can permit the creation of subsequent compound blocks. The subsequently created compound blocks may be utilized to expose certain desirable inputs or outputs of the basic block for particular use cases. For example, a compound block can be created from a glide record (database access) block that is configured to query for all records matching criteria x and y from a predefined table (e.g. an incident table). Such a compound block can greatly simplify the use of the visual programming system by replacing a large number of possible inputs and outputs and reducing those inputs and outputs to two—criteria x and y.
When compound blocks are created, the scoping of variables can be addressed to avoid collisions between variables within and outside of the compound block. Compound blocks can create new scope because the inside of the compound block is not visible, and the inside of a compound block can be quite complex.
In an implementation, a two-pass system may be used where initially the first pass generator produces source code, and then this source code is parsed to resolve variable conflicts and scoping issues (i.e., patching up the variables) to produce final source code. This final source code can then be compiled or interpreted to have executable code. However, in another implementation, the scoping can be performed without going in and patching up the variables. By way of example, a user writes code defining variables A through D, and the user wants to use a compound block. The compound block opens a record, scans it, and then allows the user to perform functions like ask “what is the value of name”, “what is the value of an employee identification number”, etc., on the record. The visualization of the compound block has a hole for a statement input, and floating above it are the variables that are available for the user to use in the record. The user defines the variables A through D above the compound block context, but these variables need to be input into the compound block. But the compound block also has its own variables for the record that are found within the scope of the compound block. By ensuring that the variable names do not collide, it is possible to perform a stacking of scopes on top of each other.
The avoiding of variable name collisions when combining blocks may be achieved by automatically renaming variables so that they are globally unique. Thus, when a first compound block is dropped into a second compound block, there are two different workspaces, and the internals are changed to prevent name conflicts.
In a first way to avoid collisions, a global list of all variables can be maintained. By way of example, a scoped variable, like a glide record, can require each query to be handled in a particular way. If a user adds many of these queries to a block, that glide record should be renamed for each of the queries. A new variable name is created for each instance (e.g., glide record A, glide record B). There is a mapping that indicates which (and how) each scoped variable is renamed. The system can keep track of the order that the scoped variable is declared at each of the statement inputs, so it can access its scoped variables and determine their original names. Thus, the system can determine which names need to be injected so as to keep variables from colliding in different scopes. In a second way to avoid collisions, self-calling functions may be utilized (as in the nested code example of
All or a portion of aspects of the invention described herein can be implemented using a general-purpose computer/processor with a computer program that, when executed, carries out any of the respective techniques, algorithms and/or instructions described herein. In addition, or alternatively, for example, a special-purpose computer/processor can be utilized which can contain specialized hardware for carrying out any of the techniques, algorithms, or instructions described herein.
The implementations of computing devices as described herein (and the algorithms, methods, instructions, etc., stored thereon and/or executed thereby) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application-specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination.
For example, one or more computing devices can include an ASIC or programmable logic array such as a field-programmable gate array (FPGA) configured as a special-purpose processor to perform one or more of the operations or operations described or claimed herein. An example FPGA can include a collection of logic blocks and random access memory (RAM) blocks that can be individually configured and/or configurably interconnected in order to cause the FPGA to perform certain functions. Certain FPGA'"'"'s may contain other general or special-purpose blocks as well. An example FPGA can be programmed based on a hardware definition language (HDL) design, such as VHSIC Hardware Description Language or Verilog.
The embodiments herein may be described in terms of functional block components and various processing operations. Such functional blocks may be realized by any number of hardware and/or software components that perform the specified functions. For example, the described embodiments may employ various integrated circuit components (e.g., memory elements, processing elements, logic elements, look-up tables, and the like) which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, where the elements of the described embodiments are implemented using software programming or software elements the invention may be implemented with any programming or scripting language such as C, C++, Java, assembler, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines or other programming elements. Functional aspects may be implemented in algorithms that execute on one or more processors. Furthermore, the embodiments of the invention could employ any number of conventional techniques for electronics configuration, signal processing and/or control, data processing and the like. The words “mechanism” and “element” are used broadly and are not limited to mechanical or physical embodiments, but can include software routines in conjunction with processors, etc.
Implementations or portions of implementations of the above disclosure can take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport a program or data structure for use by or in connection with any processor. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or a semiconductor device. Other suitable mediums are also available. Such computer-usable or computer-readable media can be referred to as non-transitory memory or media, and may include RAM or other volatile memory or storage devices that may change over time. A memory of an apparatus described herein, unless otherwise specified, does not have to be physically contained by the apparatus, but is one that can be accessed remotely by the apparatus, and does not have to be contiguous with other memory that might be physically contained by the apparatus.
The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise, or clear from context, “X includes A or B” is intended to mean any of the natural inclusive permutations. In other words, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. Moreover, use of the term “an implementation” or “one implementation” throughout is not intended to mean the same embodiment or implementation unless described as such.
The particular implementations shown and described herein are illustrative examples of the invention and are not intended to otherwise limit the scope of the invention in any way. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems (and components of the individual operating components of the systems) may not be described in detail. Furthermore, the connecting lines, or connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the invention unless the element is specifically described as “essential” or “critical”.
The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless specified or limited otherwise, the terms “mounted,” “connected,” “supported,” and “coupled” and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings. Further, “connected” and “coupled” are not restricted to physical or mechanical connections or couplings.
The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention (especially in the context of the following claims) should be construed to cover both the singular and the plural. Furthermore, recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. Finally, the operations of all methods described herein are performable in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed.
All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated as incorporated by reference and were set forth in its entirety herein.
The above-described embodiments have been described in order to allow easy understanding of the present invention and do not limit the present invention. To the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structure as is permitted under the law.