Immersive visual programming system

• US 5,758,122 A
• Filed: 03/16/1995
• Issued: 05/26/1998
• Est. Priority Date: 03/16/1995
• Status: Expired due to Fees

• Current Assignee: Navy United States of Aamerica As Represented By The Secretary of The
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Corda, Albert J., Kuchinski, Michael J.
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): Siek, Vuthe
• Application Number:
  US08/408,985
• Time in Patent Office: 1,167 Days
• Field of Search: 395/500, 395/700, 395/650, 395/966, 395/967, 395/978, 395/701, 395/704, 395/705, 395/348, 395/349, 395/127, 395/800, 345/156, 128/774
• US Class Current: 717/125
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

A method for programming and executing a computer program in an immersive sual programming environment, i.e., virtual reality environment, includes steps for assembling data objects and algorithmic components selected from a set of prefabricated virtual components to construct the computer program, compiling the computer program, and executing the computer program by initiating sequential flow of data objects through the algorithmic components. According to one aspect of the invention, the compiling step is performed while providing visual and audio clues to a user in response to a determined compiler error; the clues indicate the location and nature of the detected compiler error. Additionally, the executing step permits the flow of data objects and the interaction among the data objects to be visually displayed to the user. An apparatus suitable for performing the program and execution of a computer program in a virtual reality environment is also described.

Claims

1. A method for programming and executing a computer program in an immersive visual programming environment, said method comprising the steps of:

assembling data objects and algorithmic components selected from a set of pre-fabricated virtual components to construct the computer program;

compiling the computer program while providing visual and audio clues to a user in response to a detected compiler error, said clues indicating a location and nature of the detected compiler error, and

executing the computer program by initiating sequential flow of said data objects through said algorithmic components wherein the flow of said data objects and interaction among respective ones of said data objects is visually displayed to the user, wherein a visible execution pointer moves through the assembled algorithmic components during the execution step to indicate an instruction that is currently being performed.

2. The method for programming and executing a computer program as claimed in claim 1, further comprising the steps of providing said visual and audio clues to the user during the execution of the computer program to indicate the nature and location of a detected run-time error.

3. The method for programming and executing a computer program as claimed in claim 1, wherein said data objects each comprise a closed surface possessing a set of user-defined attributes defining the interaction of the respective closed surface with other data objects within the immersive visual programming environment.

4. The method for programming and executing a computer program as claimed in claim 3, wherein said set of attributes of said closed surface is modified in response to the interaction of the respective closed surface with the other data objects within the immersive visual programming environment.

5. The method for programming and executing a computer program as claimed in claim 1, wherein said data objects each comprise a predetermined number of nested boundary surfaces wherein each one of said nested boundary surfaces possesses a set of user-defined attributes defining the interaction of the respective nested boundary surface with other data objects within the immersive visual programming environment.

6. The method for programming and executing a computer program as claimed in claim 5, wherein said set of attributes of each one of said nested boundary surfaces is modified in response to the interaction of the respective nested boundary surface with the other data objects within the immersive visual programming environment.

7. The method for programming and executing a computer program as claimed in claim 1, further comprised of said algorithmic components each having an ingress surface and an egress surface through which said data objects enter and exit, respectively, said ingress surface and said egress surface each exhibiting varying degrees of transparency in dependence upon said data objects'"'"' varying ability to pass through each respective surface.

8. The method for programming and executing a computer program as claimed in claim 1, further comprised of said assembling step being performed by grasping and placing said pre-fabricated virtual components within the immersive visual programming environment using a data glove that provides tactile feedback in dependence upon the placement of said pre-fabricated virtual components.

9. The method for programming and executing a computer program as claimed in claim 1, wherein a single identifier data object is represented in the immersive visual programming environment by a three-dimensional, disk-like object having an identifier name on an outer surface and a value within an inner portion.

10. The method for programming and executing a computer program as claimed in claim 9, wherein said single identifier data object representative of a variable quantity is capable of being physically opened by the user during the execution of the computer program to change the value within said inner portion.

11. The method for programming and executing a computer program as claimed in claim 9, wherein each separate numeric type of said single identifier data object is indicated to the user by a separate color.

12. The method for programming and executing a computer program as claimed in claim 1, wherein said algorithmic components comprise a sequential structure visually represented by a cube having transparent lateral surfaces.

13. The method for programming and executing a computer program as claimed in claim 1, wherein said algorithmic components comprise a conditional branching structure visually represented by a cylindrical chamber, said cylindrical chamber being rotatable during the execution step to align with an execution path for passage of said data objects.

14. A method for programming and executing a computer program in an immersive visual programming environment, said method comprising the steps of:

assembling data objects and algorithmic components selected from a set of pre-fabricated virtual components to construct the computer program;

compiling the computer program while providing visual and audio clues to a user in response to a detected compiler error, said clues indicating a location and nature of the detected compiler error;

executing the computer program by initiating sequential flow of said data objects through said algorithmic components wherein said flow of said data objects and interaction among respective ones of said data objects is visually displayed to the user, said flow of said data objects being indicated by a visible execution pointer that moves through said algorithmic components during execution of the computer program to indicate an instruction that is currently being performed; and

providing one of said visual and audio clues to the user during the execution of the computer program to indicate a location and nature of a detected run-time error.

15. The method for programming and executing a computer program as claimed in claim 14, wherein said data objects each comprise a closed surface possessing a set of user-defined attributes defining the interaction of the respective closed surface with other data objects within the immersive visual programming environment.

16. The method for programming and executing a computer program as claimed in claim 15, wherein said set of attributes of said closed surface is modified in response to the interaction of the respective closed surface with the other data objects within the immersive visual programming environment.

17. The method for programming and executing a computer program as claimed in claim 14, wherein said data objects each comprise a predetermined number of nested boundary surfaces wherein each one of said nested boundary surfaces possesses a set of user-defined attributes defining the interaction of the respective nested boundary surface with other data objects within the immersive visual programming environment.

18. The method for programming and executing a computer program as claimed in claim 17, wherein said set of attributes of each one of said nested boundary surfaces is modified in response to the interaction of the respective nested boundary surface with the other data objects within the immersive visual programming environment.

19. The method for programming and executing a computer program as claimed in claim 14, further comprised of said algorithmic components each having an ingress surface and an egress surface through which said data objects enter and exit, respectively, said ingress surface and said egress surface each exhibiting varying degrees of transparency in dependence upon said data objects'"'"' varying ability to pass through each respective surface.

20. The method for programming and executing a computer program as claimed in claim 19, further comprised of said egress surface of a first one of said algorithmic components exhibiting a magnetic-like attraction towards said ingress surface of a second one of said algorithmic components during the assembly step to indicate proper syntax.

21. The method for programming and executing a computer program as claimed in claim 19, further comprised of said egress surface of a first one of said algorithmic components exhibiting a repulsion towards said ingress surface of a second one of said algorithmic components during the assembly step to indicate improper syntax.

22. The method for programming and executing a computer program as claimed in claim 14, further comprised of said assembling step being performed by grasping and placing said pre-fabricated virtual components within the immersive visual programming environment using a data glove.

23. The method for programming and executing a computer program as claimed in claim 14, wherein a single identifier data object is represented in the immersive visual programming environment by a three-dimensional, disk-like object having an identifier name on an outer surface and a value within an inner portion.

24. The method for programming and executing a computer program as claimed in claim 23, wherein said single identifier data object representative of a variable quantity is capable of being physically opened by the user during the execution of the computer program to change the value within said inner portion.

25. The method for programming and executing a computer program as claimed in claim 23, wherein each separate numeric type of said single identifier data object is indicated to the user by a separate color.

26. A method for programming and executing a computer program in an immersive visual programming environment, said method comprising the steps of:

assembling data objects and algorithmic components selected from a set of pre-fabricated virtual components to construct the computer program, said data objects each comprising a closed surface possessing a set of user-defined attributes defining interactions of the respective closed surface with other data objects within the immersive visual programming environment, said assembling step being performed by grasping and placing said pre-fabricated virtual components within the immersive visual programming environment using a data glove that provides tactile feedback in dependence upon the placement of said pre-fabricated virtual components;

compiling the computer program while providing one of visual and audio clues to the user in response to a detected compiler error, said clues indicating a location and nature of the detected compiler error;

executing the computer program by initiating sequential flow of said data objects through said algorithmic components wherein said flow of said data objects and interaction among respective ones of said data objects is visually displayed to the user, said flow of said data objects being indicated by a visible execution pointer that moves through said algorithmic components to indicate an instruction that is currently being performed; and

providing one of said visual and audio clues to the user during the execution of the computer program to indicate a location and nature of a detected run-time error.

27. The method for programming and executing a computer program as claimed in claim 26, wherein said set of attributes of said closed surface is modified in response to the interaction of the respective closed surface with the other data objects within the immersive visual programming environment.

28. The method for programming and executing a computer program as claimed in claim 26, wherein said data objects each further comprise a predetermined number of nested boundary surfaces wherein each one of said nested boundary surfaces possesses a set of user-defined attributes defining the interaction of the respective nested boundary surface with other data objects within the immersive visual programming environment.

29. The method for programming and executing a computer program as claimed in claim 26, wherein said set of attributes of each one of said nested boundary surfaces is modified in response to the interaction of the respective nested boundary surface with the other data objects within the immersive visual programming environment.

30. The method for programming and executing a computer program as claimed in claim 26, further comprised of said algorithmic components each having an ingress surface and an egress surface through which said data objects enter and exit, respectively, said ingress surface and said egress surface each exhibiting varying degrees of transparency in dependence upon said data objects'"'"' varying ability to pass through each respective surface.

31. The method for programming and executing a computer program as claimed in claim 30, further comprised of said egress surface of a first one of said algorithmic components exhibiting a magnetic-like attraction towards said ingress surface of a second one of said algorithmic components during the assembly step to indicate proper syntax.

32. The method for programming and executing a computer program as claimed in claim 30, further comprised of said egress surface of a first one of said algorithmic components exhibiting a repulsion towards said ingress surface of a second one of said algorithmic components during the assembly step to indicate improper syntax.

33. A computer programming system for creating and executing a computer program in an immersive visual programming environment, said system comprising:

means for assembling data objects and algorithmic components selected from a set of pre-fabricated virtual components to construct the computer program, said data objects each comprising a closed surface possessing a set of user-defined attributes defining interactions of the respective closed surface with other data objects within the immersive visual programming environment, said pre-fabricated virtual components being grasped and placed within the immersive visual programming environment using a data glove that provides tactile feedback in dependence upon the placement of said pre-fabricated virtual components;

means for compiling the computer program while providing one of visual and audio clues to the user in response to a detected compiler error, said clues indicating a location and nature of the detected compiler error;

means for executing the computer program by initiating sequential flow of said data objects through said algorithmic components wherein said flow of said data objects and interaction among respective ones of said data objects is visually displayed to the user, said flow of said data objects being indicated by a visible execution pointer that moves through said algorithmic components to indicate an instruction that is currently being performed; and

means for providing one of said visual and audio clues to the user during the execution of the computer program to indicate a location and nature of a detected run-time error.