Computer programming using tangible user interface where physical icons (phicons) indicate: beginning and end of statements and program constructs; statements generated with re-programmable phicons and stored

• US 6,175,954 B1
• Filed: 10/30/1997
• Issued: 01/16/2001
• Est. Priority Date: 10/30/1997
• Status: Expired due to Term

• Current Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation), Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation), Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Nelson, Lester David, Adams, Lia
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): INGBERG, TODD D
• Application Number:
  US08/961,020
• Time in Patent Office: 1,174 Days
• Field of Search: 395/701, 395/702, 395/703, 345/156, 345/358, 717/1, 717/2, 717/7, 717/8
• US Class Current: 717/105
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

Tangible computing is the manipulation of real, physical objects to instruct and control computers. Computer programs can be created by manipulating physical objects that represent program language elements of a program or programming language. A tangible computing system calls for observation by a computer of a real, three-dimensional work space, where each physical object can be assigned a program language element. Assignment of attributes to, and/or movement of each real physical object results in the creation of a new program component. In this manner, a user can manipulate real, physical objects to create programs. Thus, the real world itself becomes a human-computer interface that allows a user to create programs and control digital processes.

Claims

1. An apparatus for generating computer programs, comprising:

a physical space in which physical objects can be manipulated by a user;

a plurality of physical objects locatable within the physical space, each object representing at least one of an information element, an interation and an action command;

at least one sensor that observes physical manipulations on the physical objects occurring in the physical space and that generates corresponding electric signals based on the physical manipulations and the information elements, the iterations or the action commands represented by the manipulated physical objects; and

a processor that receives as inputs the electric signals, generates portions of information content corresponding to the information elements, the iterations or the action commands and stores the portions of information content for use as at least one of a computer program and a data file.

2. The apparatus of claim 1, wherein the information elements of the physical objects are assignable by the user.

3. The apparatus of claim 1, wherein the information element of a physical object is altered when the user manipulates that physical object.

4. The apparatus of claim 1, wherein the manipulations of the physical object include picking up the physical object, putting down the physical object, sliding the physical object and touching the physical object to another physical object.

5. The apparatus of claim 1, wherein the at least one sensor comprises a reference grid that senses movement of objects about the grid.

6. The apparatus of claim 1, wherein the sensor is a camera.

7. The apparatus of claim 1, wherein the physical objects actively participate in the observation.

8. The apparatus of claim 7, wherein the physical objects transmit a signal to the sensor.

9. The apparatus of claim 1, wherein each of the information elements of each of the physical objects represents a variable or a programming statement element or operator, and the information content is program code having a syntax and semantics.

10. The apparatus of claim 1, wherein the physical space is a physical symbolic representation.

11. The apparatus of claim 10, wherein the physical symbolic representation represents people, places or things.

12. The apparatus of claim 10, wherein the physical symbolic representation is used to create a narration of an event.

13. The apparatus of claim 10, wherein the physical symbolic representation is a logical situation.

14. The apparatus of claim 1, wherein the at least one sensor is a physical switch-based system.

15. The apparatus of claim 1, wherein the physical objects reassignably represent at least one of an information element, an iteration and an action command.

16. A method for generating computer programs, comprising:

placing a plurality of physical objects into a physical space, each object representing at least one of an information element, an iteration and an action command;

physically manipulating at least one of the plurality of physical objects in the physical space;

generating electric signals based on the physical manipulations and the at least one of the information element, the iteration and the action command of each at least one manipulated physical object; and

generating and storing for use as at least one of a computer program or data file, information content based on the information elements, the iterations or the action commands and the observed physical manipulations represents by the electric signals.

17. The method of claim 16, wherein the information elements are predetermined.

18. The method of claim 16, wherein physically manipulating comprises:

selecting at least a first one of the plurality of physical objects, the first physical object having at least one predetermined information element; and

repositioning the selected first physical object in the physical space.

19. The method of claim 18, wherein physically manipulating comprises:

selecting a second one of the plurality of physical objects, the second physical object having at least one predetermined information element; and

repositioning the second object in the physical space.

20. The method of claim 19, wherein repositioning of one of the first and second objects changes the at least one predetermined information element of that object.

21. The method of claim 16, further comprising generating an external representation corresponding to the physical manipulations.

22. The method of claim 21, wherein generating the external representation comprises generating and outputting an image to a visual display device.

23. The method of claim 16, wherein:

placing the physical objects in the physical space assigns at least one information element to the physical objects; and

relationships between the objects are formed based on relative locations of the physical objects within the physical space.

24. The method of claim 16, wherein each of the information elements of each of the physical objects represents a variable or a programming statement element or operator, and the information content is program code having a syntax and semantics.

25. The method of claim 16, further comprising:

a start operation that begins the observation of the physical manipulations; and

a stop operation that ends the observation of the physical manipulations.