Method and system for state-based control of objects

• US 9,128,519 B1
• Filed: 04/15/2005
• Issued: 09/08/2015
• Est. Priority Date: 04/15/2005
• Status: Active Grant

• Current Assignee: Intellectual Ventures Holding 81 LLC (Intellectual Ventures LLC)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Coates, Malik, Belfer, Russell H., Fink, Steve, D'India, John Paul
• Primary Examiner(s): DWIVEDI, MAHESH H
• Application Number:
  US11/106,898
• Time in Patent Office: 3,798 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 9/5033 considering data affinity
• CPC Class Codes: G09F 19/18 involving the use of optical projection means, e.g. projection of images on clouds projection apparatus per se G03B
• CPC Class Codes: G09F 19/22 Advertising or display means on roads, walls or similar surfaces, e.g. illuminated
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel

Abstract

A method and system for state-based control of objects. A simulator controls data associated with a plurality of objects, the simulator including a first list of the plurality of objects and a data structure comprising data defining the plurality of objects. A state manager includes a plurality of states, wherein a state includes a second list of a subset of objects of the plurality of objects and determines a display appearance and a display action of the subset of objects. A plurality of software nodes are operable to perform associated operations on subsets of objects and to update data of the data structure for the subset of the objects.

Claims

1. A method for state-based control of objects in an interactive video display system including a camera based interface, the method comprising:

storing an object list comprising a plurality of objects;

storing a data structure comprising properties associated with respective objects, wherein the properties indicate display actions or display appearances of respective objects;

defining a state to include a subset of objects of the object list that are selected by user input;

associating a first software node with the state;

associating a second software node with the state;

in response to receiving a first signal indicating execution of the first software node, executing the first software node on the subset of objects of the state to modify one or more properties of the subset of objects in the state stored in the data structure, wherein the first signal is representative of a first user interaction with an interactive video display system;

in response to receiving a second signal indicating execution of the second software node, executing the second software node on the subset of objects of the state to modify one or more properties of the subset of objects in the state stored in the data structure, wherein the second signal is representative of a second user interaction with the interactive video display system; and

rendering the objects that have been modified by the software nodes on a display device of the interactive video display system.

2. The method of claim 1, further comprising executing a third software node to create a new object based on a property of another object.

3. The method of claim 1, further comprising executing a third software node to delete an object based on a property of another object.

4. The method of claim 1, wherein the first software node calculates a force to be applied to each of the objects indicated by the state based at least in part on the first user interaction with the interactive video display system.

5. The method of claim 1, wherein the first software node calculates an alteration of position or velocity to be applied to each of the objects indicated by the state based at least in part on the first user interaction with the interactive video display system.

6. The method of claim 1, wherein the first signal is further representative of a timing signal that controls the execution of the first software node.

7. A computerized method for controlling objects in an interactive video display system, the method comprising:

storing properties of a plurality of objects in memory, the plurality of objects appearing on an interactive video display;

receiving user interaction data indicating one or more interactions of a user with the interactive video display;

associating a first subset of the plurality of objects with a first software node based at least in part on the user interaction data, wherein the first subset comprises two or more objects;

associating a second subset of the plurality of objects with a second software node based at least in part on the user interaction data, wherein the second subset comprises two or more objects, including at least one object that is in the first subset of objects;

executing the first software node on each of the objects in the first subset of objects at a computing device, wherein execution of the first software node changes at least a first property of each of the objects in the first subset of objects;

executing the second software node on each of the objects in the second subset of objects at a computing device, wherein execution of the second software node changes at least a second property of each of the objects in the second subset of objects; and

initiating display of the subset of objects on the interactive video display.

8. The method of claim 7, wherein the at least a first property defines a display appearance or a display action.

9. The method of claim 7, wherein the first subset of the plurality of objects is selected based in part on a property of the objects.

10. The method of claim 7, further comprising executing a third software node to create a new object based on a property of an object in the first subset of objects.

11. The method of claim 7, further comprising executing a third software node to remove an object from the first subset of objects based on a property of the object.

12. A computing system for managing objects in an interactive video display system including an imaging device, the system comprising:

a non-transitory computer readable medium configured to store properties associated with respective objects of a plurality of objects, wherein at least some of the properties associated with the respective objects indicate appearance characteristics of the respective objects;

an interface configured to receive user interaction data from an imaging device, the user interaction data indicating user interactions with the a video display; and

a processor configured to;

associate a first subset of the plurality of objects with a first state based on the user interaction data, wherein the first subset comprises two or more objects;

associate a second subset of the plurality of objects with a second state based on the user interaction data, wherein the second subset comprises two or more objects, including at least one of the objects of the first subset;

execute a first software node on each of the plurality of objects associated with the first state, wherein execution of the first software node changes a first property of the objects in the first subset; and

execute a second software node on each of the plurality of objects associated with the second state, wherein execution of the second software node changes a second property of the objects in the second subset.

13. The system of claim 12, wherein the processor is further configured to:

associate the first software node with the first state, and

execute the first software node on objects associated with the first state.

14. The system of claim 12, wherein the objects in the first subset of the plurality of objects are associated with the first state based at least in part on the user interaction data.

15. The system of claim 13, wherein executing the first software node on the first subset of the plurality of objects modifies a property of the respective objects based at least in part on the user interaction data.

16. The system of claim 13, wherein executing the first software node performs one or more of assigning a property to the objects associated with the first state, applying a force to the objects associated with the first state, deleting the objects associated with the first state, changing the state associated with the objects associated with the first state, and enabling a feature of the objects associated with the first state.

17. The system of claim 12, wherein the processor is further configured to:

initiate display of at least one of the objects associated with the first state on the video display.

18. A non-transitory tangible computer readable storage medium having software instructions stored thereon, the software instructions readable by a computing system comprising one or more computing devices, wherein the software instructions are executable on the computing system in order to cause the computing system to perform operations comprising:

storing properties of respective objects of a plurality of objects in a storage device, wherein the properties indicate respective display appearances or display actions of the respective objects;

receiving input data from a camera indicative of a first user interaction and a second user interaction with a video display;

generating a first list of two or more objects in response to the first user interaction;

generating a second list of two or more objects in response to the second user interaction, wherein the first and second lists each include a first object;

executing a first software node on the objects indicated by the first list to modify a respective first property of the objects;

executing a second software node on the objects indicated by the second list to modify a respective second property of the objects; and

initiating display of the modified objects on the video display.

19. The non-transitory tangible computer readable storage medium of claim 18, wherein the first list corresponds to a display effect of the objects indicated by the first list.

20. The non-transitory tangible computer readable storage medium of claim 19, wherein the operations further comprise:

associating the first software node with the first list of objects; and

executing the first software node only on the objects indicated by the first list of objects.