METHOD AND APPARATUS FOR CONTENT ASSOCIATION AND HISTORY TRACKING IN VIRTUAL AND AUGMENTED REALITY

• US 20170132844A1
• Filed: 01/19/2017
• Published: 05/11/2017
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Fateh, Sina, Butterworth, Ron, Chehade, Mohamed Nabil Hajj, Yang, Allen Yang, Itani, Sleiman
• Granted Patent: US 10,019,845 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/24569 Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A machine-implemented method includes establishing a virtual or augmented reality entity, and establishing a state for the entity having a state time and state properties including a state spatial arrangement. The data entity and state are stored, and are subsequently received and outputted at a time other than the state time so as to exhibit a “virtual time machine” functionality. An apparatus includes a processor, a data store, and an output. A data entity establisher, a state establisher, a storer, a data entity receiver, a state receiver, and an outputter are instantiated on the processor.

Claims

1. A method comprising:

instantiating a data entity establisher comprising executable instructions on a processor, said data entity establisher being adapted to establish data entities;

instantiating a state establisher comprising executable instructions on said processor, said state establisher being adapted to establish states for said data entities;

instantiating a storer comprising executable instructions on said processor, said storer being adapted to store said data entities and said states in a data store in communication with said processor;

instantiating an outputter comprising executable instructions on said processor, said outputter being adapted to output said data entities and said states to an output in communication with said processor;

establishing a first data entity via said data entity establisher;

establishing a first state for said first data entity via said state establisher;

establishing a second state for said first first data entity via said state establisher, said second state being distinct from said first state;

establishing a second data entity via said data entity establisher, said second data entity being distinct from said first data entity;

establishing a third state for said second data entity via said state establisher, said third state being distinct from said first and second states;

wherein each of said states comprises a state time and a plurality of state properties at least substantially corresponding to a respective of said data entities substantially at said state time and comprising a state spatial arrangement thereof;

wherein a first state time of said first state is substantially different from a second state time of said second state;

wherein a third state time of said third state is substantially different from said first state time of said first state;

storing said first and second data entities and said first, second, and third states in said data store via said storer;

retrieving from said data store and outputting via said outputter to at least one of an augmented reality environment and a virtual reality environment at a same output time at least one of;

a first iteration of said first data entity exhibiting at least a portion of said first state and a second iteration of said first data entity exhibiting at least a portion of said second state, said output time being substantially different from said first and second state times; and

said first iteration of said first data entity exhibiting said at least said portion of said first state and a first iteration of said second data entity exhibiting at least a portion of said third state, said output time being substantially different from said first and third state times.

2. A method comprising:

establishing a first data entity via a data entity establisher instantiated on a processor;

establishing a first state for said first data entity via a state establisher instantiated on said processor;

establishing a second state for said first first data entity via said state establisher, said second state being distinct from said first state;

establishing a second data entity via said data entity establisher, said second data entity being distinct from said first data entity;

establishing a third state for said second data entity via said state establisher, said third state being distinct from said first and second states;

wherein each of said states comprises a state time and a plurality of state properties at least substantially corresponding to a respective of said data entities substantially at said state time and comprising a state spatial arrangement thereof;

wherein a first state time of said first state is substantially different from a second state time of said second state;

wherein a third state time of said third state is substantially different from said first state time of said first state;

storing said first and second data entities and said first, second, and third states in a data store in communication with said processor via a storer instantiated on said processor;

retrieving from said data store and outputting to an output in communication with said processor via an outputter instantiated on said processor at a same output time at least one of;

a first iteration of said first data entity exhibiting at least a portion of said first state and a second iteration of said first data entity exhibiting at least a portion of said second state, said output time being substantially different from said first and second state times; and

said first iteration of said first data entity exhibiting said at least said portion of said first state and a first iteration of said second data entity exhibiting at least a portion of said third state, said output time being substantially different from said first and third state times.

3. The method of claim 2, wherein:

establishing said first and second data entities comprises selecting said first and second data entities, said first and second data entities being in existence prior to a selection thereof.

4. The method of claim 2, wherein:

establishing said first and second data entities comprises creating said first and second data entities.

5. The method of claim 2, wherein:

establishing said first data entity comprises generating said first data entity from a first parent entity; and

establishing said second data entity comprises generating said second data entity from a second parent entity.

6. The method of claim 5, wherein:

said first parent entity comprises at least one of a group consisting of a physical object, a physical background, a physical environment, a physical creature, and a physical phenomenon; and

said second parent entity comprises at least one of said group consisting of said physical object, said physical background, said physical environment, said physical creature, and said physical phenomenon.

7. The method of claim 6, wherein:

generating said first data entity from said first parent entity comprises sensing said first parent entity with a sensor in communication with said processor; and

generating said second data entity from said second parent entity comprises sensing said second parent entity with said sensor.

8. The method of claim 5, wherein:

said first parent entity comprises at least one of a group consisting of an augmented reality object and a virtual reality object; and

said second parent entity comprises at least one of said group consisting of said augmented reality object and said virtual reality object.

9. The method of claim 2, wherein:

at least a portion of each said state is incorporated within a respective of said data entities.

10. The method of claim 2, wherein:

at least a portion of each said state is distinct from said data entities.

11. The method of claim 2, wherein:

at least one of said state properties comprises an identity of a respective of said data entities.

12. The method of claim 2, wherein:

said state spatial arrangement comprises at least one of a group consisting of an absolute position of a respective of said data entities and an absolute orientation of said respective data entity.

13. The method of claim 2, wherein:

said state spatial arrangement comprises at least one of a group consisting of a relative position of a respective of said data entity and a relative orientation of said respective data entity.

14. The method of claim 2, wherein:

said plurality of state properties comprises at least one of a group consisting of a still image, a video, audio, olfactory data, a 2D model, a 3D model, text, numerical data, an environmental condition, animation, resolution, frame rate, bit depth, sampling rate, color, color distribution, spectral signature, brightness, brightness distribution, reflectivity, transmissivity, absorptivity, surface texture, geometry, mobility, motion, speed, direction, acceleration, temperature, temperature distribution, composition, chemical concentration, electrical potential, electrical current, mass, mass distribution, density, density distribution, price, quantity, nutritional information, user review, presence, visibility, RFID data, barcode data, a file, executable instructions, a hyperlink, a data connection, a communication link, contents, association, creator, and system ID.

15. The method of claim 2, wherein:

establishing said first and second data entities comprises distinguishing said first and data entities from surroundings thereof.

16. The method of claim 2, wherein:

said first data entity comprises a graphical entity;

said second data entity comprises a second graphical entity; and

said output is a graphical display.

17. The method of claim 2, wherein:

said processor, said data store, and said output are disposed on a head mounted display.

18. The method of claim 7, wherein:

said processor, said data store, said output, and said sensor are disposed on a head mounted display.

19. A method comprising:

establishing a first data entity via a data entity establisher instantiated on a processor;

establishing a first state for said first data entity via a state establisher instantiated on said processor;

establishing a second state for said first first data entity via said state establisher, said second state being distinct from said first state;

wherein each of said states comprises a state time and a plurality of state properties at least substantially corresponding to a respective of said data entities substantially at said state time and comprising a state spatial arrangement thereof;

wherein a first state time of said first state is substantially different from a second state time of said second state;

storing said first data entity and said first and second states in a data store in communication with said processor via a storer instantiated on said processor;

retrieving from said data store and outputting to an output in communication with said processor via an outputter instantiated on said processor at a same output time a first iteration of said first data entity exhibiting at least a portion of said first state and a second iteration of said first data entity exhibiting at least a portion of said second state, said output time being substantially different from said first and second state times.

20. A method comprising:

establishing a first data entity via a data entity establisher instantiated on a processor;

establishing a first state for said first data entity via a state establisher instantiated on said processor;

establishing a second data entity via said data entity establisher, said second data entity being distinct from said first data entity;

establishing a second state for said second data entity via said state establisher, said second state being distinct from said first state;

wherein each of said states comprises a state time and a plurality of state properties at least substantially corresponding to a respective of said data entities substantially at said state time and comprising a state spatial arrangement thereof;

wherein a first state time of said first state is substantially different from a second state time of said second state;

storing said first and second data entities and said first and second states in a data store in communication with said processor via a storer instantiated on said processor;

retrieving from said data store and outputting to an output in communication with said processor via an outputter instantiated on said processor an iteration of said first data entity exhibiting said at least said portion of said first state and an iteration of said second data entity exhibiting at least a portion of said second state, said output time being substantially different from said first and second state times.

21. An apparatus comprising:

a processor;

a data store in communication with said processor;

an output in communication with said processor;

a data entity establisher disposed on said processor and adapted to establish a first data entity and a second data entity;

a state establisher disposed on said processor and adapted to establish a first state for said first data entity, a second state for said first data entity, and a third state for said second data entity;

wherein each of said states comprises a state time and a plurality of state properties at least substantially corresponding to a respective of said data entities substantially at said state time and comprising a state spatial arrangement thereof;

wherein a first state time of said first state is substantially different from a second state time of said second state;

wherein a third state time of said third state is substantially different from said first state time of said first state;

a storer disposed on said processor and adapted to store said first and second data entities and said first, second, and third states in said data store;

an outputter disposed on said processor and adapted to output to an output at a same output time at least one of;

a first iteration of said first data entity exhibiting at least a portion of said first state and a second iteration of said first data entity exhibiting at least a portion of said second state, said output time being substantially different from said first and second state times; and

said first iteration of said first data entity exhibiting said at least said portion of said first state and a first iteration of said second data entity exhibiting at least a portion of said third state, said output time being substantially different from said first and third state times.