Method and apparatus for content association and history tracking in virtual and augmented reality

• US 9,589,000 B2
• Filed: 08/29/2013
• Issued: 03/07/2017
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Fateh, Sina, Butterworth, Ron, Chehade, Mohamed Nabil Hajj, Yang, Allen Yang, Itani, Sleiman
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): BEARD, CHARLES LLOYD
• Application Number:
  US14/014,296
• Publication Number:
  US 20140067768A1
• Time in Patent Office: 1,286 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/24569 Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A machine-implemented method includes establishing a virtual or augmented reality entity, and establishing a state for the entity having a state time and state properties including a state spatial arrangement. The data entity and state are stored, and are subsequently received and outputted at a time other than the state time so as to exhibit a “virtual history machine” functionality. An apparatus includes a processor, a data store, and an output. A data entity establisher, a state establisher, a storer, a data entity receiver, a state receiver, and an outputter are instantiated on the processor.

Claims

1. An apparatus, comprising:

a processor adapted to execute executable instructions;

a data entity establisher instantiated on said processor, said data entity establisher comprising executable instructions adapted for establishing a plurality of distinct data entities, each said distinct data entity comprising at least one of a group consisting of an augmented reality entity and a virtual reality entity;

a state establisher instantiated on said processor, said state establisher comprising executable instructions adapted for establishing a plurality of distinct states for each said data entity, each said state comprising;

a state time;

a plurality of state properties at least substantially corresponding to properties of one said data entity substantially at said state time, said plurality of state properties comprising a state spatial arrangement of said data entity;

a data store in communication with said processor;

a storer instantiated on said processor, said storer comprising executable instructions adapted for storing in said data store said plurality of distinct data entities and said distinct states therefor, so as to enable output of said distinct data entities to at least one of an augmented reality environment and a virtual reality environment with each said data entity exhibiting at least a portion of one of said distinct states therefor;

a user output in communication with said processor;

wherein;

said data entity establisher and said state establisher are adapted to cooperate to establish;

a first data entity with a first state and a second state, a first state time of said first state being substantially different from a second state time of said second state; and

said first data entity with said first state, and a second data entity with a third state, said first state time of said first state being substantially different from a third state time of said third state;

said storer is adapted to store for substantially concurrent output at a same output time;

a first iteration of said first data entity exhibiting said first state and a second iteration of said first data entity exhibiting said second state, said output time being substantially different from said first and second state times; and

said first iteration of said first data entity exhibiting said first state and a first iteration of said second data entity exhibiting said third state, said output time being substantially different from said first and third state times;

said user output is adapted to output at said same output time so as to be sensed by a user;

said first iteration of said first data entity exhibiting said first state and said second iteration of said first data entity exhibiting said second state; and

said first iteration of said first data entity exhibiting said first state and said first iteration of said second data entity exhibiting said third state.

2. The apparatus of claim 1, comprising:

a chronometer adapted to determine said output time.

3. The apparatus of claim 1, comprising:

at least one sensor in communication with said processor.

4. The apparatus of claim 3, wherein:

said at least one sensor is adapted to determine at least one of a group consisting of a spatial arrangement of said apparatus at said state time, a spatial arrangement of said apparatus at said output time, and said state spatial arrangement of said data entity.

5. The apparatus of claim 3, wherein:

said at least one sensor comprises at least one of a group consisting of an accelerometer, a gyroscope, an imager, a stereo pair of imagers, a GPS sensor, a magnetometer, a structured light sensor, a time-of-flight sensor, an ultrasonic sensor, and a wireless signal triangulation sensor.

6. The apparatus of claim 3, wherein:

said at least one sensor comprises at least one of a group consisting of a bar code reader, a chemical sensor, an electrical sensor, an electrical field sensor, a gas detector, a humidity sensor, an imager, a stereo pair of imagers, a light sensor, a magnetic field sensor, a microphone, a motion sensor, a pressure sensor, a radar sensor, a radiation sensor, an RFID sensor, a smoke sensor, a spectrometer, a thermal sensor, an ultrasonic sensor, and a vibration sensor.

7. The apparatus of claim 1, comprising:

a data entity distinguisher instantiated on said processor, said data entity distinguisher comprising executable instructions adapted for distinguishing said data entities from a surrounding thereof.

8. The apparatus of claim 1, comprising:

a data entity identifier instantiated on said processor, said data entity identifier comprising executable instructions adapted for identifying at least one of a group consisting of said data entities, parent entities for said data entities, and at least one of said state properties.

9. The apparatus of claim 1, wherein:

at least a portion of said apparatus is disposed on a head mounted display.

10. The apparatus of claim 1, comprising:

a data entity receiver instantiated on said processor, said data entity receiver comprising executable instructions adapted for receiving at least one of said data entities from said data store;

a state receiver instantiated on said processor, said state receiver comprising executable instructions adapted for receiving from said data store at least one said state for said at least one data entity;

an output in communication with said processor, said output being adapted to output said at least one data entity;

an outputter instantiated on said processor, said outputter comprising executable instructions adapted for outputting said plurality of data entities via said user output.

11. The apparatus of claim 1, wherein:

said states for said data entities are incorporated into said data entities.

12. An apparatus, comprising:

a processor adapted to execute executable instructions;

a data store in communication with said processor;

a data entity receiver instantiated on said processor, said data entity receiver comprising executable instructions adapted for receiving from said data store a plurality of distinct data entities, each said distinct data entity comprising at least one of a group consisting of an augmented reality entity and a virtual reality entity;

a state receiver instantiated on said processor, said state receiver comprising executable instructions adapted for receiving from said data store a plurality of distinct states for each said data entity, each said state comprising;

a state time;

a plurality of state properties at least substantially corresponding to properties of one said data entity substantially at said state time, said plurality of state properties comprising a state spatial arrangement of said data entity;

as a user output in communication with said processor;

an outputter instantiated on said processor, said outputter comprising executable instructions adapted for outputting said plurality of data entities via said user output to said at least one of said augmented reality environment and said virtual reality environment with each said data entity exhibiting at least a portion of one of said states therefor so as to be sensed by a user;

wherein;

said data receiver and said state receiver are adapted to cooperate to establish;

a first data entity with a first state and a second state, a first state time of said first state being substantially different from a second state time of said second state; and

said first data entity with said first state, and a second data entity with a third state, said first state time of said first state being substantially different from a third state time of said third state;

said outputter is adapted to output via said user output at as a same output time;

a first iteration of said first data entity exhibiting at least said portion of said first stat, and a second iteration of said first data entity exhibiting at least said portion of said second state, said output time being substantially different from said first and second state times; and

said first iteration of said first data entity exhibiting at least said portion of said first state and a first iteration of said second data entity exhibiting at least said portion of said third stat, said output time being substantially different from said first and third state times.

13. The apparatus of claim 12, comprising:

a chronometer adapted to determine said output time.

14. The apparatus of claim 12, comprising:

at least one sensor in communication with said processor.

15. The apparatus of claim 14, wherein:

said at least one sensor is adapted to determine at least one of a group consisting of a spatial arrangement of said apparatus at at least one of said state times, a spatial arrangement of said apparatus at said at least one of said output times, and at least one of said state spatial arrangements of said data entity substantially corresponding with said at least one of said state times.

16. The apparatus of claim 14, wherein:

said at least one sensor comprises at least one of a group consisting of an accelerometer, a gyroscope, an imager, a stereo pair of imagers, a GPS sensor, a magnetometer, a structured light sensor, a time-of-flight sensor, an ultrasonic sensor, and a wireless signal triangulation sensor.

17. The apparatus of claim 12, wherein:

said user output comprises a visual display.

18. The apparatus of claim 12, wherein:

said user output comprises a stereo pair of visual displays.

19. The apparatus of claim 12, wherein:

at least a portion of said apparatus is disposed on a head mounted display.

20. The method of claim 12, wherein:

said states for said data entities are incorporated into said data entities.

21. A method comprising:

instantiating on a processor a data entity establisher, said data entity establisher comprising executable instructions adapted for establishing a plurality of distinct data entities, each said distinct data entity comprising at least one of a group consisting of an augmented reality entity and a virtual reality entity;

instantiating on said processor a state establisher, said state establisher comprising executable instructions adapted for establishing a plurality of distinct states for each said data entity, each said state comprising;

a state time;

a plurality of state properties at least substantially corresponding to properties of one said data entity substantially at said state time, said plurality of state properties comprising a state spatial arrangement of said data entity;

instantiating on said processor a storer, said storer comprising executable instructions adapted for storing said plurality of distinct data entities and said distinct states therefor, so as to enable output of said data distinct entities to at least one of an augmented reality environment and a virtual reality environment with each said data entity exhibiting at least a portion of one of said distinct states therefor;

disposing a user output in communication with said processor;

wherein;

said data entity establisher and said state establisher are adapted to cooperate to establish;

a first data entity with a first state and a second state, a first state time of said first state being substantially different from a second state time of said second state; and

said first data entity with said first state, and a second data entity with a third state, said first state time of said first state being substantially different from a third state time of said third state;

said storer is adapted to store for output at a same output time;

a first iteration of said first data entity exhibiting said first state and a second iteration of said first data entity exhibiting said second state, said output time being substantially different from said first and second state times; and

said first iteration of said first data entity exhibiting said first state and a first iteration of said second data entity exhibiting said third state, said output time being substantially different from said first and third state times;

said user output is adapted to output at said same output time so as to be sensed by a user;

said first iteration of said first data entity exhibiting said first state and said second iteration of said first data entity exhibiting said second state; and

said first iteration of said first data entity exhibiting said first state and said first iteration of said second data entity exhibiting said third state.

22. The method of claim 21, wherein:

said data entity establisher is adapted to select at least one of said plurality of data entities, said at least one data entity being in existence prior to a selection thereof.

23. The method of claim 21, wherein:

said data entity establisher is adapted to create at least one of said plurality of data entities.

24. The method of claim 21, wherein:

said data entity establisher is adapted to generate at least one of said plurality of data entities from a parent entity.

25. The method of claim 24, wherein:

said parent entity comprises at least one of a group consisting of a physical object, a physical background, a physical environment, a physical creature, and a physical phenomenon.

26. The method of claim 24, wherein:

said parent entity comprises at least one of a group consisting of an augmented reality object and a virtual reality object.

27. The method of claim 21, comprising:

instantiating on said processor a data entity distinguisher, said data entity distinguisher comprising executable instructions adapted for distinguishing at least one of said plurality of data entities from a surrounding thereof.

28. The method of claim 21, comprising:

instantiating on said processor an identifier, said identifier comprising executable instructions adapted for identifying at least one of a group consisting of at least one of said plurality of data entities, a parent entity for said at least one of said plurality of data entities, and at least one of said state properties therefor.

29. The method of claim 21, comprising:

instantiating on said processor a data entity receiver, said data entity receiver comprising executable instructions adapted for receiving from said data store at least one of said plurality of data entities;

instantiating on said processor a state receiver, said state receiver comprising executable instructions adapted for receiving from said data store at least one said state for said at least one data entity;

instantiating on said processor an outputter, said outputter comprising executable instructions adapted for outputting said plurality of data entities via said user output.

30. The apparatus of claim 21, wherein:

said states for said data entities are incorporated into said data entities.

31. A method comprising:

instantiating on said processor a data entity receiver, said data entity receiver comprising executable instructions adapted for receiving from a data store a plurality of distinct data entities, each said distinct data entity comprising at least one of a group consisting of an augmented reality entity and a virtual reality entity;

instantiating on said processor a state receiver, said state receiver comprising executable instructions adapted for receiving from said data store a plurality of distinct states for each said data entity, each said state comprising;

a state time;

a plurality of state properties at least substantially corresponding to properties of said data entity substantially at said state time, said plurality of state properties comprising a state spatial arrangement of said data entity;

disposing a user output in communication with said processor;

instantiating on said processor an outputter, said outputter comprising executable instructions adapted for outputting said plurality of data entities via said user output to said at least one of said augmented reality environment and said virtual reality environment, with each said data entity exhibiting at least a portion of one of said states therefor so as to be sensed by a user;

wherein;

said data entity receiver and said state receiver are adapted to cooperate to establish;

a first data entity with a first state and a second state, a first state time of said first state being substantially different from a second state time of said second state; and

said first data entity with said first state, and a second data entity with a third state, said first state time of said first state being substantially different from a third state time of said third state;

said outputter is adapted to output via said user output at at a same output time;

a first iteration of said first data entity exhibiting at least said portion of said first state, and a second iteration of said first data entity exhibiting at least said portion of said second state, said output time being substantially different from said first and second state times; and

said first iteration of said data entity exhibiting at least said portion of said first state and a first iteration of said second data entity exhibiting at least said portion of said third state, said output time being substantially different from said first and third state times.

32. The method of claim 31, comprising:

instantiating on said processor an identifier, said identifier comprising executable instructions adapted for identifying at least one of a group consisting of at least one of said plurality of data entities and at least one of said state properties therefor.

33. The method of claim 31, wherein:

said states for said data entities are incorporated into said data entities.