Spatial data visualization

• US 9,799,143 B2
• Filed: 08/15/2014
• Issued: 10/24/2017
• Est. Priority Date: 08/15/2014
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: DAQRI, LLC
• Inventors: Mullins, Brian, Kammerait, Matthew, Irving, Jr., Frank Chester
• Primary Examiner(s): He, Yingchun
• Application Number:
  US14/461,277
• Publication Number:
  US 20160049007A1
• Time in Patent Office: 1,166 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system and method for spatial data visualization are described. An analytics computation of users'"'"' interactions with an augmented reality content is performed based on a physical object captured by a viewing device. The analytics computation comprises a computation of geometric paths of the users'"'"' interactions with the augmented reality content. A display of a visualization of the analytics computation is displayed based on the computation of the geometric paths of the users'"'"' interactions with the augmented reality content.

Claims

1. A server comprising:

a hardware processor comprising an augmented reality application, the augmented reality application configured to perform operations comprising;

performing an analytics computation of users'"'"' interactions with an augmented reality content based on a physical object captured by a viewing device, the analytics computation comprising a computation of geometric paths of the users'"'"' interactions with the augmented reality content;

causing a display of a visualization of the analytics computation from the users'"'"' interactions with the augmented reality content;

accessing and processing path bundle data packages received from the viewing device, the path bundle data packages describing a user'"'"'s interaction of the viewing device with the augmented reality content based on a physical object captured by the viewing device, the path bundle data packages generated based on sensor data using a data model comprising a data header and a data payload, the data header comprising a contextual header having data identifying the viewing device and a user of the viewing device, a path header having data identifying a path of the user interaction with the augmented reality content, and a sensor header having data identifying a plurality of sensors, the data payload comprising dynamically sized sampling data from the sensor data;

normalizing the path bundle data packages;

aggregating the normalized path bundle data packages; and

performing the analytics computation on the normalized and aggregated path bundle data packages.

2. The server of claim 1, wherein the display includes a heatmap identifying areas with relatively higher number of users'"'"' interactions with the augmented reality content.

3. The server of claim 1, wherein the display includes trajectory paths of the users'"'"' interactions with the augmented reality content, the trajectory paths identifying rotations and movements of the augmented reality content caused by the users'"'"' interactions.

4. The server of claim 3, wherein the display includes an identification of most common trajectory paths of the users'"'"' interactions with the augmented reality content.

5. The server of claim 1, wherein the geometric paths of the users'"'"' interactions comprise a geometric trajectory based on a dwelling time exceeding a threshold for a view of the augmented reality content.

6. The server of claim 1, wherein the operations further comprise generating an extension of the data model to represent the normalized and aggregated path bundle data packages, accessing trigger conditions, to perform performing the analytics computation on the extension of the data model based on the trigger conditions, and identifying a spatial pattern based on the trigger conditions.

7. The server of claim 6, wherein the operations further comprise normalizing and aggregating spatial and sensor data from multiple sources across multiple dimensions from the path to filter singular and aggregate data.

8. The server of claim 1, wherein the operations further comprise:

identifying spatial patterns for accidents and event predictions;

recognizing similar patterns and warn operators of the viewing device;

assigning weighting schemes for switching amongst sensors of the viewing device;

identifying user focus points and user loss of focus and redirect the user;

ranking task complexity and calibrate user skill levels based on the normalized and aggregated path bundle data packages; and

identifying spatial movement-based gestures and interfaces based on the normalized and aggregated path bundle data packages.

9. The server of claim 1, wherein the sensor data is related to a position and an orientation of the viewing device over time, the plurality of sensors comprising optical sensors and inertia sensors,

wherein the contextual header comprises a viewing device identifier, a session identifier, a physical location, and a viewing device type,

wherein the path header comprises a path start time, a path end time, a type of sensor identifier for each corresponding sensor, a total duration of the path, and a total distance of the path,

wherein the sensor header comprises, for each sensor, a sensor identifier, a sensor position identifier, three-dimensional position data, orientation data, and a number of data samples,

wherein the dynamically sized sampling data comprises a sensor identifier, a data sample number, a relative time, a duration percentage through the path, and a time percentage through the path.

10. A method comprising:

performing, using a hardware processor of a machine, an analytics computation of users'"'"' interactions with an augmented reality content based on a physical object captured by a viewing device, the analytics computation comprising a computation of geometric paths of the users'"'"' interactions with the augmented reality content;

causing a display of a visualization of the analytics computation from the users'"'"' interactions with the augmented reality content;

accessing and processing path bundle data packages received from the viewing device, the path bundle data packages describing a user interaction of the viewing device with the augmented reality content based on a physical object captured by the viewing device, the path bundle data packages generated based on sensor data using a data model comprising a data header and a data payload, the data header comprising a contextual header having data identifying the viewing device and a user of the viewing device, a path header having data identifying a path of the user interaction with the augmented reality content, and a sensor header having data identifying a plurality of sensors, the data payload comprising dynamically sized sampling data from the sensor data, the path bundle processing module comprising a normalization module;

normalizing the path bundle data packages;

aggregating the normalized path bundle data packages; and

performing the analytics computation on the normalized and aggregated path bundle data packages.

11. The method of claim 10, wherein the display includes a heatmap identifying areas with relatively higher number of users'"'"' interactions with the augmented reality content.

12. The method of claim 10, wherein the display includes trajectory paths of the users'"'"' interactions with the augmented reality content, the trajectory paths identifying rotations and movements of the augmented reality content caused by the users'"'"' interactions.

13. The method of claim 12, wherein the display includes an identification of most common trajectory paths of the users'"'"' interactions with the augmented reality content.

14. The method of claim 10, wherein the geometric paths of the users'"'"' interactions comprise a geometric trajectory based on a dwelling time exceeding a threshold for a view of the augmented reality content.

15. The method of claim 10, further comprising:

generating an extension of the data model to represent the normalized and aggregated path bundle data packages, to access trigger conditions, to perform the analytics computation on the extension of the data model based on the trigger conditions, and to identify a spatial pattern based on the trigger conditions; and

normalizing and aggregating spatial and sensor data from multiple sources across multiple dimensions from the path to filter singular and aggregate data.

16. The method of claim 10, further comprising:

identifying spatial patterns for accidents and event predictions;

recognizing similar patterns and warning operators of the viewing device;

assigning weighting schemes for switching amongst sensors of the viewing device;

identifying user focus points and user loss of focus and redirecting the user;

ranking task complexity and calibrating user skill levels based on the normalized and aggregated path bundle data packages; and

identifying spatial movement-based gestures and interfaces based on the normalized and aggregated path bundle data packages.

17. The method of claim 10, wherein the sensor data is related to a position and an orientation of the viewing device over time, the plurality of sensors comprising optical sensors and inertia sensors, wherein the contextual header comprises a viewing device identifier, a session identifier, a physical location, and a viewing device type, wherein the path header comprises a path start time, a path end time, a type of sensor identifier for each corresponding sensor, a total duration of the path, and a total distance of the path, wherein the sensor header comprises, for each sensor, a sensor identifier, a sensor position identifier, three-dimensional position data, orientation data, and a number of data samples, wherein the dynamically sized sampling data comprises a sensor identifier, a data sample number, a relative time, a duration percentage through the path, and a time percentage through the path.

18. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

performing an analytics computation of users'"'"' interactions with an augmented reality content based on a physical object captured by a viewing device, the analytics computation comprising a computation of geometric paths of the users'"'"' interactions with the augmented reality content;

causing a display of a visualization of the analytics computation from the users'"'"' interactions with the augmented reality content;

accessing and processing path bundle data packages received from the viewing device, the path bundle data packages describing a user interaction of the viewing device with the augmented reality content based on a physical object captured by the viewing device, the path bundle data packages generated based on sensor data using a data model comprising a data header and a data payload, the data header comprising a contextual header having data identifying the viewing device and a user of the viewing device, a path header having data identifying a path of the user interaction with the augmented reality content, and a sensor header having data identifying a plurality of sensors, the data payload comprising dynamically sized sampling data from the sensor data, the path bundle processing module comprising a normalization module;

normalizing the path bundle data packages;

aggregating the normalized path bundle data packages; and

performing the analytics computation on the normalized and aggregated path bundle data packages.