SYSTEM AND METHOD FOR AUTHORING AND SHARING CONTENT IN AUGMENTED REALITY

• US 20180253900A1
• Filed: 03/02/2017
• Published: 09/06/2018
• Est. Priority Date: 03/02/2017
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: DAQRI, LLC
• Inventors: Finding, Samuel, Mullins, Brian, Gupta, Noopur, Reyes, Anthony L., Aalto, Neil
• Granted Patent: US 10,685,489 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A device has an optical sensor, an inertial sensor, and a hardware processor. The optical sensor generates image data. The inertial sensor generates inertia data. The hardware processor receives an augmented reality (AR) authoring template authored at a client device, generates media content using the image data, and receives a selection of spatial coordinates within a three-dimensional region using the inertia data and the image data. The three-dimensional region is identified in the AR authoring template. The hardware processor further identifies an entry in the AR authoring template corresponding to the media content, places the media content at the selected spatial coordinates in the entry in the AR authoring template, and forms AR content using the media content at the selected spatial coordinates placed in the entry in the AR authoring template.

Claims

1. A device comprising:

an optical sensor configured to generate image data;

an inertial sensor configured to generate inertia data;

one or more hardware processors; and

a memory storing instructions that, when executed by the one or more hardware processors, configure the wearable device to perform operations comprising;

receiving an augmented reality (AR) authoring template authored at a client device;

generating media content using the image data;

receiving a selection of spatial coordinates within a three-dimensional region using the inertia data and the image data, the three-dimensional region identified in the AR authoring template;

identifying an entry in the AR authoring template corresponding to the media content;

placing the media content at the selected spatial coordinates in the entry in the AR authoring template; and

forming AR content using the media content at the selected spatial coordinates placed in the entry in the AR authoring template.

2. The device of claim 1, wherein the AR authoring template includes a flow process of instructions for operating one or more physical objects within the three-dimensional region.

3. The device of claim 1, wherein the media content includes a video recording generated at a geographic location corresponding to the selected spatial coordinates within the three-dimensional region.

4. The device of claim 3, wherein the video recording includes a video of an operation of a physical object by a user of the device, the physical object being located at the geographic location.

5. The device of claim 1, wherein receiving the selection of spatial coordinates further comprises:

determining a geographic location of the device using a combination of the image data and the inertia data;

determining a spatial location identified by a user of the device via a user interface of the device;

defining the selected spatial coordinates to identify the spatial location; and

associating the AR content with the spatial location.

6. The device of claim 5, wherein the selected spatial coordinates are defined relative to a location of a physical object at the geographic location of the device.

7. The device of claim 5, wherein the selected spatial coordinates are defined relative to a location, position, and orientation of the device within the three-dimensional region.

8. The device of claim 1, wherein the operations further comprise:

publishing the AR content to a second device based on a parameter matching, the second device configured to trigger a display of the media content displayed at a spatial location in response to a determination of matching parameters.

9. The device of claim 8, wherein the second device is configured to adjust the spatial location associated with the media content.

10. The device of claim 1, wherein the operations further comprise:

determining that a first task at a first location within the three-dimensional region depends on the completion of a second task at a second location within the same three-dimensional region;

determining that the second task at the second location has not been completed;

generating an AR notification that the second task has not been completed;

displaying the AR notification in a transparent display of the device; and

displaying directions to the second location in the transparent display of the device.

11. A method comprising:

generating image data at a device using an optical sensor of the device;

generating inertial sensor data at the device using an inertial sensor of the device;

receiving an augmented reality (AR) authoring template authored at a client device;

generating media content using the image data;

receiving a selection of spatial coordinates within a three-dimensional region using the inertia data and the image data, the three-dimensional region identified in the AR authoring template;

identifying an entry in the AR authoring template corresponding to the media content;

placing the media content at the selected spatial coordinates in the entry in the AR authoring template; and

using a hardware processor to form AR content using the media content at the selected spatial coordinates placed in the entry in the AR authoring template.

12. The method of claim 11, wherein the AR authoring template includes a flow process of instructions for operating one or more physical objects within the three-dimensional region.

13. The method of claim 11, wherein the media content includes a video recording generated at a geographic location corresponding to the selected spatial coordinates within the three-dimensional region.

14. The method of claim 13, wherein the video recording includes a video of an operation of a physical object by a user of the device, the physical object being located at the geographic location.

15. The method of claim 11, wherein receiving the selection of spatial coordinates further comprises:

determining a geographic location of the device using a combination of the image data and the inertia data;

determining a spatial location identified by a user of the device via a user interface of the device;

defining the selected spatial coordinates to identify the spatial location; and

associating the AR content with the spatial location.

16. The method of claim 15, wherein the selected spatial coordinates are defined relative to a location of a physical object at the geographic location of the device.

17. The method of claim 15, wherein the selected spatial coordinates are defined relative to a location, position, and orientation of the device within the three-dimensional region.

18. The method of claim 11, wherein the operations further comprise:

publishing the AR content to a second device located within the three-dimensional region, the second device configured to trigger a display of the media content displayed at a spatial location corresponding to a physical object within the three-dimensional region, the spatial location identified by a user of the device via a user interface of the device.

19. The method of claim 11, further comprising:

determining that a first task at a first location within the three-dimensional region depends on the completion of a second task at a second location within the same three-dimensional region;

determining that the second task at the second location has not been completed;

generating a AR notification that the second task has not been completed;

displaying the AR notification in a transparent display of the device; and

displaying directions to the second location in the transparent display of the device.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

generating image data at a device using an optical sensor of the device;

generating inertial sensor data at the device using an inertial sensor of the device;

receiving an augmented reality (AR) authoring template authored at a client device;

generating media content using the image data;

receiving a selection of spatial coordinates within a three-dimensional region using the inertia data and the image data, the three-dimensional region identified in the AR authoring template;

identifying an entry in the AR authoring template corresponding to the media content;

placing the media content at the selected spatial coordinates in the entry in the AR authoring template; and

forming AR content using the media content at the selected spatial coordinates placed in the entry in the AR authoring template.