System and method for authoring and sharing content in augmented reality

• US 10,685,489 B2
• Filed: 03/02/2017
• Issued: 06/16/2020
• Est. Priority Date: 03/02/2017
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: DAQRI, LLC
• Inventors: Finding, Samuel, Mullins, Brian, Gupta, Noopur, Reyes, Anthony L., Aalto, Neil
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US15/448,391
• Publication Number:
  US 20180253900A1
• Time in Patent Office: 1,202 Days
• Field of Search: 345419
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A device has an optical sensor, an inertial sensor, and a hardware processor. The optical sensor generates image data. The inertial sensor generates inertia data. The hardware processor receives an augmented reality (AR) authoring template authored at a client device, generates media content using the image data, and receives a selection of spatial coordinates within a three-dimensional region using the inertia data and the image data. The three-dimensional region is identified in the AR authoring template. The hardware processor further identifies an entry in the AR authoring template corresponding to the media content, places the media content at the selected spatial coordinates in the entry in the AR authoring template, and forms AR content using the media content at the selected spatial coordinates placed in the entry in the AR authoring template.

Claims

1. An augmented reality (AR) device comprising:

one or more hardware processors; and

a memory storing instructions that, when executed by the one or more hardware processors, configure the AR device to perform operations comprising;

receiving an AR authoring template from a remote server, the AR authoring template being associated with a first defined region and comprising a first entry corresponding to a first physical object located in the first defined region;

recording a video of a user physically manipulating the first physical object and demonstrating instructions for operating the first physical object, yielding a first media content item;

capturing a first set of spatial coordinates defining a first geographic location, the first geographic location indicating a location of the AR device while the AR device is recording the video of the user physically manipulating the first physical object and demonstrating instructions for operating the first physical object;

placing the first media content item and the first set of spatial coordinates in the first entry of the AR authoring template; and

generating AR content for the first defined region based on the AR authoring template, the AR content including a first trigger condition based on the first entry of the AR authoring template, the first trigger condition causing a second AR device executing the AR content to cause playback of the first media content item on a display of the second AR device in response to the second AR device determining that the second AR device is within a predetermined geographic distance of the first geographic location defined by the first set of special coordinates.

2. The AR device of claim 1, wherein the AR authoring template includes a second entry corresponding to a second physical object located in the first defined region.

3. The AR device of claim 2, the operations further comprising:

recording a video of the user physically manipulating the second physical object and demonstrating instructions for operating the second physical object, yielding a second media content item;

capturing a second set of spatial coordinates defining a second geographic location, the second geographic location indicating a location of the AR device while the AR device is recording the video of the user physically manipulating the second physical object and demonstrating instructions for operating the second physical object;

placing the second media content item and the second set of spatial coordinates in the second entry of the AR authoring template.

4. The AR device of claim 3, wherein generating the AR content for the first defined region is further based on second entry of AR authoring template.

5. The AR device of claim 4, wherein the AR content includes a second trigger condition based on the second entry of the AR authoring template, the second trigger condition causing the second AR device executing the AR content to cause playback of the second media content item on the display of the second AR device in response to the second AR device determining that the second AR device is within a predetermined geographic distance of the second geographic location defined by the second set of special coordinates.

6. The AR device of claim 1, the operations further comprising:

transmitting the AR content to the remote server, the remote server providing the AR content to at least the second AR device.

7. The AR device of claim 1, wherein the first set of spatial coordinates are defined relative to a position and orientation of the AR device within the first defined region.

8. A method comprising:

receiving, by an augmented reality (AR) device, an AR authoring template from a remote server, the AR authoring template being associated with a first defined region and comprising a first entry corresponding to a first physical object located in the first defined region;

recording a video of a user physically manipulating the first physical object and demonstrating instructions for operating the first physical object, yielding a first media content item;

capturing a first set of spatial coordinates defining a first geographic location, the first geographic location indicating a location of the AR device while the AR device is recording the video of the user physically manipulating the first physical object and demonstrating instructions for operating the first physical object;

placing the first media content item and the first set of spatial coordinates in the first entry of the AR authoring template; and

generating AR content for the first defined region based on the AR authoring template, the AR content including a first trigger condition based on the first entry of the AR authoring template, the first trigger condition causing a second AR device executing the AR content to cause playback of the first media content item on a display of the second AR device in response to the second AR device determining that the second AR device is within a predetermined geographic distance of the first geographic location defined by the first set of special coordinates.

9. The method of claim 8, wherein the AR authoring template includes a second entry corresponding to a second physical object located in the first defined region.

10. The method of claim 9, further comprising:

recording a video of the user physically manipulating the second physical object and demonstrating instructions for operating the second physical object, yielding a second media content item;

capturing a second set of spatial coordinates defining a second geographic location, the second geographic location indicating a location of the AR device while the AR device is recording the video of the user physically manipulating the second physical object and demonstrating instructions for operating the second physical object;

placing the second media content item and the second set of spatial coordinates in the second entry of the AR authoring template.

11. The method of claim 10, wherein generating the AR content for the first defined region is further based on second entry of AR authoring template.

12. The method of claim 11, wherein the AR content includes a second trigger condition based on the second entry of the AR authoring template, the second trigger condition causing the second AR device executing the AR content to cause playback of the second media content item on the display of the second AR device in response to the second AR device determining that the second AR device is within a predetermined geographic distance of the second geographic location defined by the second set of special coordinates.

13. The method of claim 8, further comprising:

transmitting the AR content to the remote server, the remote server providing the AR content to at least the second AR device.

14. The method of claim 8, wherein the first set of spatial coordinates are defined relative to a position and orientation of the AR device within the first defined region.

15. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of an augmented reality (AR) device, cause the AR device to perform operations comprising:

receiving an AR authoring template from a remote server, the AR authoring template being associated with a first defined region and comprising a first entry corresponding to a first physical object located in the first defined region;

recording a video of a user physically manipulating the first physical object and demonstrating instructions for operating the first physical object, yielding a first media content item;

capturing a first set of spatial coordinates defining a first geographic location, the first geographic location indicating a location of the AR device while the AR device is recording the video of the user physically manipulating the first physical object and demonstrating instructions for operating the first physical object;

placing the first media content item and the first set of spatial coordinates in the first entry of the AR authoring template; and

generating AR content for the first defined region based on the AR authoring template, the AR content including a first trigger condition based on the first entry of the AR authoring template, the first trigger condition causing a second AR device executing the AR content to cause playback of the first media content item on a display of the second AR device in response to the second AR device determining that the second AR device is within a predetermined geographic distance of the first geographic location defined by the first set of special coordinates.

16. The non-transitory computer-readable medium of claim 15, wherein the AR authoring template includes a second entry corresponding to a second physical object located in the first defined region.

17. The non-transitory computer-readable medium of claim 16, the operations further comprising:

recording a video of the user physically manipulating the second physical object and demonstrating instructions for operating the second physical object, yielding a second media content item;

capturing a second set of spatial coordinates defining a second geographic location, the second geographic location indicating a location of the AR device while the AR device is recording the video of the user physically manipulating the second physical object and demonstrating instructions for operating the second physical object;

placing the second media content item and the second set of spatial coordinates in the second entry of the AR authoring template.

18. The non-transitory computer-readable medium of claim 17, wherein generating the AR content for the first defined region is further based on second entry of AR authoring template.

19. The non-transitory computer-readable medium of claim 18, wherein the AR content includes a second trigger condition based on the second entry of the AR authoring template, the second trigger condition causing the second AR device executing the AR content to cause playback of the second media content item on the display of the second AR device in response to the second AR device determining that the second AR device is within a predetermined geographic distance of the second geographic location defined by the second set of special coordinates.

20. The non-transitory computer-readable medium of claim 16, the operations further comprising:

transmitting the AR content to the remote server, the remote server providing the AR content to at least the second AR device.