Marker-based augmented reality authoring tools

• US 9,892,560 B2
• Filed: 09/11/2015
• Issued: 02/13/2018
• Est. Priority Date: 09/11/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Sudol, Jeremi M, Wiacek, John
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US14/851,897
• Publication Number:
  US 20160078683A1
• Time in Patent Office: 886 Days
• Field of Search: 345633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts

Abstract

An augmented reality-based content authoring tool is presented. A content author arranges machine-recognizable markers in a physical environment. A computing device operating as the authoring tool recognizes the markers and their arrangement based on a captured digital representation of the physical environment. Once recognized, augmented reality primitives corresponding to the markers can be bound together via their primitive interfaces to give rise to a content set. The individual primitives and content set are instantiated based on the nature of the marker'"'"'s arrangement.

Claims

1. An augmented reality-based content authoring tool comprising:

at least one processor;

at least one non-transitory computer readable memory for storing software instructions executable by said at least one processor;

a recognition engine executable on said at least one processor according to the stored software instructions configured to;

receive a digital representation of an arrangement of a marker set in a physical environment; and

identify each marker'"'"'s identity in the marker set from the digital representation; and

identify at least one inter-connectable AR object primitive for each marker in the marker set from each marker'"'"'s identity; and

a content creation engine executable on said at least one processor according to the stored software instructions configured to;

obtain a content creation rules set from multiple rules sets based on a device context;

obtain a set of AR content primitives comprising the at least one inter-connectable AR object primitive for each marker in the marker set, each AR object primitive including a primitive interface;

generate an AR content set by positionally coupling at least two primitives in the set of AR content primitives via their primitive interfaces according to the content creation rules set as a function of the arrangement in the physical environment of the markers corresponding to the at least two primitives; and

cause a device to present the AR content set on a display where the AR content is superimposed on an image of the physical environment based on the arrangement.

2. The tool of claim 1, wherein markers in the marker set comprise cards.

3. The tool of claim 1, wherein markers in the marker set comprise three dimensional objects.

4. The tool of claim 1, wherein markers in the marker set comprise a scale invariant set of features.

5. The tool of claim 1, wherein the markers comprise features at multiple scales in the frequency domain.

6. The tool of claim 1, wherein the content creation rules set further comprises an application specific rules set.

7. The tool of claim 1, wherein the multiple rules sets are further based on recognizing the marker set.

8. The tool of claim 1, wherein the AR primitives comprise program code primitives.

9. The tool of claim 1, wherein the primitive interface comprises an application program interface.

10. The tool of claim 1, wherein the primitive interface comprises a graphical interface.

11. The tool of claim 1, wherein the primitive interface comprises an intermediary interface server.

12. The tool of claim 1, wherein the AR content set composes a new AR primitive.

13. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their relative proximity in the arrangement.

14. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their relative orientation in the arrangement.

15. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their location in the arrangement.

16. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their movement within in the arrangement.

17. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of a stacking within in the arrangement.

18. The tool of claim 1, wherein the content creation engine is further configured to cause the AR content set to be presented on the display relative to images of the market set.

19. The tool of claim 1, wherein the content creation engine is further configured to track position of markers in the marker set.

20. The tool of claim 1, wherein the content creation engine is further configured to couple the at least two primitives based on a pair-wise juxtaposition of markers in the marker set.

21. The tool of claim 1, wherein the content creation engine is further configured to update the content set based on movement of markers in the marker set within physical environment.

22. The tool of claim 21, wherein the content creation engine is further configured update the content set in real-time.

23. The tool of claim 1, wherein the recognition engine is further configured to register the at least one AR content primitive to a marker in the marker set.

24. The tool of claim 1, further comprising a mobile device that includes the recognition engine.

25. The tool of claim 24, wherein the mobile device further includes the content creation tool.

26. The tool of claim 24, wherein the mobile device comprises at least one of the following:

a game device, a smart phone, a tablet, a camera, a robot, an augmented reality viewer, and a watch.