Marker-based augmented reality authoring tools

• US 10,424,123 B2
• Filed: 12/18/2017
• Issued: 09/24/2019
• Est. Priority Date: 09/11/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Sudol, Jeremi M, Wiacek, John
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US15/846,041
• Publication Number:
  US 20180108181A1
• Time in Patent Office: 645 Days
• Field of Search: 345633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts

Abstract

An augmented reality-based content authoring tool is presented. A content author arranges machine-recognizable markers in a physical environment. A computing device operating as the authoring tool recognizes the markers and their arrangement based on a captured digital representation of the physical environment. Once recognized, augmented reality primitives corresponding to the markers can be bound together via their primitive interfaces to give rise to a content set. The individual primitives and content set are instantiated based on the nature of the marker'"'"'s arrangement.

Claims

1. A content authoring tool comprising:

at least one processor;

at least one non-transitory computer readable memory for storing software instructions executable by said at least one processor;

a recognition engine executable on said at least one processor according to the stored software instructions configured to;

receive a digital representation of an arrangement of a marker set in a physical environment;

identify each recognized marker'"'"'s identity in the marker set from the digital representation; and

identify at least one inter-connectable object primitive for each recognized marker in the marker set from each recognized marker'"'"'s identity; and

a content creation engine executable on said at least one processor according to the stored software instructions configured to;

obtain a content creation rules set from available rules sets based on a device context;

obtain a set of content primitives comprising the at least one inter-connectable object primitive for each recognized marker in the marker set, each object primitive including a primitive interface;

generate a content set by positionally coupling at least two primitives in the set of content primitives via their primitive interfaces according to the content creation rules set as a function of the arrangement in the physical environment of the markers corresponding to the at least two primitives; and

cause a device to present the content set on a display based on the arrangement.

2. The tool of claim 1, wherein markers in the marker set comprise cards.

3. The tool of claim 1, wherein markers in the marker set comprise three dimensional objects.

4. The tool of claim 1, wherein markers in the marker set comprise a scale invariant set of features.

5. The tool of claim 1, wherein the markers comprise features at multiple scales in the frequency domain.

6. The tool of claim 1, wherein the content creation rules set further comprises an application specific rules set.

7. The tool of claim 1, wherein the available rules sets are further based on recognizing the marker set.

8. The tool of claim 1, wherein the primitives comprise program code primitives.

9. The tool of claim 1, wherein the primitive interface comprises an application program interface.

10. The tool of claim 1, wherein the primitive interface comprises a graphical interface.

11. The tool of claim 1, wherein the primitive interface comprises an intermediary interface server.

12. The tool of claim 1, wherein the content set composes a new primitive.

13. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their relative proximity in the arrangement.

14. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their relative orientation in the arrangement.

15. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their location in the arrangement.

16. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of their movement within in the arrangement.

17. The tool of claim 1, wherein the at least two primitives are coupled via their primitive interfaces according to the content creation rules set as a function of a stacking within in the arrangement.

18. The tool of claim 1, wherein the content creation engine is further configured to cause the content set to be presented on the display relative to images of the market set.

19. The tool of claim 1, wherein the content creation engine is further configured to track position of markers in the marker set.

20. The tool of claim 1, wherein the content creation engine is further configured to couple the at least two primitives based on a pair-wise juxtaposition of markers in the marker set.

21. The tool of claim 1, wherein the content creation engine is further configured to update the content set based on movement of markers in the marker set within physical environment.

22. The tool of claim 21, wherein the content creation engine is further configured update the content set in real-time.

23. The tool of claim 1, wherein the recognition engine is further configured to register the at least one content primitive to a marker in the marker set.

24. The tool of claim 1, further comprising a mobile device that includes the recognition engine.

25. The tool of claim 24, wherein the mobile device further includes the content creation tool.

26. The tool of claim 24, wherein the mobile device comprises at least one of the following:

a game device, a smart phone, a tablet, a camera, a robot, an augmented reality viewer, and a watch.

27. The tool of claim 1, wherein the at least one inter-connectable object primitive for each recognized marker in the mark set comprises a programming module or a line of code.

28. A non-transitory computer readable medium storing software instructions executable by at least one processor to perform operations comprising:

receiving a digital representation of an arrangement of a marker set in a physical environment;

identifying each recognized marker'"'"'s identity in the marker set from the digital representation;

identifying at least one inter-connectable object primitive for each recognized marker in the marker set from each recognized marker'"'"'s identity;

obtaining a content creation rules set from available rules sets based on a device context;

obtaining a set of content primitives comprising the at least one inter-connectable object primitive for each recognized marker in the marker set, each object primitive including a primitive interface;

generating a content set by positionally coupling at least two primitives in the set of content primitives via their primitive interfaces according to the content creation rules set as a function of the arrangement in the physical environment of the markers corresponding to the at least two primitives; and

causing a device to present the content set on a display based on the arrangement.

29. A method comprising:

receiving a digital representation of an arrangement of a marker set in a physical environment;

identifying each recognized marker'"'"'s identity in the marker set from the digital representation;

identifying at least one inter-connectable object primitive for each recognized marker in the marker set from each recognized marker'"'"'s identity;

obtaining a content creation rules set from available rules sets based on a device context;

obtaining a set of content primitives comprising the at least one inter-connectable object primitive for each recognized marker in the marker set, each object primitive including a primitive interface;

generating a content set by positionally coupling at least two primitives in the set of content primitives via their primitive interfaces according to the content creation rules set as a function of the arrangement in the physical environment of the markers corresponding to the at least two primitives; and

causing a device to present the content set on a display based on the arrangement.