Systems and methods for contextual augmented reality sharing and performance

• US 9,754,397 B1
• Filed: 04/07/2017
• Issued: 09/05/2017
• Est. Priority Date: 04/07/2017
• Status: Expired due to Fees

• Current Assignee: Patrick S. Piemonte, Ryan P. Staake
• Original Assignee: Mirage Worlds, Inc.
• Inventors: Piemonte, Patrick S., Staake, Ryan P.
• Primary Examiner(s): Chu, David H
• Application Number:
  US15/482,644
• Time in Patent Office: 151 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06Q 20/123 Shopping for digital content
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 84/12 WLAN [Wireless Local Area Networks]

Abstract

Methods, hardware, and software perform augmented reality created from a separate source in context with, such as synchronized and positioned in, captured media, so as to replicate a similar augmented reality at a different instance. A computer processor in a network of communications devices handles a combination of augmented reality information, anchor information that provides the context-matching, limitation information that controls if such information is transmitted or acted upon, and captured media information. The computer processor compares the anchor information with the media to identify triggering media and how augmented reality elements should appear in context with such media. If successful, the augmented reality is performed on a communications device based on the media. Augmented reality can thus be performed in strict reference to actually-captured media, in only desired or restricted conditions, and information transmission and processing to enable the same may be undertaken only at useful, resource-conserving instances.

Claims

1. A method of handling contextual augmented reality in a computer processor-based system, the method comprising:

receiving, with the computer processor, augmented reality information and anchor information for the augmented reality information, wherein the augmented reality information and the anchor information are received from a first user communications device;

receiving, with the computer processor, media independent of the augmented reality information and the anchor information, wherein the media is captured by a second user communications device at a different time from creation of the augmented reality information, and wherein the first user communications device and the second user communications device are separately owned;

comparing, with the computer processor, the anchor information with the media;

if the media is identified by the anchor information based on the comparing, performing the augmented reality information synchronized in the media in a manner defined by the anchor information.

2. The method of claim 1, wherein,

the anchor information identifies a first visual scene captured by a communications device and a position in the first visual scene,

the media is a second visual scene captured by a communications device,

the comparing compares the first visual scene and the second visual scene to determine if they are the same scene,

the augmented reality information is graphical, and

the performing performs the augmented reality information in the second visual scene at the position.

3. The method of claim 2, wherein the comparing includes zero mean normalized cross-correlating pixels of the first visual scene with pixels of the second visual scene to determine if the images correlate within a match threshold.

4. The method of claim 2, wherein the computer processor is in a mobile communications device having a camera capturing the second visual scene, and wherein the performing displays the augmented reality information in real time with the captures second visual scene on a screen of the mobile communications device.

5. The method of claim 1, further comprising:

receiving, with the computer processor, origin information limiting circumstances in which the augmented reality information may be performed;

receiving, with the computer processor, limitation information of a user communications device; and

comparing, with the computer processor, the limitation information with the origin information to determine if the user communications device satisfies the circumstances, wherein the performing is executed only if the user communications device satisfies the circumstances based on the comparing.

6. The method of claim 5, wherein the origin information is at least one of a geographic area containing subject matter identified by the anchor information and a span of time containing the subject matter identified by the anchor information, wherein the limitation information is at least one of a geographic position of the user communications device and a time of capture of the media, and wherein the circumstances are satisfied if at least one of the geographic position is within the geographic area and the time is within the timespan.

7. The method of claim 1, wherein the augmented reality information is a graphical user interface responsive to the media, and wherein the performing is executed in real time with the media being captured.

8. The method of claim 1, wherein the augmented reality information and the media are visual, and wherein the augmented reality information is sized, positioned, and moved in the media in a manner defined by the anchor information.

9. The method of claim 1, further comprising:

receiving, with the computer processor, origin information limiting circumstances in which the augmented reality information may be performed;

notifying a user of the origin information; and

if the performing is executed by the user, gathering at least one of feedback and payment from the user.

10. A method of augmenting a visual performance on a communications device, the method comprising:

receiving, with the communications device, visual information to be added to create augmented reality and anchor information including a description of eligible media and a placement of the visual information in the eligible media, wherein the visual information and the anchor information are received from a separately-owned communications device;

determining, with the communications device, a geographic position of the communications device,

receiving, with a camera of the communications device, independent visual media, wherein the media is captured at a different time from creation of the visual information and anchor information;

if the description matches the independent visual media and if the geographic position of the communications device is within a geographic area to which performance of the augmented reality is limited, performing the visual information in the independent visual media at the placement in the anchor information.

11. The method of claim 10, wherein the communications device is mobile, and wherein the determining is executed with at least one of GPS information and WiFi information through a service native to an operating system of the communications device.

12. The method of claim 10, further comprising:

notifying, with the communications device, a user of at least one of the description and the geographic area.

13. The method of claim 10, wherein the independent visual media is a live video captured by the camera of the communications device, and wherein the performing is executed in real time with the live video on a screen of the communications device.

14. The method of claim 10, further comprising:

receiving, with the communications device, origin information including the geographic area; and

comparing, with the communications device, the origin information with the geographic position of the communications device.

15. The method of claim 14, wherein the origin information includes additional user identification limitations, the method further comprising:

receiving, with the communications device, additional user identification, wherein the comparing further includes comparing the additional user identification with the additional user identification limitations, wherein the performing is executed only if the additional user identification satisfies the additional user identification limitations.

16. The method of claim 10, further comprising:

comparing the description with the independent visual media, wherein the comparing is performed only if the geographic position of the communications device is within a geographic area to which performance of the augmented reality is limited.

17. The method of claim 10, wherein the determining the geographic position of the communications device and the receiving the independent visual media are continuously repeated if the description does not match the independent visual media or if the geographic position of the communications device is not within the geographic area to which performance of the augmented reality is limited.

18. A processor-based network of communications devices for sharing and performing augmented reality, the network comprising:

a first user communications device connected to the network;

a second user communications device connected to the network;

an application host connected to the first user and the second user, wherein the application host is configured to,

receive, from the first user communications device, augmented reality information, anchor information for the augmented reality information, and origin information limiting circumstances in which the augmented reality information may be performed;

receive, from the second user communications device, limitation information of the second user communications device;

compare the origin information and the limitation information to determine if the second user communications device satisfied the limited circumstances; and

if the second user communications device satisfies the limited circumstances based on the comparing, transmitting the augmented reality information and anchor information to the second user communications device so that the second user communications device can perform the augmented reality information synchronized in media captured by the second user communications device in a manner defined by the anchor information.

19. The network of claim 18, wherein the application host is further configured to,

receive the media captured by the second communications device, and

compare the anchor information with the media to determine and match, wherein the transmitting the augmented reality information and anchor information to the second user communications device is executed only if a match is determined by the comparing.