User-generated content in a virtual reality environment

• US 9,338,589 B2
• Filed: 04/15/2014
• Issued: 05/10/2016
• Est. Priority Date: 05/10/2012
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Aurasma Limited (HP Inc.)
• Inventors: Loxam, James, Blanchflower, Sean Mark
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Nguyen, Vu
• Application Number:
  US14/253,757
• Publication Number:
  US 20140253743A1
• Time in Patent Office: 756 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/185 by embedding added-value information into content, e.g. geo-tagging

Abstract

The systems and methods allow users of mobile computing devices to generate augmented reality scenarios. Augmented reality content is paired with a real world trigger item to generate the augmented reality scenario. The augmented reality content is overlaid onto frames on a video when a trigger item is detected. Each mobile computing device may have an augmented reality application resident on the mobile computing device to allow a user to generate the augmented reality scenarios.

Claims

1. A method for generating augmented reality scenarios, comprising:

receiving, on a mobile computing device, a notification indicating a published augmented reality (AR) scenario proximate to a current location of the mobile computing device, the published AR scenario showing a matching AR item with a real world item;

initiating, on the mobile computing device, an AR application that matches data corresponding to a plurality of real-world trigger items with a plurality of AR items in a memory index;

monitoring, using a camera of the mobile computing device, a video stream from the camera;

identifying the real-world item, corresponding to the published AR scenario, in the video stream;

comparing features of the real-world item to the data corresponding to the plurality of real-world trigger items in the memory index to determine whether the real-world item corresponds to one of the plurality of real-world trigger items;

in response to determining that the real-world item corresponds to a respective one of the plurality of real-world trigger items, selecting the matching AR item from the plurality of AR items, the matching AR item being associated with the respective real-world trigger item in the memory index; and

generating the published AR scenario on a display of the mobile computing device by superimposing the matching AR item onto the video stream;

wherein the notification is received based on a user of the mobile computing device being subscribed to an interest group that enables subscribers of the interest group to create and share AR scenarios.

2. The method of claim 1, wherein the matching AR item comprises at least one of a 3-dimensional (3D) object, video content associated with the real-world item, or a uniform resource locator (URL) link to a website associated with the real-world item.

3. The method of claim 1, further comprising:

while generating the published AR scenario, sizing, scaling, and positioning the matching AR item in the video stream in relation to a size, scale, and position of the real-world item in the video stream.

4. The method of claim 1, wherein the matching AR item comprises information related to the real-world item.

5. The method of claim 1, wherein comparing features of the real-world item to the data corresponding to the plurality of real-world trigger items comprises breaking the real-world item down into dot-to-dot X-Y coordinates and performing a hierarchical filtering operation of the plurality of real-world trigger items in the memory index.

6. The method of claim 1, wherein each of the plurality of real-world trigger items are stored in the memory index with metadata comprising location coordinates, and wherein comparing features of the real-world item to the data corresponding to the plurality of real-world trigger items comprises determining the current location of the mobile computing device and comparing the current location to the location coordinates for the plurality of real-world trigger items in the memory index.

7. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a mobile computing device, cause the mobile computing device to:

receive a notification indicating a published augmented reality (AR) scenario proximate to a current location of the mobile computing device, the published AR scenario showing a matching AR item with a real world item;

initiate an AR application that matches data corresponding to a plurality of real-world trigger items with a plurality of AR items in a memory index;

monitor, using a camera of the mobile computing device, a video stream from the camera;

identify the real-world item, corresponding to the published AR scenario, in the video stream;

compare features of the real-world item to the data corresponding to the plurality of real-world trigger items in the memory index to determine whether the real-world item corresponds to one of the plurality of real-world trigger items;

in response to determining that the real-world item corresponds to a respective one of the plurality of real-world trigger items, select the matching AR item from the plurality of AR items, the matching AR item being associated with the respective real-world trigger item in the memory index; and

generate the published AR scenario on a display of the mobile computing device by superimposing the matching AR item onto the video stream;

wherein the notification is received based on a user of the mobile computing device being subscribed to an interest group that enables subscribers of the interest group to create and share AR scenarios.

8. The non-transitory computer-readable medium of claim 7, wherein the matching AR item comprises at least one of a 3-dimensional (3D) object, video content associated with the real-world item, or a uniform resource locator (URL) link to a website associated with the real-world item.

9. The non-transitory computer-readable medium of claim 7, wherein the executed instructions further cause the mobile computing device to:

while generating the published AR scenario, size, scale, and position the matching AR item in the video stream in relation to a size, scale, and position of the real world item in the video stream.

10. The non-transitory computer-readable medium of claim 7, wherein the matching AR item comprises information related to the real world item.

11. The non-transitory computer-readable medium of claim 7, wherein comparing features of the real-world item to the data corresponding to the plurality of real-world trigger items comprises breaking the real-world item down into dot-to-dot X-Y coordinates and performing a hierarchical filtering operation of the plurality of real-world trigger items in the memory index.

12. The non-transitory computer-readable medium of claim 7, wherein each of the plurality of real-world trigger items are stored in the memory index with metadata comprising location coordinates, and wherein comparing features of the real-world item to the data corresponding to the plurality of real-world trigger items comprises determining the current location of the mobile computing device and comparing the current location to the location coordinates for the plurality of real-world trigger items in the memory index.

13. A mobile computing device, comprising:

a camera;

a display;

one or more memory resources storing instructions; and

a processor, coupled to the camera, the display, and the one or more memory resources, to execute the instructions, causing the mobile computing device to;

receive a notification indicating a published augmented reality (AR) scenario proximate to a current location of the mobile computing device;

initiate an AR application that matches data corresponding to a plurality of real-world trigger items with a plurality of AR items in a memory index;

monitor, using the camera, a video stream from the camera;

identify the real-world item, corresponding to the published AR scenario, in the video stream;

compare features of the real-world item to the data corresponding to the plurality of real-world trigger items in the memory index to determine whether the real-world item corresponds to one of the plurality of real-world trigger items;

in response to determining that the real-world item corresponds to a respective one of the plurality of real-world trigger items, select the matching AR item from the plurality of AR items, the matching AR item being associated with the respective real-world trigger item; and

generate the published AR scenario on the display by superimposing the matching AR item onto the video stream;

wherein the notification is received based on a user of the mobile computing device being subscribed to an interest group that enables subscribers of the interest group to create and share AR scenarios.

14. The mobile computing device of claim 13, wherein the matching AR item comprises at least one of a 3-dimensional (3D) object, video content associated with the real-world item, or a uniform resource locator (URL link to a website associated with the real-world item.

15. The mobile computing device of claim 13, wherein the executed instructions further cause the mobile computing device to:

while generating the published AR scenario, size, scale, and position the matching AR item in the video stream in relation to a size, scale, and position of the real-world item in the video stream.

16. The mobile computing device of claim 13, wherein the matching AR item comprises information related to the real-world item.

17. The mobile computing device of claim 13, wherein comparing features of the real-world item to the data corresponding to the plurality of real-world trigger items comprises breaking the real-world item down into dot-to-dot X-Y coordinates and performing a hierarchical filtering operation of the plurality of real-world trigger items in the memory index.