User-generated content in a virtual reality environment

• US 9,066,200 B1
• Filed: 05/10/2012
• Issued: 06/23/2015
• Est. Priority Date: 05/10/2012
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Longsand Limited (Open Text Corporation)
• Inventors: Loxam, James, Blanchflower, Sean Mark
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Nguyen, Vu
• Application Number:
  US13/468,222
• Time in Patent Office: 1,139 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/185 by embedding added-value information into content, e.g. geo-tagging

Abstract

The systems and methods allow users of mobile computing devices to generate augmented reality scenarios. Augmented reality content is paired with a real world trigger item to generate the augmented reality scenario. The augmented reality content is overlaid onto frames on a video when a trigger item is detected. Each mobile computing device may have an augmented reality application resident on the mobile computing device to allow a user to generate the augmented reality scenarios.

Claims

1. A computer-aided method comprising:

selecting augmented reality content to include in an augmented reality scenario;

selecting a real world trigger item for activating the augmented reality scenario;

pairing the augmented reality content with the real world trigger item to generate the augmented reality scenario;

submitting the augmented reality scenario to a server system that cooperates with multiple instances of the augmented reality application, where the augmented reality scenario is indexed and stored in at least one of a channel or an interest group;

pushing the augmented reality scenario indexed and stored in the at least one channel or interest group to at least all of the instances of the augmented reality application associated with subscribers to the at least one channel or interest group;

monitoring, by an augmented reality application resident on a mobile computing device, a video stream captured by a video camera of the mobile computing device;

detecting, by the augmented reality application, the real world trigger item in the video stream captured by the video camera of the mobile computing device;

in response to detecting the real world trigger item in the video stream, overlaying the augmented reality content onto frames of the video stream for presentation on a display of the mobile computing device.

2. The method of claim 1, further comprising:

sizing, scaling, and positioning the augmented reality content in the augmented reality scenario to a desired effect in relation to the real-world trigger item; and

storing the submitted augmented reality scenario paired with its real world trigger item in one or more databases in the server system.

3. The method of claim 1, further comprising:

activating an icon within the augmented reality application to invoke a step-by-step built-in wizard routine to guide the user through generating the augmented reality scenario.

4. The method of claim 1, further comprising:

previewing the augmented reality scenario prior to submitting the augmented reality scenario to the server system.

5. The method of claim 1, further comprising:

allowing a user of the augmented reality application to share the augmented reality scenario by uploading the augmented reality scenario into the one or more databases;

publish the augmented reality scenario in a first channel, which publishes the augmented reality scenario for all the instances of the augmented reality application to view and potentially download; and

presenting an option of making the augmented reality scenario to be held private and thus be visible to this user only.

6. The method of claim 1, further comprising:

selecting one or more real world trigger items to be paired with the user-generated augmented reality scenario from any of

     1) a target image and

     2) a target object of stock images of real world images and objects that are already known and identified by the server system, or selecting one or more custom real world trigger items from any of

     1) a target image and

     2) a target object image from another source including any of

     1) a source on the Internet and

     2) a source file local on the mobile computing device, where when the user indicates to the augmented reality application the user'"'"'s desire to select a custom real world trigger item, then the augmented reality application displays an indication on the display screen that assists the user to choose a satisfactory angle and exposure of the target image or target object, and visually conveys the indication when the augmented reality application determines that a satisfactory angle and exposure of the target image or target object will work well as a trigger item, and then the augmented reality application is configured to capture and store one or more satisfactory angles and exposures of the target image or target object and then upload those for trigger item processing in server system.

7. The method of claim 1, wherein the augmented reality scenario is generated on a second instance of the augmented reality application resident on a second mobile computing device.

8. The method of claim 7, wherein the augmented reality application resident on the mobile computing device subscribes to a channel associated with user created augmented reality content from the second instance of the mobile application resident on the second mobile computing device.

9. The method of claim 1, wherein detecting the real world trigger item comprises comparing objects in the video stream to known real world trigger items stored in 1) a database communicatively connected to the mobile computing device over the network, 2) a local cache in the mobile computing device and 3) any combination of the two.

10. The method of claim 1, further comprising:

creating an interest group around a category of created augmented reality scenarios including user-created augmented reality scenarios and selected third party augmented reality scenarios to share to those augmented reality scenarios to instances of the augmented reality application associated with users subscribed to that interest group or with user behavior that indicates interest in that category of created augmented reality scenarios.

11. A computer-aided method comprising:

storing augmented reality content paired with one or more real world trigger items and their geographic location for a location-based augmented reality scenario, which was created by a user from a first instance of the augmented reality application;

publishing the location-based augmented reality scenario in a channel or group of an on-line social network, wherein the augmented reality application is configured to present the option of making the augmented reality scenario private by a user who generated the augmented reality scenario so that the augmented reality scenario cannot be viewed by other instances of the downloadable augmented reality application;

transmitting, using a push protocol over a network, the location-based augmented reality scenario to instances of the augmented reality application that subscribe to that channel or group of the on-line social network; and

configuring a second instance of the augmented reality application resident on a mobile computing device to, when it detects that the Global Position System of the mobile computing device indicates coordinates that are near by a geographic location of the location-based augmented reality scenario, cause an icon to appear on a display on the mobile computing device indicating a direction and how far away the geographic location of the mobile computing device is relative to a geographic location of the location-based augmented reality scenario that is located nearby.

12. The method of claim 11, further comprising displaying the icon without including the one or more real world trigger items or the geographic location in the display.

13. A mobile computing device comprising:

a hardware processor;

a video camera;

a display screen;

a communication module to wirelessly connect to a server system via a network;

a trigger item engine executable on the hardware processor to access characteristics associated with potential trigger item objects associated with a plurality of augmented reality scenarios, where each of the potential trigger item objects is to activate one of the plurality of augmented reality scenarios, where each of the plurality of augmented reality scenarios is published in a channel or group subscribed to by a user of the mobile computing device;

a trigger detection engine executable on the hardware processor to compare current characteristics of physical objects found in one or more frames of a video stream to the characteristics to identify a particular trigger item, where the video stream is captured by the video camera of the mobile computing device; and

an augmentation engine executable on the hardware processor to, in response to an identification of the particular trigger item, provide augmented reality content overlaid onto the frames of the video stream in the mobile computing device, where the augmented reality content is overlaid onto the frames before the video stream is displayed on the display screen of the mobile computing device, where the augmented reality content is defined by a particular augmented reality scenario activated by the particular trigger item.

14. The mobile computing device of claim 13, where the augmentation engine is also configured to allow a user to create augmented reality content from stock locations including any combination of 1) off of a local memory of the mobile computing device, 2) from Internet sources, and 3) from an augment information database maintained at the server system, and then to associate that augmented reality content with at least one trigger item from the trigger item engine;

and

a step-by-step built-in wizard routine is part of the augmented reality application and is configured to guide the user through performing the operations to create the augmented reality scenario.

15. The mobile computing device of claim 13, wherein the augmented reality content is generated on a second mobile computing device.

16. The mobile computing device of claim 13, further comprising:

a local cache that provides a dedicated local storage area on the mobile computing device that is used to replicate portions of the various databases maintained at the server system including known trigger item information and augmented reality content information, for sole use by the augmented reality application in the mobile computing device, and

where the augmented reality application is configured to cooperate with a transmitter in the communication module of the mobile computing device to transmit the user-generated augmented reality scenarios to the server system located over the wide area network such that the augmented reality scenarios can be added to a user'"'"'s channel and shared with other users over a social networking site.