Local cache of augmented reality content in a mobile computing device

• US 9,064,326 B1
• Filed: 05/10/2012
• Issued: 06/23/2015
• Est. Priority Date: 05/10/2012
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Longsand Limited (Open Text Corporation)
• Inventors: Loxam, James, Blanchflower, Sean Mark
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Nguyen, Vu
• Application Number:
  US13/468,261
• Time in Patent Office: 1,139 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

An augmented reality application is coded to operate on a mobile computing device. A routine allocates a portion of a memory in the mobile computing to be a local cache for the downloadable application to store augmented reality content and information and characteristics concerning a plurality of real world trigger items. A space utilization algorithm and a content selection algorithm are incorporated into the augmented reality application. The amount and ways the real world trigger items and augmented reality content are stored is balanced against an amount of memory space set or allowed for a size of the local cache on that particular mobile computing device.

Claims

1. A non-transitory computer readable storage medium storing an augmented reality application that upon execution causes a mobile computing device to:

cooperate with an operating system of the mobile computing device to cause the operating system to allocate a portion of a memory in the mobile computing device to be a local cache for the augmented reality application to store augmented reality content and information and characteristics associated with real world trigger items;

balance, using space utilization instructions, storing in the local cache a subset of the augmented reality content from at least one database of a server system to be displayed as part of an augmented reality scenario on a display screen of the mobile computing device and storing in the local cache the real world trigger items, against an amount of memory space set for the local cache on the mobile computing device, the balancing further comprising determining whether actual data of the subset of the augmented reality content or just hyperlinks to at least one source containing the actual data are stored in the local cache; and

predict, using content selection instructions, relevant augmented reality content and real world trigger items that are most relevant for a particular user.

2. The non-transitory computer readable storage medium of claim 1, wherein the content selection instructions are further configured to store in the local cache 1) user selected augmented reality content, 2) augmented reality content generated by the particular user, and 3) intelligence engine selected augmented reality content, and the augmented reality application upon execution causes the mobile computing device to:

provide the augmented reality content overlaid onto frames of a video stream in the mobile computing device when a first trigger item is detected, and where use of data in the local cache reduces an amount of wireless communications between the mobile computing device and the server system connected over a wireless network with the mobile computing device.

3. The non-transitory computer readable storage medium of claim 1, wherein the augmented reality application upon execution causes the mobile computing device to:

store characteristics and geographical information associated with potential trigger item objects, where the potential trigger item objects include any combination of

     1) predetermined stock objects and images maintained by an object database in the server system connected over a network with the mobile computing device,

     2) custom objects and images selected and created by a user of the augmented reality application to be used as a trigger item, or

     3) both;

compare current characteristics and geographical information of objects found in or associated with one or more frames in a video stream to the stored characteristics and geographical information to identify a trigger item; and

provide augmented reality content overlaid onto the one or more frames of the video stream in the mobile computing device, where selected augmented reality content is overlaid onto the one or more frames before the video stream is displayed on the display screen of the mobile computing device.

4. The non-transitory computer readable storage medium of claim 1, wherein the augmented reality application upon execution causes the mobile computing device to:

cooperate with a backend Internet-based management server system to receive content to store in the local cache based on selection by the content selection instructions, where the content selection instructions are further configured to choose to store in the local cache at least one of;

     1) user selected augmented reality content in the local cache, including channels subscribed to,

     2) trigger items and augmented reality content that support particular themed kernels of the augmented reality application downloaded onto the mobile computing device,

     3) trigger items and augmented reality content that an intelligence engine selects as relevant based on intelligence factors including a geographic location of the mobile computing device, and

     4) augmented reality content generated by the particular user.

5. The non-transitory computer readable storage medium of claim 1, wherein one or more databases in the server system have different instances of the augmented reality scenario customized based on human languages set for the mobile computing device, and the content selection instructions intelligently select, from the instances, an instance of the augmented reality scenario to be downloaded based on a human language set for the mobile computing device.

6. The non-transitory computer readable storage medium of claim 1, wherein one or more databases in the server system have different instances of the augmented reality scenario customized based on geographic locations of the mobile computing device, and the content selection instructions intelligently select, from the instances, an instance of the augmented reality scenario to be downloaded based on a current geographic location of the mobile computing device.

7. The non-transitory computer readable storage medium of claim 1, wherein the augmented reality application upon execution causes the mobile computing device to:

interface with other blocks of functionality in the mobile computing device to determine a geographic location of the mobile computing device, a selected human language of operation of the mobile computing device, browsing shopping habits of the particular user of the mobile computing device, and any friends of the particular user in an online social network.

8. A method, comprising:

allocating a portion of a memory of a mobile computing device to be a local cache for an augmented reality application to store augmented reality content and one or more trigger items, where the augmented reality content and one or more trigger items form a subset of augmented reality content and trigger items stored in one or more databases in a server system connected through a wireless network with the mobile computing device;

balancing, using space utilization instructions executed in the mobile computing device, storing in the local cache the subset of the augmented reality content from the one or more databases in the server system to be displayed as part of an augmented reality scenario on a display screen of the mobile computing device and storing in the local cache the one or more trigger items, against an amount of memory space set for the local cache on the mobile computing device, the balancing further comprising determining whether actual data of the subset of the augmented reality content or just hyperlinks to at least one source containing the actual data are stored in the local cache;

predicting, by content selection instructions executed in the mobile computing device, relevant content for a particular user; and

interfacing, using snooping instructions executed in the mobile computing device, with other blocks of functionality in the mobile computing device to determine a geographic location of the mobile computing device, a selected human language of operation of the mobile computing device, browsing shopping habits of the particular user, and any friends of the particular user in an online social network.

9. The method of claim 8, further comprising:

1) storing user selected augmented reality content in the local cache, and (2) then as space permits storing augmented reality content selected by an intelligence engine in the local cache based on intelligence factors including the shopping habits of the particular user, the friends of the particular user in the online social network, and the geographic location of the mobile computing device, and (3) then as space permits storing augmented reality content generated by the particular user in the local cache.

10. The method of claim 8, wherein a combination of storing user selected augmented reality content, augmented reality content generated by the particular user, and intelligence engine selected augmented reality content in the local cache reduces an amount of wireless communications between the mobile computing device and a server system connected over a wireless network with the mobile computing device.

11. The method of claim 8, further comprising:

selecting a subset of augmented reality content from an Internet-based set of databases for the particular user and storing the selected subset of augmented reality content in the local cache;

allowing third party generated augmented reality content to have different instances that include different augmented reality scenarios customized based on human languages set for the mobile computing device; and

intelligently selecting, from among the instances, an instance to be downloaded based on a human language set for the mobile computing device.

12. The method of claim 8, further comprising:

allowing third party generated augmented reality content to have different instances including different augmented reality scenarios customized based on geographic locations of the mobile computing device;

intelligently selecting, from among the instances, an instance to be downloaded based on a current geographic location of the mobile computing device.

13. The method of claim 8, further comprising:

causing augmented reality content to be overlaid onto frames of a video stream in the mobile computing device, where selected augmented reality content is overlaid onto the frames before the video stream is displayed on a display screen of the mobile computing device.

14. A system comprising:

at least one physical processor to;

allocate a portion of a memory of the mobile computing device to be a local cache for an augmented reality application to store augmented reality content and one or more trigger items, where the augmented reality content and one or more trigger items form a subset of augmented reality content and trigger items stored in one or more databases in a server system that connects through a wireless network with the mobile computing device;

execute space utilization instructions to store, in the local cache, the one or more trigger items and the subset of the augmented reality content in the server system and hyperlinks to remotely stored data files used in a display of an augmented reality scenario on the mobile computing device;

balance (1) the storing of the subset of augmented reality data content in the server system and (2) whether actual data of the subset of augmented reality content is stored or just hyperlinks to at least one source containing the actual data is stored, based on an amount of memory space set for the local cache in the mobile computing device; and

execute content selection instructions to predict relevant content for a particular user.

15. The system of claim 14, wherein the at least one processor is to further:

store characteristics and geographical information associated with potential trigger item objects;

compare current characteristics and geographical information of objects found in or associated with one or more frames in a video stream to the stored characteristics and geographical information to identify a trigger item; and

provide augmented reality content overlaid onto the one or more frames in the video stream in the mobile computing device.

16. The system of claim 14, wherein the at least one processor is to further:

store user selected augment reality content in the local cache, (2) then as space permits store augmented reality content selected by an intelligence engine in the local cache based on intelligence factors including shopping habits of the particular user, the particular user'"'"'s network of friends from one or more social network sites, and a geographic location of the mobile computing device, and (3) then as space permits store augmented reality content generated by the particular user in the local cache.