Context-aware tagging for augmented reality environments

• US 9,286,323 B2
• Filed: 02/25/2013
• Issued: 03/15/2016
• Est. Priority Date: 02/25/2013
• Status: Expired due to Fees

• Current Assignee: Maplebear Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Cardonha, Carlos H., Koch, Fernando L., Kraemer, James R.
• Primary Examiner(s): NGUYEN, CAM LINH T
• Application Number:
  US13/775,694
• Publication Number:
  US 20140244595A1
• Time in Patent Office: 1,114 Days
• Field of Search: 707/707, 707/758
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A method for tag-based search includes capturing an image, extracting a tag from the image, identifying a location associated with the captured image, and querying stored content for information that matches the location and the tag. Local storage is checked for the information first, and remote storage may be checked subsequently. Any located information may be used to augment the image. Information located in the remote storage may be saved in the local storage until it reaches a certain age, until it fails to be accessed for a threshold period of time, or until the location moves outside a threshold radius associated with a location of the information located in the remote storage.

Claims

1. A method for augmenting an image, the method comprising:

capturing an image using a mobile device;

extracting a machine-readable label from the image;

querying stored content for information that is associated with the machine-readable label, wherein the information includes non-location information;

identifying a location associated with the mobile device;

generating a tentative identification of an object depicted in the image, based at least in part on the information;

producing a model of a statistical distribution of objects that share characteristics with the object and that are known to be present in the location associated with the system;

inferring a probability that the tentative identification is correct, from the model of the statistical distribution; and

augmenting the image by visually superimposing at least some of the information over the image, wherein the at least some of the information is related to the tentative identification, when the probability indicates that the tentative identification is likely to be correct,

wherein at least one of;

the extracting, the querying, or the augmenting is performed using a processor of the mobile device.

2. The method of claim 1, wherein the machine-readable label is a bar code.

3. The method of claim 1, wherein the machine-readable label is a color code.

4. The method of claim 1, wherein the identifying is performed using information from an open-air tracking system.

5. The method of claim 4, wherein the open-air tracking system comprises a global positioning system.

6. The method of claim 4, wherein the open-air tracking system comprises a Bluetooth beacon.

7. The method of claim 4, wherein the open-air tracking system comprises a cellular communications tower.

8. The method of claim 4, wherein the open-air tracking system comprises a radio frequency identification tag.

9. The method of claim 1, wherein the querying comprises:

checking local storage for the information.

10. The method of claim 9, further comprising:

checking remote storage for the information after checking the local storage.

11. The method of claim 10, further comprising:

saving in the local storage a portion of the information that is located in the remote storage.

12. The method of claim 11, further comprising:

deleting from the local storage the portion of the information when the portion of the information reaches a threshold age.

13. The method of claim 11, further comprising:

deleting from the local storage the portion of the information when the location is determined to be outside a threshold radius of the location.

14. The method of claim 11, further comprising:

deleting from the local storage the portion of the information when the portion of the information is not accessed for a threshold period of time.

15. The method of claim 1, wherein the information includes a distribution of content from the stored content that is associated with tags that are similar to the machine-readable label and associated with the location.

16. The method of claim 1, wherein an identifier associated with the machine-readable label is non-unique, and wherein the machine-readable label is resolved based on a geographic context inferred from the location associated with the mobile device.

17. The method of claim 1, further comprising:

identifying a direction in which the mobile device is positioned.

18. The method of claim 17, wherein the statistically analyzing the information is further performed in accordance with the direction in which the mobile device is positioned.