IMAGE MATCHING TO AUGMENT REALITY

• US 20150186730A1
• Filed: 03/11/2015
• Published: 07/02/2015
• Est. Priority Date: 11/02/2009
• Status: Abandoned Application

• Current Assignee: Huawei Device Company Limited (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Hannaford, Blake
• Application Number:
  US14/644,957
• Publication Number:
  US 20150186730A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04L 67/289 Intermediate processing functionally located close to the data consumer application, e.g. in same machine, in same home or in same sub-network
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/561 Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Technology for augmenting reality using image matching is described. The technology can receive a captured image, determine location information relating to the received image, and transmit the location information to a server computing device. The technology may then receive one or more tags from the server computing device, wherein each tag can include one or more patches and supplementary information corresponding to at least one of the patches and each patch can be associated with the location information. The technology can then determine using an image matching process whether at least one of the received one or more patches appears in the received image, and if a patch appears in the received image, render the supplementary information corresponding to the patch.

Claims

1. A method performed by a server computing device to augment reality, comprising:

receiving a captured image;

receiving a first location information relating to the received captured image;

determining a first tag of one or more tags,

wherein a tag includes one or more templates and supplementary information corresponding to at least one of the one or more templates,

wherein a template includes one or more images and a second location information that specifies a second location that is based on a first location specified by the received first location information, and

wherein determining the first tag of the one or more tags includes;

determining, using a template matching process, whether at least one of a first one or more images associated with a first template appears in the received captured image, wherein the first template is associated with the first tag; and

in response to the determination that at least one of the first one or more images associated with the first template appears in the received captured image, selecting the first tag; and

transmitting the first tag to a client computing device to enable display of a first supplementary information corresponding to the first template.

2. The method of claim 1, wherein the second location is at or near the first location specified by the received first location information.

3. The method of claim 1, wherein the second location is a larger geographic area than the first location specified by the received first location information.

4. The method of claim 1, wherein the first supplementary information is an advertisement.

5. The method of claim 1, wherein the determination that at least one of the first one or more images associated with the first template appears in the received captured image includes a determination that the at least one of the first one or more images matches less than the entire received captured image.

6. The method of claim 1, wherein the determining of the first tag of the one or more tags further includes selecting templates to be used in the template matching process based on a predetermined association between the selected templates and the first location.

7. A method performed by a computing system to augment reality, comprising:

receiving a captured image;

determining a first location information relating to the received captured image;

transmitting the determined first location information to a server computing device;

receiving a first tag of one or more tags from the server computing device,

wherein a tag includes one or more templates and supplementary information corresponding to at least one of the one or more templates, and

wherein a template includes one or more images and a second location information that specifies a second location that is based on a first location specified by the determined first location information;

determining, using a template matching process, whether at least one of a first one or more images associated with a first template appears in the received captured image, wherein the first template is associated with the received first tag; and

in response to the determination that at least one of the first one or more images associated with the first template appears in the received captured image, rendering a first supplementary information corresponding to the first template.

8. The method of claim 7, wherein the second location is at or near the first location specified by the determined first location information.

9. The method of claim 7, wherein the second location is a larger geographic area than the first location specified by the determined first location information.

10. The method of claim 7, wherein the first supplementary information is an advertisement.

11. The method of claim 7, wherein the determination that at least one of the first one or more images associated with the first template appears in the received captured image includes a determination that the at least one of the first one or more images matches less than the entire received captured image.

12. A client computing device to augment reality, comprising:

a processor and memory;

an image capturing component;

a transceiver configured to transmit image information of an image captured by the image capturing component and to receive one or more tags, wherein a tag includes one or more templates and associated information associated with the one or more templates, and wherein a template includes one or more images; and

an imaging component configured to employ a template matching routine to determine whether at least one image associated with a template of the one or more templates appears in the captured image, wherein the template of the one or more templates is associated with a tag of the received one or more tags, and, in response to the determination that at least one image associated with the template appears in the captured image, to render supplementary information corresponding to the template.

13. The client computing device of claim 12,

wherein the associated information includes the supplementary information and additional information that specifies how to render the supplementary information.

14. The client computing device of claim 12, further comprising:

a component configured to determine a first location information relating to the captured image;

wherein the associated information includes a second location information that specifies a second location that is based on a first location specified by the determined first location information.

15. The client computing device of claim 14, wherein the second location is at or near the first location.

16. The client computing device of claim 14, wherein the second location is a larger geographic area than the first location.

17. The client computing device of claim 12, wherein the supplementary information is an advertisement.

18. The client computing device of claim 12, wherein the client computing device further comprises a user-worn visual perception device that includes the image capturing component, an audio component, a housing, and a display arranged to enable user visual perception, wherein the imaging component is configured to render at least part of the supplementary information via the display.

19. The client computing device of claim 12, wherein the client computing device further comprises a user-worn visual perception device that includes the image capturing component, an audio component, a housing, and a display arranged to enable user visual perception, wherein the imaging component is configured to render at least part of the supplementary information via the audio component.

20. The client computing device of claim 12, wherein the determination that at least one image associated with the template appears in the captured image includes a determination that the at least one image matches less than the entire captured image.