IMAGE MATCHING TO AUGMENT REALITY

• US 20110102605A1
• Filed: 11/02/2009
• Published: 05/05/2011
• Est. Priority Date: 11/02/2009
• Status: Active Grant

• Current Assignee: Huawei Device Company Limited (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC), Glitter Technology LLP
• Inventors: Hannaford, Blake
• Granted Patent: US 9,001,252 B2
• Time in Patent Office: Days
• US Class Current: 348/207.1
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04L 67/289 Intermediate processing functionally located close to the data consumer application, e.g. in same machine, in same home or in same sub-network
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/561 Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Technology for augmenting reality using image matching is described. The technology can receive a captured image, determine location information relating to the received image, and transmit the location information to a server computing device. The technology may then receive one or more tags from the server computing device, wherein each tag can include one or more patches and supplementary information corresponding to at least one of the patches and each patch can be associated with the location information. The technology can then determine using an image matching process whether at least one of the received one or more patches appears in the received image, and if a patch appears in the received image, render the supplementary information corresponding to the patch.

Claims

1. A method performed by a computing system for augmenting reality, comprising:

receiving a captured image;

determining location information relating to the received image;

transmitting the location information to a server computing device;

receiving one or more tags from the server computing device, each tag including one or more patches and supplementary information corresponding to at least one of the patches, each patch associated with the location information;

determining using an image matching process whether at least one of the received one or more patches appears in the received image; and

if a patch appears in the received image, rendering the supplementary information corresponding to the patch.

2. The method of claim 1, wherein the image is received from a digital camera associated with the computing system.

3. The method of claim 1, wherein the location information is determined based on information received from a GPS signal.

4. The method of claim 1, wherein the location information transforms a physical condition sensed by an accelerometer into an electrical signal.

5. The method of claim 1, wherein the supplementary information is associated with additional information that specifies how to render the supplementary information.

6. The method of claim 1, wherein the image matching process is at least partially performed by a hardware-based video compression processor.

7. The method of claim 1, wherein the rendering includes superimposing an image specified by the supplementary information.

8. The method of claim 1, wherein the rendering includes playing an audible sound specified by the supplementary information.

9. A computer-readable medium storing computer-executable instructions that, when executed, perform a method for augmenting reality, the method comprising:

receiving a captured digital image that is digitized by an image capturing device;

determining location information for the image capturing device at the time the image is digitized;

transmitting the location information;

receiving one or more tags;

determining using an image matching process whether at least one of the tags includes a patch that appears in the captured digital image; and

if a patch appears in the digital image, rendering supplementary information corresponding to the patch.

10. The computer-readable medium of claim 9 further comprising a data structure, the data structure storing at least the digital image, the patch, and the supplementary information, wherein the supplementary information specifies information to render if the patch appears in the digital image.

11. The computer-readable medium of claim 10, wherein the supplementary information is an image to superimpose on the digital image.

12. The computer-readable medium of claim 10, wherein the supplementary information is an audible sound.

13. The computer-readable medium of claim 9, wherein at least one of the one or more tags comprises the patch and the supplementary information.

14. The computer-readable medium of claim 13, wherein the supplementary information comprises additional information specifying rendering information.

15. The computer-readable medium of claim 9, wherein the location information is transmitted using a wireless data communications protocol supported by a mobile telephone service.

16. A system for augmenting reality, comprising:

a processor and memory;

an image capturing component;

a hardware component configured to detect position information when an image is captured by the image capturing component;

a transceiver configured to transmit the position information and to receive a patch; and

an imaging component configured to determine whether the received patch appears within the captured image and to render supplementary information if the received patch appears within the captured image.

17. The system of claim 16, wherein the image capturing component is a digital camera associated with a client computing device with which the processor and memory are associated.

18. The system of claim 16, wherein the hardware component is at least one of a GPS receiver, an accelerometer, or a compass.

19. The system of claim 16, wherein the imaging component functions in association with a hardware component that implements video compression specification.

20. The system of claim 16, wherein the supplementary information is rendered by superimposition of an image on the captured image.