INFORMATION SHARING BETWEEN IMAGES

• US 20080209010A1
• Filed: 02/26/2007
• Published: 08/28/2008
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hughes, Richard Lloyd, Zitnick III, Charles Lawrence
• Granted Patent: US 8,788,529 B2
• Time in Patent Office: Days
• US Class Current: 709/219
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A content-based image retrieval technique for sharing image information between images. In the present image information sharing technique digital photographs containing relevant comments and links to web pages are supplied by a set of authors. A user of the image information sharing technique takes a digital picture of something that interests them, such as, for example, a poster, a product, a painting, and uploads it to an image information sharing server. The image information sharing technique then matches the photo to an image in an image information sharing database. The comments and web links associated with the matched photo are subsequently sent to the user.

Claims

1. A computer-implemented process for obtaining information about an object in an image, comprising:

authoring images by appending information about an object in each image to obtain a database of authored images;

acquiring an image containing an object at a client;

uploading the acquired image from the client to a server entity;

matching the object in the image uploaded to the server entity to an object in an authored image in the database using a content-based image retrieval technique; and

extracting appended information about the object that was found in the matched images from the matched authored image.

2. The computer-implemented process of claim 1 further comprising sending the appended information to the client.

3. The computer-implemented process of claim 1 wherein the following actions are repeated to obtain information on one or more additional objects in images:

acquiring an image containing an object at a client;

uploading the acquired image from the client to a server entity;

matching the object in the image uploaded to the server entity to an object in an authored image in the database using a content-based image retrieval technique; and

extracting the appended information about the object that was found in the matched image from the matched authored image.

4. The computer-implemented process of claim 1 further comprising using the appended information to obtain additional information about an object.

5. The computer-implemented process of claim 1 wherein the client is a cellular phone.

6. The computer-implemented process of claim 1 wherein the appended information comprises comments about an object in an authored image and a website address relevant to an object in the authored image.

7. The computer-implemented process of claim 1 wherein the content-based image retrieval technique is an object instance recognition technique.

8. The computer-implemented process of claim 7 wherein the object instance recognition technique comprises:

finding features in the acquired image;

grouping three of the features into a feature triplet;

computing an affine transformation using neighboring feature positions to warp the feature triplet into a geometric shape in canonical space; and

matching each of the features in the feature triplet with features in the feature triplet of the authored images in the database to identify an object.

9. A computer-implemented process for obtaining information about an object in an image, comprising:

acquiring an image containing an object at a client;

computing information used for matching the object in the image at the client;

querying a database using the computed information used for matching;

matching the object in the acquired image to an object in the image in the database using a content-based image retrieval technique with the computed information used for matching;

extracting information about the matched object in the image in the database; and

sending the extracted information to the client that queried the database.

10. The computer-implemented process of claim 9 wherein the content-based image retrieval technique uses triplets of features to match the object in the acquired image to an object in the image in the database.

11. The computer-implemented process of claim 9 wherein the images in the database are appended with information comprising comments about the object.

12. The computer-implemented process of claim 9 wherein the images in the database are appended with a website address for obtaining additional information about the object.

13. The computer-implemented process of claim 9 wherein the images in the database are appended with one of:

location information about the object;

or

key words helpful in searching for other information about the image.

14. A computer-readable medium having computer-executable instructions for performing the process recited in claim 9.

15. A system for querying a database, comprising:

a general purpose computing device;

a computer program comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to,

acquire an image at a client; and

query a database using the acquired image using an object instance recognition technique.

16. The system of claim 15 further comprising modules to:

match the acquired image to an image in the database using a content-based image retrieval technique;

extract information about the matched image from the image in the database; and

send the extracted information to the client.

17. The system of claim 15 wherein the database contains images and wherein the images and information that can be extracted are uploaded in bulk using a bulk loading module.

18. The system of claim 15 wherein the client is a cellular phone that can query the database by pushing a control device on the cellular phone to send an image with which to query the database.

19. The system of claim 16 further comprising a module for the client to compute an image signature and wherein the image signature is used to query the database and match the acquired image to an image in the database.

20. The system of claim 19 wherein a user must subscribe to a service to use the system and wherein only users that subscribe to the service to use the system can upload images to the database.