Mobile image-based information retrieval system

• US 7,962,128 B2
• Filed: 05/21/2010
• Issued: 06/14/2011
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Neven, Hartmut Sr., Neven, Hartmut
• Primary Examiner(s): Tran; Tuan A
• Application Number:
  US12/785,341
• Publication Number:
  US 20100260373A1
• Time in Patent Office: 389 Days
• Field of Search: 4554141-4144, 455418-420, 455/458, 455/550.1, 4555561-5562, 455 301- 306, 382/182, 382/229, 382/181, 382/118, 707/104.1, 707 3- 10, 707200-204, 348 1401- 1402, 345418-419
• US Class Current: 455/414.3
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

An image-based information retrieval system, including a mobile telephone, a remote recognition server, and a remote media server, the mobile telephone having a built-in camera and a communication link for transmitting an image from the built-in camera to the remote recognition server and for receiving mobile media content from the remote media server, the remote recognition server for matching an image from the mobile telephone with an object representation in a database and forwarding an associated text identifier to the remote media server, and the remote media server for forwarding mobile media content to the mobile telephone based on the associated text identifier.

Claims

1. A method of recognizing an object in an image, the method comprising:

storing objects extracted from images, wherein the objects are associated with a respective time of day at which their corresponding images were created and are stored sorted according to the time of day,

receiving an image having a time of day at which the image was created; and

recognizing an object in the image by comparing the object to the stored objects in order of the proximity of their times to the time at which the image was created.

2. The method of claim 1, wherein the object is an advertising billboard, the method further comprising requesting a web page address related to the advertising billboard from a remote server.

3. The method of claim 1, wherein the object is a car, the method further comprising requesting a car manual related to the car from a remote server.

4. The method of claim 1, wherein the object is a product, the method further comprising requesting a payment confirmation related to the product from a remote server.

5. The method of claim 1, wherein the object is a book, the method further comprising requesting an audio stream related to the book from a remote server.

6. The method of claim 1, wherein the object is a bus stop sign, the method further comprising requesting real-time information on the arrival of a next bus from a remote server.

7. An image-based information retrieval system, comprising:

an image database storing objects extracted from images, wherein the objects are associated in the database with a respective time of day at which their corresponding images were created and are stored sorted according to the time of day; and

a recognition engine for recognizing an object in an image, the recognition engine performing actions comprising;

receiving an image having a time of day at which the image was created; and

recognizing an object in the image by comparing the object to the stored objects in order of the proximity of their times to the time at which the image was created.

8. The system of claim 7, wherein the object is an advertising billboard, the system further comprising an information retrieval module that requests a web page address related to the advertising billboard from a remote server.

9. The system of claim 7, wherein the object is a car, the system further comprising an information retrieval module that requests a car manual related to the car from a remote server.

10. The system of claim 7, wherein the object is a product, the system further comprising an information retrieval module that requests a payment confirmation related to the product from a remote server.

11. The system of claim 7, wherein the object is a book, the system further comprising an information retrieval module that requests an audio stream related to the book from a remote server.

12. The system of claim 7, wherein the object is a bus stop sign, the system further comprising an information retrieval module that requests requesting real-time information on the arrival of a next bus from a remote server.

13. A computer-readable storage medium having executable computer program instructions embodied therein for recognizing an object in an image, actions of the computer program instructions comprising:

storing objects extracted from images, wherein the objects are associated with a respective time of day at which their corresponding images were created and are stored sorted according to the time of day;

receiving an image having a time of day at which the image was created; and

recognizing an object in the image by comparing the object to the stored objects in order of the proximity of their times to the time at which the image was created.

14. The computer-readable storage medium of claim 13, wherein the object is an advertising billboard, the method further comprising requesting a web page address related to the advertising billboard from a remote server.

15. The computer-readable storage medium of claim 13, wherein the object is a car, the method further comprising requesting a car manual related to the car from a remote server.

16. The computer-readable storage medium of claim 13, wherein the object is a product, the method further comprising requesting a payment confirmation related to the product from a remote server.

17. The computer-readable storage medium of claim 13, wherein the object is a book, the method further comprising requesting an audio stream related to the book from a remote server.

18. The computer-readable storage medium of claim 13, wherein the object is a bus stop sign, the method further comprising requesting real-time information on the arrival of a next bus from a remote server.