Mobile image-based information retrieval system

• US 20060240862A1
• Filed: 05/12/2006
• Published: 10/26/2006
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Neven, Hartmut Sr., Neven, Hartmut
• Granted Patent: US 7,751,805 B2
• Time in Patent Office: Days
• US Class Current: 455/550.100
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

An image-based information retrieval system, including a mobile telephone, a remote recognition server, and a remote media server,the mobile telephone having a built-in camera and a communication link for transmitting an image from the built-in camera to the remote recognition server and for receiving mobile media content from the remote media server, the recognition server for matching an image from the mobile telephone with an object representation in a database and forwarding an associated text identifier to the remote server, and the remote media server for forwarding mobile media content to the mobile telephone based on the associated text identifier.

Claims

1. An image-based information retrieval system, comprising a mobile telephone, a remote recognition server, and a remote media server, the mobile telephone having a built-in camera and a communication link for transmitting an image from the built-in camera to the remote recognition server and for receiving mobile media content from the remote media server, the remote recognition server for matching an image from the mobile telephone with an object representation in a database and forwarding an associated text identifier to the remote media server, and the remote media server for forwarding mobile media content to the mobile telephone based on the associated text identifier.

2. An image-based information retrieval system as defined in claim 1, wherein the remote recognition server has an optical character recognition engine for generating a first confidence value based on an image from the mobile telephone, an object recognition engine for generating a second confidence value based on an image from the mobile telephone, a face recognition engine for generating a third confidence value based on an image from the mobile telephone, and an integrator module for receiving the first, second, and third confidence values and generating the associated text identifier.

3. An image-based information retrieval system as defined in claim 2, wherein the object recognition engine comprises a textured object recognition engine.

4. An image-based information retrieval system as defined in claim 2, wherein the object recognition engine comprises a rigid texture object recognition engine.

5. An image-based information retrieval system as defined in claim 2, wherein the object recognition engine comprises an articulate object recognition engine.

6. An image-based information retrieval system as defined in claim 1, wherein the remote recognition server includes means for adding an object representation to the database using the mobile telephone.

7. An image-based information retrieval system, comprising a mobile telephone and a remote server, the mobile telephone having a built-in camera and a communication link for transmitting an image from the built-in camera to the remote server, the remote server having an optical character recognition engine for generating a first confidence value based on an image from the mobile telephone, an object recognition engine for generating a second confidence value based on an image from the mobile telephone, a face recognition engine for generating a third confidence value based on an image from the mobile telephone, and an integrator module for receiving the first, second, and third confidence values and generating a recognition output.

8. An image-based information retrieval system as defined in claim 7, wherein the object recognition engine comprises a textured object recognition engine.

9. An image-based information retrieval system as defined in claim 7, wherein the object recognition engine comprises a rigid texture object recognition engine.

10. An image-based information retrieval system as defined in claim 7, wherein the object recognition engine comprises an articulate object recognition engine.

11. An image-based information retrieval system, comprising a mobile telephone and a remote server, the mobile telephone having a built-in camera, a recognition engine for recognizing an object in an image from the built-in camera, and a communication link for requesting information from the remote server related to a recognized object.

12. An image-based information retrieval system as defined in claim 11, wherein the object is an advertising billboard and the related information is a web page address.

13. An image-based information retrieval system as defined in claim 11, wherein the object is a car and the related information is a car manual.

14. An image-based information retrieval system as defined in claim 11, wherein the object is a product and the related information is a payment confirmation.

15. An image-based information retrieval system as defined in claim 11, wherein the object is a book and the related information is an audio stream.

16. An image-based information retrieval system as defined in claim 11, wherein the object is a bus stop sign and the related information is real-time information on the arrival of the next bus.

17. An image-based information retrieval system, comprising a mobile telephone and a remote server, the mobile telephone having a built-in camera, a recognition engine for recognizing an object feature in an image from the built-in camera, and a communication link for requesting information from the remote server related to a recognized object feature.

18. An image-based information retrieval system as defined in claim 17, wherein the object feature is text and the related information is a translation of the text.

19. An image-based information retrieval system as defined in claim 17, wherein the object feature is text and the related information is a web page address.

20. An image-based information retrieval system as defined in claim 19, wherein the remote server provides the web page address to the mobile telephone in real-time.

21. An image-based information retrieval system as defined in claim 17, wherein the object feature is an advertisement and the related information is a web page address.

22. An image-based information retrieval system as defined in claim 17, wherein the object feature is a picture and the related information is an audio stream.

23. An image-based information retrieval system as defined in claim 17, wherein the object feature is an equipment part and the related information is an operation and maintenance manual for the equipment.