Data access based on content of image recorded by a mobile device

• US 8,406,531 B2
• Filed: 05/15/2008
• Issued: 03/26/2013
• Est. Priority Date: 05/15/2008
• Status: Active Grant

• Current Assignee: Yahoo Assets LLC
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Ramanujapuram, Arun, Slaney, Malcolm
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): PERLMAN, DAVID S
• Application Number:
  US12/121,259
• Publication Number:
  US 20090285492A1
• Time in Patent Office: 1,776 Days
• Field of Search: 382/167, 382/187, 382/305, 455/550.1
• US Class Current: 382/209
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/535 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text

Abstract

Embodiments of the invention are directed to using image data and contextual data to determine information about a scene, based on one or more previously obtained images. Contextual data, such location of image capture, can be used to determine previously obtained images related to the contextual data and other location-related information, such as billboard locations. With even low resolution devices, such as cell phone, image attributes, such as a histogram or optically recognized characters, can be compared between the previously obtained images and the newly captured image. Attributes matching within a predefined threshold indicate matching images. Information on the content of matching previously obtained images can be provided back to a user who captured the new image. User profile data can refine the content information. The content information can also be used as search terms for additional searching or other processing.

Claims

1. A method for accessing information, comprising:

receiving image data representing an image captured by a mobile device;

receiving location data indicating a location at which the image was captured by the mobile device;

determining image attribute information from the image data;

determining whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location at which the image was captured by the mobile device;

accessing image content information based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, wherein the content information identifies content of the image that is associated with the previously obtained image data and associated with the location, and the previously obtained attribute and image data further includes a plurality of profile data, contextual data, and a plurality of captured image data from a plurality of different devices to enable a community-based image recognition;

determining search terms that are relevant to the captured image based on the image content information; and

providing the determined search terms to the mobile device for use by the mobile device in submitting a subsequent search using the determined search terms.

2. The method of claim 1, wherein the image attribute information comprises a histogram of the image, and the previously obtained attribute information comprises a previously determined histogram of the prior image.

3. The method of claim 1, wherein the image attribute information comprises a character determined from the image by optical character recognition, and the previously obtained attribute information comprises a previously determined character of the prior image.

4. The method of claim 1, further comprising accessing location-related information based on the image content information if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, wherein the location-related information is associated with the previously obtained image data and associated with the location.

5. The method of claim 1, wherein the image content information comprises an identifier of merchant information.

6. The method of claim 1, further comprising:

receiving a mobile device identifier of the mobile device;

accessing profile information associated with the mobile device identifier; and

selecting a portion of the image content information based on the profile information.

7. The method of claim 6, wherein the profile information includes user behavior information associated with a user of the mobile device.

8. The method of claim 1, further comprising:

determining a priority level of each of a plurality of prior images captured within the predefined distance; and

selecting the previously obtained attribute information based on a determined priority level of the prior image represented by the previously obtained image data.

9. The method of claim 1, further comprising returning the image content information to the mobile device.

10. A non-transitory machine readable medium holding instructions that cause an electronic device to perform a plurality of operations, including the steps of claim 1.

11. A server for accessing information, comprising:

a processor;

a communication interface in communication with the processor and with a network; and

a memory in communication with the processor and storing data and instructions that cause the processor to a plurality of operations, including;

receiving image data representing an image captured by a mobile device in communication with the network;

receiving location data indicating a location at which the image was captured by the mobile device;

determining image attribute information from the image data;

determining whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location at which the image was captured by the mobile device;

accessing image content information from a data store based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, wherein the content information identifies content of the image that is associated with the previously obtained image data and associated with the location, and the previously obtained attribute and image data further includes a plurality of profile data, contextual data, and a plurality of captured image data from a plurality of different devices to enable a community-based image recognition;

determining search terms that are relevant to the captured image based on the image content information; and

providing the determined search terms to the mobile device for use by the mobile device in submitting a subsequent search using the determined search terms.

12. The server of claim 11, wherein the image attribute information comprises at least one of the following:

a histogram of the image, and wherein the previously obtained attribute information comprises a previously determined histogram of the prior image; and

a character determined from the image by optical character recognition, and wherein the previously obtained attribute information comprises a previously determined character of the prior image.

13. The server of claim 11, wherein the instructions further cause the processor to perform the operations of:

performing an other data search based on the search terms and providing the results of the other search to the mobile device.

14. The server of claim 11, wherein the instructions further cause the processor to perform the operations of:

receiving a mobile device identifier of the mobile device;

accessing profile information associated with the mobile device identifier, wherein the profile information includes user behavior information associated with a user of the mobile device; and

selecting a portion of the image content information based on the profile information.

15. A method for accessing information, comprising:

capturing an image with an imaging device for a mobile device;

determining a location of the imaging device;

creating image data from the image;

determining image attribute information from the image data;

transmitting the image attribute information through a network interface to a data server, wherein the image attribute information is transmitted with a request to determine whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location at which the image was captured by the mobile device;

receiving image content information based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, wherein the content information identifies content of the image that is associated with the previously obtained image data and associated with the location, and the previously obtained attribute and image data further includes a plurality of profile data, contextual data, and a plurality of captured image data from a plurality of different devices to enable a community-based image recognition;

determining search terms that are relevant to the captured image based on the image content information; and

providing the determined search terms to the mobile device for use by the mobile device in submitting a subsequent search using the determined search terms.

16. The method of claim 15, further comprising transmitting a search request to a search engine, wherein the search request includes at least a portion of the image content information.

17. A network device for providing access to information, comprising:

a processor;

a communication interface in communication with the processor and with a network;

an image processing component in communication with the processor;

a location detector in communication with the processor;

an output component in communication with the processor; and

a memory in communication with the processor and storing data and instructions that cause the processor to a plurality of operations, including;

causing the image processing component to capture an image for a mobile device;

causing the location detector to determine a location at which the image was captured;

creating image data from the image;

determining image attribute information from the image data;

transmitting the image attribute information through the communication interface to a data server, wherein the image attribute information is transmitted with a request to determine whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location at which the image was captured by the mobile device;

receiving image content information based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, wherein the content information identifies content of the image that is associated with the previously obtained image data and associated with the location, and the previously obtained attribute and image data further includes a plurality of profile data, contextual data, and a plurality of captured image data from a plurality of different devices to enable a community-based image recognition;

determining search terms that are relevant to the captured image based on the image content information; and

providing the determined search terms to the mobile device for use by the mobile device in submitting a subsequent search using the determined search terms.

18. The network device of claim 17, wherein the network device comprises a mobile device.