SCALABLE VISUAL SEARCH SYSTEM SIMPLIFYING ACCESS TO NETWORK AND DEVICE FUNCTIONALITY

• US 20140365472A1
• Filed: 08/21/2014
• Published: 12/11/2014
• Est. Priority Date: 11/04/2005
• Status: Active Grant

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: SCHLOTER, Philipp
• Granted Patent: US 9,600,584 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/16 Communication-related supplementary services, e.g. call-transfer or call-hold

Abstract

In one embodiment, an indication of information desired by a user is received, and a list of candidates for the desired information is provided for presentation on a mobile device of the user.

Claims

1. A method comprising:

receiving in real time at a mobile device an indication of information desired by a user once the mobile device is pointed to one or more objects in a real world by the user, wherein the indication of the desired information comprises live visual media content presenting the one or more objects therein, meta-information providing one or more properties of the media content, and one or more characteristics associated with at least one of the mobile device and the user;

causing, at least in part, providing candidates for the desired information for presentation on the mobile device based, at least in part, on at least one of the live visual media content and the meta-information;

determining a set of properties for the candidates; and

adjusting, based on the determined set of properties for the candidates, an access of the candidates for the information desired by the user,

wherein the candidates concurrently presented with the live visual media on one screen to the user for selection in real time without requiring the user to make any clicks from the time the user starts pointing the mobile device to the objects.

2. The method of claim 1, further comprising:

searching a database for data matching the indication of the desired information for the candidates; and

constraining a scope of a search based on at least one of the media content and the meta-information.

3. The method of claim 2, further comprising:

identifying the one or more objects based upon one or more recognition parameters,

wherein the search in the database for the candidates is further based upon one or more identified objects,

each candidate is a link that maps the one or more objects, the one or more characteristics, or a combination thereof to a desired information item, and

the desired information item is a web site, a phone number, a message template, an application, an application state, a web page entry, a phone functionality, a keyword, a computer functionality, a document, a television program, a radio program, a music recording, or a combination thereof.

4. The method of claim 2, further comprising:

creating a plurality of links automatically based on the live visual media content, if no matching data is found in the database;

associating each of the plurality of links with the live visual media content, the meta-information, or a combination thereof; and

storing the plurality of links in the database.

5. The method of claim 2, further comprising:

creating an intermediate representation of the media content, the intermediate representation being used to search the database for data matching the indication of the desired information,

wherein the scope of the search is constrained by one or more other objects in the real world that are geographically near-by the one or more objects.

6. The method of claim 1, wherein:

the candidates are overlaid on top of the live visual media content within a viewfinder of the mobile device; and

the candidates are presented to the user without requiring the user to make any clicks to capture an image of the objects, from the time the user starts pointing the mobile device to the objects.

7. The method of claim 6, further comprising:

in response to a subsequent search query of the user, searching within the stored association and the at least one of the media content and the meta-information to assemble the candidates as search results.

8. The method of claim 6, wherein:

at least one candidate has a title that includes a preview of the desired information; and

the method further comprises causing display of the title including the preview of the desired information when presenting the candidates to the user,

wherein the displaying provides user access to the preview of the desired information when the user points the mobile device to one or more other objects.

9. The method of claim 1, further comprising:

determining that the user has not selected a candidate;

presenting a directory of information items to the user;

allowing the user to select an information item from the directory;

creating a link mapping the selected information item to the received indication of the desired information; and

storing the link in a database.

10. The method of claim 1, further comprising selecting the candidates by searching a link database to identify one or more candidates associated with the indication of the desired information and generating the one or more candidates based upon one or more predefined rules.

11. The method of claim 1, further comprising:

causing, at least in part, reception of an indication of a selection of the user in response to the providing of the candidates;

creating an association between the selection of the user and the indication of information desired by the user; and

causing, at least in part, storage of the association and at least one of the media content and the meta-information,

wherein the media content is produced by the mobile device and comprises one or more digital photographs, one or more videos, one or more audio recordings, or a combination thereof, and

the meta-information comprises one or more characteristics of the media content, one or more environmental characteristics associated with the mobile device, one or more geographic characteristics associated with the mobile device, one or more personal characteristics associated with the user, one or more online behavior characteristics of the user, or a combination thereof.

12. The method of claim 1, further comprising:

causing, at least in part, starting of a timer when the mobile device is pointed to one or more objects in a real world by the user; and

causing, at least in part, providing the candidates for the desired information based on regular or irregular intervals based on the timer.

13. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive in real time at a mobile device an indication of information desired by a user once the mobile device is pointed to one or more objects in a real world by the user, wherein the indication of the desired information comprises live visual media content presenting the one or more objects therein, meta-information providing one or more properties of the media content, and one or more characteristics associated with at least one of the mobile device and the user;

cause, at least in part, providing candidates for the desired information for presentation on the mobile device based, at least in part, on at least one of the live visual media content and the meta-information;

determine a set of properties for the candidates; and

adjust, based on the determined set of properties for the candidates, an access of the candidates for the information desired by the user,

wherein the candidates concurrently presented with the live visual media on one screen to the user for selection in real time without requiring the user to make any clicks from the time the user starts pointing the mobile device to the objects.

14. The apparatus of claim 13, wherein the apparatus is further caused to select the candidates by searching a link database to identify one or more candidates associated with the indication of the desired information and generate the one or more candidates based upon one or more predefined rules.

15. The apparatus of claim 13, wherein the apparatus is further caused to:

receive an indication of a selection of the user in response to the providing of the candidates;

create an association between the selection of the user and the indication of information desired by the user; and

store the association and at least one of the media content and the meta-information,

wherein the media content is produced by the mobile device and comprises one or more digital photographs, one or more videos, one or more audio recordings, or a combination thereof, and

the meta-information comprises one or more characteristics of the media content, one or more environmental characteristics associated with the mobile device, one or more geographic characteristics associated with the mobile device, one or more personal characteristics associated with the user, one or more online behavior characteristics of the user, or a combination thereof.

16. The apparatus of claim 13, wherein the apparatus is further caused to:

cause, at least in part, starting of a timer when the mobile device is pointed to one or more objects in a real world by the user; and

cause, at least in part, providing the candidates for the desired information based on regular or irregular intervals based on the timer.

17. A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause the apparatus to at least perform the following steps:

receiving in real time at a mobile device an indication of information desired by a user once the mobile device is pointed to one or more objects in a real world by the user, wherein the indication of the desired information comprises live visual media content presenting the one or more objects therein, meta-information providing one or more properties of the media content, and one or more characteristics associated with at least one of the mobile device and the user;

causing, at least in part, providing candidates for the desired information for presentation on the mobile device based, at least in part, on at least one of the live visual media content and the meta-information;

determining a set of properties for the candidates; and

adjusting, based on the determined set of properties for the candidates, an access of the candidates for the information desired by the user,

wherein the candidates concurrently presented with the live visual media on one screen to the user for selection in real time without requiring the user to make any clicks from the time the user starts pointing the mobile device to the objects.

18. The non-transitory computer-readable storage medium of claim 17, wherein the apparatus is further cause to perform selecting the candidates by searching a link database to identify one or more candidates associated with the indication of the desired information and generate the one or more candidates based upon one or more predefined rules.

19. The non-transitory computer-readable storage medium of claim 17, wherein the apparatus is further cause to perform:

receiving an indication of a selection of the user in response to the providing of the candidates;

creating an association between the selection of the user and the indication of information desired by the user; and

storing the association and at least one of the media content and the meta-information;

wherein the media content is produced by the mobile device and comprises one or more digital photographs, one or more videos, one or more audio recordings, or a combination thereof, and

the meta-information comprises one or more characteristics of the media content, one or more environmental characteristics associated with the mobile device, one or more geographic characteristics associated with the mobile device, one or more personal characteristics associated with the user, one or more online behavior characteristics of the user, or a combination thereof.

20. The non-transitory computer-readable storage medium of claim 17, wherein the apparatus is further cause to perform:

causing, at least in part, starting of a timer when the mobile device is pointed to one or more objects in a real world by the user; and

causing, at least in part, providing the candidates for the desired information based on regular or irregular intervals based on the timer.