METHOD AND APPARATUS FOR CONDUCTING A SEARCH BASED ON AVAILABLE DATA MODES

• US 20130073583A1
• Filed: 09/20/2011
• Published: 03/21/2013
• Est. Priority Date: 09/20/2011
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Nokia Corporation
• Inventors: Licata, Aaron, Bamidele, Adetokunbo, Fulks, Mark Travis
• Granted Patent: US 9,245,051 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 30/19167 Active pattern learning
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

An approach is provided for conducting a search based on an extraction of a search term from available sensor data. The approach involves determining sensor data associated with at least one device, the sensor data determined from among a plurality of available data modes. The approach also involved processing and/or facilitating a processing of the sensor data to cause, at least in part, an extraction of one or more search terms for at least one query. The approach further involves determining one or more results of the at least one query based, at least in part, on context information associated with the at least one device, user profile information associated with the at least one device, or a combination thereof.

Claims

1. A method comprising facilitating a processing of and/or processing (1) data and/or (2) information and/or (3) at least one signal, the (1) data and/or (2) information and/or (3) at least one signal based, at least in part, on the following:

sensor data associated with at least one device, the sensor data determined from among a plurality of available data modes;

a processing of the sensor data to cause, at least in part, an extraction of one or more search terms for at least one query; and

at least one determination of one or more results of the at least one query based, at least in part, on context information associated with the at least one device, user profile information associated with the at least one device, or a combination thereof.

2. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a creation of one or more bounding boxes around one or more estimated text portions of the sensor data;

an optical character recognition of the sensor data within the one or more bounding boxes;

a processing of the one or more bounding boxes to cause, at least in part, an expansion of the one or more bounding boxes;

the expansion of the one or more bounding boxes to encompass a line of text; and

one or more selected bounding boxes,

wherein the extraction is based, at least in part, on the selected one or more bounding boxes.

3. A method of claim 2, wherein the one or more bounding boxes are expanded by geometrically merging determined bounding boxes per line of text.

4. A method of claim 2, wherein the one or more bounding boxes are assigned an identification number and ranked in order from a top of an image sensed by way of the optical sensor.

5. A method of claim 2, wherein the optical sensor is associated with an augmented reality display.

6. A method of claim 5, wherein the sensor data is detected and processed in real time with respect to what is shown by the augmented reality display.

7. A method of claim 1, wherein the one or more results of the at least one query are a translation of the sensor data based, at least in part, on the user profile information associated with the at least one device.

8. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

information comprising one or more preferences and/or behavioral tendencies of the user by way of mining the user profile to cause, at least in part, the one or more results of the at least one query to be modified based on the information.

9. A method of claim 1, wherein the sensor data is an audio input, and the audio input is converted to textual information.

10. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of a processing engine of one or more available processing engines for processing the sensor data based, at least in part, on a confidence value that the sensor data is of at least one data mode type that corresponds to a respective processing engine of the one or more available processing engines.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

determine sensor data associated with at least one device, the sensor data determined from among a plurality of available data modes;

process and/or facilitate a processing of the sensor data to cause, at least in part, an extraction of one or more search terms for at least one query; and

determine one or more results of the at least one query based, at least in part, on context information associated with the at least one device, user profile information associated with the at least one device, or a combination thereof.

12. An apparatus of claim 11, wherein the sensor data is detected by way of an optical sensor, and the apparatus is further caused to:

cause, at least in part, a creation of one or more bounding boxes around one or more estimated text portions of the sensor data;

cause, at least in part, an optical character recognition of the sensor data within the one or more bounding boxes;

process and/or facilitate a processing of the one or more bounding boxes to cause, at least in part, an expansion of the one or more bounding boxes;

cause, at least in part, the expansion of the one or more bounding boxes to encompass a line of text; and

determine one or more selected bounding boxes,

wherein the extraction is based, at least in part, on the selected one or more bounding boxes.

13. An apparatus of claim 12, wherein the one or more bounding boxes are expanded by geometrically merging determined bounding boxes per line of text.

14. An apparatus of claim 12, wherein the one or more bounding boxes are assigned an identification number and ranked in order from a top of an image sensed by way of the optical sensor.

15. An apparatus of claim 12, wherein the optical sensor is associated with an augmented reality display.

16. An apparatus of claim 15, wherein the sensor data is detected and processed in real time with respect to what is shown by the augmented reality display.

17. An apparatus of claim 11, wherein the one or more results of the at least one query are a translation of the sensor data based, at least in part, on the user profile information associated with the at least one device.

18. An apparatus of claim 11, wherein the apparatus is further caused to:

determine information comprising one or more preferences and/or behavioral tendencies of the user by way of mining the user profile to cause, at least in part, the one or more results of the at least one query to be modified based on the information.

19. An apparatus of claim 11, wherein the sensor data is an audio input, and the audio input is converted to textual information.

20. An apparatus of claim 11, wherein the apparatus is further caused to:

determine a processing engine of one or more available processing engines for processing the sensor data based, at least in part, on a confidence value that the sensor data is of at least one data mode type that corresponds to a respective processing engine of the one or more available processing engines.

21-48. -48. (canceled)