Content and object metadata based search in e-reader environment

• US 9,298,712 B2
• Filed: 12/13/2012
• Issued: 03/29/2016
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Velagapudi, Suresh, Narayanan, Raman, Liu, Ming, Nickolov, Radoslav, Zeng, Wei, Gil, Aleksandr
• Primary Examiner(s): TO, BAOQUOC N
• Application Number:
  US13/713,465
• Publication Number:
  US 20140172825A1
• Time in Patent Office: 1,202 Days
• Field of Search: 707/705, 707/706
• US Class Current: 1/1
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/9032 Query formulation

Abstract

Metadata based enhanced search capabilities are provided in conjunction with e-reader experience. Metadata associated with content and objects within displayed content is created based on default and/or custom rules. The metadata such as attributes, tags, etc. is indexed and used to augment search queries based on selected portions of the displayed content. Additional search augmentation is performed based on user-defined metadata or processing of objects such as image recognition of image objects and similar processes.

Claims

1. A method executed on a computing device to provide a metadata based search in an e-reader environment, the method comprising:

determining search terms based on a selected portion of displayed content;

generating a query for a search based on the selected portion;

determining the metadata associated with one or more objects associated with the search terms, wherein the one or more objects include embedded images, embedded graphical objects, and one or more interactive objects, interactive puzzle objects, embedded video objects, and embedded audio objects;

performing optical character recognition on each of the embedded images and the embedded graphical objects to retrieve textual content and key terms from each of the embedded images and the embedded graphical objects;

indexing the metadata, wherein the metadata includes the textual content and the key terms;

augmenting the query based on the metadata, wherein the metadata is associated with one or more attributes of each of the one or more objects;

categorizing a summary of query results based on the metadata; and

presenting the summary of the query results in a results pane on a user interface.

2. The method of claim 1, further comprising:

determining the metadata associated with the one or more objects within a predefined vicinity of the selected portion.

3. The method of claim 1, wherein the one or more attributes include one or more of a dimension of the one or more objects, a type of the one or more objects, and a title of the one or more objects.

4. The method of claim 1, further comprising:

determining the metadata from one or more tags associated with each of the one or more objects.

5. The method of claim 4, wherein the metadata includes one or more of a type of each of the one or more objects a type of each of the embedded audio objects, a title of each of the embedded audio objects, a title of each of the embedded video objects, a duration of each of the embedded audio objects, and a duration of each of the embedded video objects.

6. The method of claim 1, further comprising:

processing the one or more objects to retrieve the metadata.

7. The method of claim 1, further comprising:

retrieving the key terms from the embedded audio objects and the embedded video objects.

8. The method of claim 1, further comprising:

generating the query in response to detecting activation of a search control following the selection of the portion of the displayed content.

9. A computing device to provide a metadata based search in an e-reader environment, the computing device comprising:

a memory configured to store instructions; and

a processor coupled to the memory, the processor executing an e-reader application in conjunction with the instructions stored in the memory, wherein the e-reader application is configured to;

determine search terms based on a selected portion of displayed content;

generate a query for a search based on the selected portion;

determine the metadata associated with one or more objects, wherein the one or more objects include embedded images, embedded graphical objects, and one or more interactive objects, interactive puzzle objects, embedded video objects, and embedded audio objects;

perform optical character recognition on each of the embedded images and the embedded graphical objects to retrieve textual content and key terms from each of the embedded images and the embedded graphical objects;

index the metadata, wherein the metadata includes the textual content and the key terms;

augment the query based on the metadata, wherein the metadata is associated with one or more attributes of each of the one or more objects and includes at least one from a set of dimensions, types, titles, and tags of the one or more objects;

categorize a summary of query results based on the metadata; and

present the summary of the query results in a results pane on a user interface.

10. The computing device of claim 9, wherein the metadata is created for the one or more objects based on one or more of a default rule and a custom rule.

11. The computing device of claim 9, wherein the metadata is user-defined metadata provided through a search user interface managed by the e-reader application.

12. The computing device of claim 9, wherein the e-reader application is further configured to:

enable interaction for a user through one or more of a touch input, a gesture input, a mouse input, a keyboard input, a gyroscopic input, and an eye-tracking input.

13. The computing device of claim 9, wherein the e-reader application is one of a locally executed application and a client application accessing an e-reader service.

14. A method executed on a computing device to provide a metadata based search in an e-reader environment, the method comprising:

determining search terms based on a selected portion of displayed content;

generating a query for a search based on the selected portion;

determining the metadata associated with one or more objects within a predefined vicinity of the selected portion by processing the one or more objects to retrieve metadata,

wherein the one or more objects include embedded images, embedded graphical objects, and one or more interactive objects, interactive puzzle objects, embedded video objects, and embedded audio objects, and

wherein the interactive objects are associated with the search terms;

performing optical character recognition on each of the embedded images and the embedded graphical objects to retrieve textual content and key terms from each of the embedded images and the embedded graphical objects;

indexing the metadata, wherein the metadata includes the textual content and the key terms;

augmenting the query based on the metadata, wherein the metadata is associated with one or more attributes of each of the one or more objects and includes at least one from a set of dimensions, types, titles, and tags of the one or more objects;

categorizing a summary of query results based on the metadata; and

presenting the summary of the query results in a results pane on a user interface.

15. The method of claim 14, further comprising:

generating the query in response to detecting one of the selection of the portion of the displayed content and activation of a search control following the selection of the portion of the displayed content.

16. The method of claim 14, further comprising:

categorizing search results of the query based on the metadata.

17. The method of claim 14, further comprising:

enabling execution of the search in one or more of web resources, a user library associated with an e-reader application, a collection of user documents, and a collection of user notes generated through the e-reader application.