IMAGE BASED SEARCH TO IDENTIFY OBJECTS IN DOCUMENTS

• US 20160026858A1
• Filed: 07/28/2014
• Published: 01/28/2016
• Est. Priority Date: 07/28/2014
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Vogel, Matthew
• Application Number:
  US14/445,040
• Publication Number:
  US 20160026858A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

An image based search is provided to identify objects in documents. An image may be processed to identify an object within a portion of the image. The image is embedded within a document. Portion of the image is converted into the object. The object includes a chart, a table, among others. Searchable content associated with the object is detected. The object and the searchable content is provided for export.

Claims

1. A method executed on a computing device to provide an image based search to identify objects in documents, the method comprising:

processing an image to identify an object within a portion of the image;

converting the portion into the object;

detecting searchable content associated with the object; and

providing the object and the searchable content for export.

2. The method of claim 1, further comprising:

retrieving the image from a document.

3. The method of claim 1, further comprising:

providing the searchable content as metadata embedded within the object.

4. The method of claim 1, further comprising:

processing the image through an image identification module that includes augmented optical character recognition (OCR) to identify text based data as the object in a structured format that includes one from a set of;

a tabular format and a table format from the portion.

5. The method of claim 1, further comprising:

identifying a table as the object.

6. The method of claim 5, further comprising:

detecting one or more from a set of;

one or more row titles, one or more column titles, a table title, one or more cell values of the table as the searchable content.

7. The method of claim 1, further comprising:

identifying a chart as the object;

8. The method of claim 7, further comprising:

detecting at least one from a set of;

a chart title, one or more axis labels, one or more dataset labels, and one or more legends as searchable content.

9. The method of claim 7, further comprising:

presenting a prompt to query a type of the chart, wherein the type includes one or more from a set of;

a bar chart, a pie chart, a line chart, an area chart, and a scatter chart; and

receiving an input that includes the type of the chart.

10. The method of claim 9, further comprising:

generating the chart from the portion based on the type of the chart acting as a model for the portion.

11. The method of claim 7, further comprising:

processing the chart to generate a table of values associated with elements of the chart;

adding the table into the chart; and

including the values and the elements in the searchable content.

12. A computing device to provide an image based search to identify objects in documents, the computing device comprising:

a memory;

a processor coupled to the memory and the display, the processor executing an application in conjunction with instructions stored in the memory, wherein the application is configured to;

process an image to identify an object within a portion of the image, wherein the image is retrieved from one from a set of;

a document and a video recording;

convert the portion into the object;

detect searchable content associated with the object; and

provide the object and the searchable content for export.

13. The computing device of claim 12, wherein the application is further configured to:

receive the video recording as one from a set of;

a video file and a video stream.

14. The computing device of claim 13, wherein the application is further configured to:

analyze a frame of the video recording as the image to detect the object from the frame for each frame of the video recording.

15. The computing device of claim 12, wherein the application is further configured to:

process the image with a set of chart types to match the portion to one of the chart types, wherein the chart types include one or more from a set of;

a bar chart, a pie chart, a line chart, an area chart, and a scatter chart; and

convert the portion into a chart as the object based on the chart type acting as a model for the portion.

16. The computing device of claim 12, wherein the application is further configured to:

detect a document type of the document, wherein the document type includes one from a set of;

a text document, a spreadsheet document, and a presentation document; and

process the image with object types associated with the document type.

17. The computing device of claim 17, wherein the application is further configured to:

detect one of the object types matching the portion of the image; and

convert the portion to the object based on the matched object type acting as a model for the portion.

18. A computer-readable memory device with instructions stored thereon to provide an image based search to identify objects in documents, the instructions comprising:

processing an image to identify an object within a portion of the image, wherein the image is retrieved from a document;

converting the portion into the object;

detecting searchable content associated with the object; and

providing the object and the searchable content for export.

19. The computer-readable memory device of claim 18, wherein the instructions further comprise:

identifying a chart as the object; and

detecting at least one from a set of;

a chart title, one or more axis labels, one or more dataset labels, and one or more legends as searchable content.

20. The computer-readable memory device of claim 18, wherein the instructions further comprise:

identifying a table as the object; and

detecting at least one from a set of;

one or more row titles, one or more column titles, a table title, one or more cell values of the table as the searchable content.