Identification, selection, and display of a region of interest in a document

• US 9,460,063 B2
• Filed: 06/07/2009
• Issued: 10/04/2016
• Est. Priority Date: 01/02/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Mansfield, Philip Andrew, Levy, Michael Robert, Clegg, Derek B.
• Primary Examiner(s): Patel, Manglesh M
• Application Number:
  US12/479,849
• Publication Number:
  US 20100174979A1
• Time in Patent Office: 2,676 Days
• Field of Search: 715/234, 715/276, 715/200, 715/243, 715/788, 715/800
• US Class Current: 1/1
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/126 Character encoding
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/163 Handling of whitespace
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

Some embodiments provide a method for displaying a structured document that includes a hierarchy of structural elements. The hierarchy is constructed by analyzing an unstructured document. The method displays the structured document on a device. The method receives a selection of a position of interest in the document. Based on the position of interest, the method identifies a structural element within the hierarchy as a region of interest. The method modifies the display of the document to highlight the identified region of interest.

Claims

1. A computer readable medium storing a computer program which when executed by at least one processor displays a document in a display area of a display device, the computer program comprising sets of instructions for:

analyzing an unstructured document that comprises a plurality of unassociated primitive elements in order to generate a plurality of structural elements arranged in a hierarchical tree structure;

displaying the document in the display area using the plurality of structural elements arranged in the hierarchical tree structure;

receiving a selection of a location on the device as a position of interest, the location corresponding to one of the primitive elements;

identifying a structural element, at a lowest level of the hierarchical tree structure, the bounds of which contain the position of interest;

traversing up to structural elements at higher levels of the hierarchical tree structure until one of a paragraph and a graphical object is identified as a region of interest, the bounds of which contain the position of interest; and

maximizing the display of the identified region of interest within the display area while displaying the entire region of interest.

2. The computer readable medium of claim 1, wherein the device is a small-screen device.

3. The computer readable medium of claim 2, wherein the device is a handheld device.

4. The computer readable medium of claim 1, wherein the set of instructions for analyzing the unstructured document comprises sets of instructions for:

creating associations between the primitive elements to define the structural elements; and

arranging the structural elements in the hierarchical tree structure.

5. The computer readable medium of claim 1, wherein the set of instructions for traversing up to structural elements at higher levels of the hierarchical tree structure comprises a set of instructions for identifying structural elements at higher levels of the hierarchical tree structure that include the structural element at previous levels of the hierarchical tree structure.

6. The computer readable medium of claim 1, wherein the first structural element is a set of glyphs associated as a word.

7. The computer readable medium of claim 1, wherein the position of interest is within a graphic element and the region of interest is the graphic element.

8. The computer readable medium of claim 1, wherein the set of instructions for receiving a selection of a position of interest comprises a set of instructions for receiving a double tap input at a particular position on a touchscreen device.

9. The computer readable medium of claim 1, wherein the maximization of the display of the identified region of interest results in centering the region of interest in the display.

10. The computer readable medium of claim 1, wherein the set of instructions for maximizing the display of the identified region of interest comprises of instructions for applying at least one of a scaling transformation, a rotation transformation, and a translation transformation to the display of the document.

11. The computer readable medium of claim 10, wherein the set of instructions for applying at least one of a scaling transformation, a rotation transformation, and a translation transformation to the display of the document comprises sets of instructions for:

defining a rectangle of interest based on the identified region of interest; and

zooming the display to maximize the size of the rectangle of interest, wherein the rectangle of interest is zoomed such that a length of the rectangle in a first direction is equal to a length of a display on the device in the first direction; and

centering the rectangle of interest in the display in a second direction.

12. The computer readable medium of claim 11, wherein the first direction is horizontal and the second direction is vertical.

13. The computer readable medium of claim 11, wherein the first direction is vertical and the second direction is horizontal.

14. The computer readable medium of claim 1, wherein the computer program further comprises sets of instructions for:

receiving an input to move from the identified region of interest; and

modifying the display of the document in order to focus the display on a second structural element that is structurally related region of interest.

15. The computer readable medium of claim 14, wherein the second structural element is a structural element that is previous to the identified region of interest in a reading order.

16. The computer readable medium of claim 14, wherein the second structural element is a structural element that is after the identified region of interest in a reading order.

17. The computer readable medium of claim 14, wherein the second structural element is a structural element on a different page in the document than the identified region of interest, wherein the hierarchical tree structure includes a link between the identified region of interest and the second structural element.

18. The computer readable medium of claim 1, wherein the unstructured document is a vector graphics document.

19. The computer readable medium of claim 1, wherein the unstructured document is a portable document format (PDF) document.

20. A computer readable medium storing a computer program for a handheld display device, the computer program for execution by at least one processor, the computer program comprising sets of instructions for:

analyzing an unstructured document that comprises a plurality of unassociated primitive elements to define a structured document comprising a hierarchy of structural elements, the structural elements defined by associating sets of primitive elements, the structured document having a reading order through the primitive elements;

detecting a first orientation of the handheld display device;

displaying a particular set of primitive elements in a first set of columns so as to maintain the reading order on the handheld display device;

detecting a second orientation of the handheld display device;

based on the second orientation, automatically reordering the particular set of primitive elements into a second set of columns, different than the first, while maintaining the reading order; and

displaying the particular set of primitive elements in the second set of columns on the handheld display device, wherein a first primitive element is located in a first direction relative to a second primitive element in the first set of columns and the first primitive element is located in a second, different direction relative to the second primitive element in the second set of columns.

21. The computer readable medium of claim 20, wherein the first orientation is a horizontal orientation and the second orientation is a vertical orientation.

22. The computer readable medium of claim 21, wherein the first set of columns comprises three columns and the second set of columns comprises two columns.

23. The computer readable medium of claim 20, wherein the handheld display device comprises an accelerometer for determining the orientation of the display device.

24. The computer readable medium of claim 20, wherein the handheld display device is an iPhone®

.

25. A method for defining a program for displaying a document, the method comprising:

analyzing an unstructured document comprising a plurality of unassociated primitive elements in order to generate a plurality of structural elements arranged in a hierarchical tree structure;

displaying the document in the display area using the plurality of structural elements arranged in the hierarchical tree structure;

receiving a selection of a location on the device as a position of interest, the location corresponding to one of the primitive elements;

identifying a first structural element, at a lowest level of the hierarchical tree structure, the bounds of which contain the position of interest;

traversing up to structural elements at higher levels of the hierarchical tree structure until a structural element that qualifies as a region of interest is reached, the bounds of which contain the position of interest; and

modifying the display of the document in order to focus the display on the identified region of interest.

26. The method of claim 25, wherein the method is performed by firmware of a handheld device.

27. The method of claim 25, wherein modifying the display of the document comprises centering the region of interest within the display vertically and horizontally.

28. The method of claim 25, wherein the region of interest has a width and a height.

29. The method of claim 28, wherein modifying the display of the document comprises zooming the display of the region of interest such that the width and height of the region of interest are maximized within the display.

30. The method of claim 25, wherein the first structural element is a word and the structural element that qualifies as a region of interest is a paragraph containing the word.

31. The method of claim 25, wherein identifying a first structural element is a graphic object that qualifies as a region of interest.