Enhancing accuracy of jumping by incorporating interestingness estimates

• US 7,836,388 B2
• Filed: 02/01/2006
• Issued: 11/16/2010
• Est. Priority Date: 02/01/2006
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: McCall, Kim, Piersol, Kurt
• Primary Examiner(s): Stork; Kyle R
• Application Number:
  US11/346,856
• Publication Number:
  US 20070180355A1
• Time in Patent Office: 1,749 Days
• Field of Search: 715/234, 715/243, 715/254, 715/255, 715/200
• US Class Current: 715/200
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A method and apparatus is disclosed herein for enhancing accuracy of jumping by incorporating interestingness estimates. In one embodiment, the method comprises serially displaying a sequence of page images; receiving a user input to stop sequencing through the page images; determining a location of a page image in the sequence of page images representing a point at which the user intended to stop in the sequence of page images but for display speed of the sequence of images and user reaction time in providing an indication of the point at which the user intended to stop, the location being selected based on visual distinctiveness of the page image at the location and the reaction time associated with the user; and automatically jumping back to the page image at the location in the sequence of page images.

Claims

1. A method comprising:

receiving, by a computing machine that displays page images, an indication from a user to stop scrolling through a sequence of the page images when the user views a target location of a page image in the sequence in the page images that the user wants to stop based on an interestingness of the target location to the user, wherein the indication corresponds to an actual stopping location of the computing machine in the sequence of page images that is past the target location due to a reaction time of the user and a display speed of the sequence of images;

determining, by the computing machine, an interestingness score of each of the page images within a predetermined range prior to the actual stopping location by comparing one or more other page images proximally located to the page image, the interestingness score based on occurrences of graphical features on the page images and distinction of graphical features on each page from graphics features on the proximally-located one or more other page images;

determining, by the computing machine, the target location based on the display speed of the sequence of images, the user reaction time, and the determined interestingness scores of the page images proximally-located previous to the actual stopping location; and

automatically jumping, by the computing machine, to the target location in order to display to the user the page image at which the user intended to stop.

2. The method defined in claim 1 wherein receiving an indication of a target location comprises:

serially displaying a sequence of page images; and

receiving a user input to stop sequencing through the page images.

3. The method defined in claim 1 wherein the indication is based on a location within a scroll bar at which a click occurred.

4. The method defined in claim 1 wherein visual distinctiveness is indicated, in part, based on an interestingness measure.

5. The method defined in claim 4 further comprising rating each page in terms of its variance from other nearby pages of the image-based document.

6. The method defined in claim 4 wherein the interestingness measure is based on one or more features on each page image.

7. The method defined in claim 6 further comprising:

rating each page in terms of its variance from other pages of the image-based document on each of the one or more features; and

computing a score for each page image based on a combination of scores associated with each of the one or more features.

8. The method defined in claim 7 wherein computing the score for each page comprises:

multiplying the score for each feature by a feature weight; and

summing products associated with each of the one or more features multiplied by its associated feature weight.

9. The method defined in claim 6 wherein the one or more features include one or more of a group consisting of chapter headings present, section headings present, number of any pictures or graphs present, size of any pictures or graphs present, number of any tables present, size of any tables present, number of any columns present, presence of a blank area preceding a page image where the blank area is disproportionately larger than blank areas on other nearby pages.

10. The method defined in claim 6 wherein each of the one or more features is weighted.

11. The method defined in claim 1 further comprising ranking page images in a region that is near and includes the page image at the location.

12. The method defined in claim 1 further comprising:

computing an interestingness measure for each of a plurality of page images;

identifying, independent of the distinctiveness measures, the area of successive page images using a probability curve, based on the reaction time;

multiplying the interestingness measures by the probability curve to favor page images near the center of the area over those toward the ends of the area; and

selecting the page image at the location based on interestingness and probability comparison to any other page images in the area.

13. The method defined in claim 12 wherein identifying the location is based on a probability curve.

14. The method defined in claim 1 wherein the location is selected based on an interestingness measure of a page at the location.

15. The method defined in claim 1 wherein the page images are from a plurality of page images corresponding to an image-based document, the plurality of page images including merged page images and each merged page image having salient content from multiple page images of the image-based document merged into one image.

16. The method defined in claim 1 wherein the plurality of page images includes a first set of merged page images having salient content from a first number of page images of the image-based document merged into one image and a second set of merged page images having salient content from a second number of page images of the image-based document merged into one image, the first and second numbers being different, and further wherein selecting the page images for display comprises selecting the first set of merged page images if the rate at which the page images are to be display is a first-rage and selecting the second set of merged page images if the rate at which the page images are to be display is at a the second rate.

17. The method defined in claim 1 further comprising:

identifying salient content on pages of the image-based document; and

creating the plurality of page images by merging identified salient content of multiple pages of the image based document into single pages.

18. The method defined in claim 17 wherein creating the plurality of pages comprises:

creating a first set of merged pages having salient content from a first number of page images of the image-based document merged into one image;

creating a second set of merged pages having salient content from a second number of page images of the image-based document merged into one image, the second number being twice the first number;

creating a third set of merged pages having salient content from a third number of page images of the image-based document merged into one image, the third number being twice the second number; and

wherein selecting the page images comprises selecting the first, second or third sets based on a speed control input.

19. The method defined in claim 17 wherein identifying the salient content comprises:

computing a histogram of each page of the image-based document; and

finding features based on information from the histogram.

20. The method defined in claim 1 wherein page images are displayed, as a function of an input control, according to a merge level indicative of a number of pages of the image-based document per merged image and a rate indicative of the number of merged page images per second based on the merge level.

21. The method defined in claim 20 wherein the page images are displayed in order forward or backward based on a direction indication.

22. An article of manufacture having one or more recordable media storing instructions thereon which, when executed by a system, causes the system to perform a method comprising:

receiving an indication from a user to stop scrolling through a sequence of the page images when the user views a target location of a page image in the sequence in the page images that the user wants to stop based on an interestingness of the target location to the user, wherein the indication corresponds to an actual stopping location of a computing machine in the sequence of page images that is past the target location due to a reaction time of the user and a display speed of the sequence of images;

determining an interestingness score of each of the page images within a predetermined range prior to the actual stopping location by comparing one or more other page images proximally located to the page image, the interestingness score based on occurrences of graphical features on the page images and distinction of graphical features on each page from graphics features on the proximally-located one or more other page images;

determining the target location based on the display speed of the sequence of images, the user reaction time and the determined interestingness scores of the page images proximally-located previous to the actual stopping location; and

automatically jumping to the target location in order to display to the user the page image at which the user intended to stop.

23. The article of manufacture defined in claim 22 wherein receiving an indication of a target location comprises:

serially displaying a sequence of page images; and

receiving a user input to stop sequencing through the page images.

24. The article of manufacture defined in claim 22 wherein the indication is based on a location within a scroll bar at which a click occurred.

25. The article of manufacture defined in claim 22 wherein visual distinctiveness is indicated, in part, based on an interestingness measure.

26. The article of manufacture defined in claim 25 wherein the method further comprises rating each page in terms of its variance from other nearby pages of the image-based document.

27. The article of manufacture defined in claim 25 wherein the interestingness measure is based on one or more features on each page image.

28. The article of manufacture defined in claim 27 wherein the method further comprises:

rating each page in terms of its variance from other pages of the image-based document on each of the one or more features; and

computing a score for each page image based on a combination of scores associated with each of the one or more features.

29. The article of manufacture defined in claim 28 wherein computing the score for each page comprises:

multiplying the score for each feature by a feature weight; and

summing products associated with each of the one or more features multiplied by its associated feature weight.

30. The article of manufacture defined in claim 27 wherein the one or more features include one or more of a group consisting of chapter headings present, section headings present, number of any pictures or graphs present, size of any pictures or graphs present, number of any tables present, size of any tables present, number of any columns present, presence of a blank area preceding a page image where the blank area is disproportionately larger than blank areas on other nearby pages.

31. The article of manufacture defined in claim 27 wherein each of the one or more features is weighted.

32. The article of manufacture defined in claim 22 wherein the method further comprises ranking page images in a region that is new and includes the page image at the location.

33. The article of manufacture defined in claim 22 wherein the method further comprises:

computing an interestingness measure for each of a plurality of page images;

identifying, independent of the distinctiveness measures, the area of successive page images using a probability curve, based on the reaction time;

multiplying the interestingness measures by the probability curve to favor page images new the center of the area over those toward the ends of the area; and

selecting the page image at the location based on interestingness and probability in comparison to any other page images in the area.

34. The article of manufacture defined in claim 33 wherein identifying the location is based on a probability curve.

35. The article of manufacture defined in claim 22 wherein the location is selected based on an interestingness measure of a page at the location.

36. The article of manufacture defined in claim 22 wherein the page images are from a plurality of page images corresponding to are image-based document, the plurality of page images including merged page images and each merged page image having salient content from multiple page images of the image-based document merged into one image.

37. The article of manufacture defined in claim 22 wherein the plurality of page images includes a first set of merged page images having salient content from a first number of page images of the image-based document merged into one image and a second set of merged page images having salient content from a second number of page images of the image-based document merged into one image, the first and second numbers being different, and further wherein selecting the page images for display comprises selecting the first set of merged page images if the rate at which the page images are to be display is a first rage and selecting the second set of merged page images if the rate at which the page images are to be display is at a the second rate.

38. The article of manufacture defined in claim 22 wherein the method further comprises:

identifying salient content on pages of the image-based document; and

creating the plurality of page images by merging identified salient content of multiple pages of the image based document into single pages.

39. The article of manufacture defined in claim 38 wherein creating the plurality of pages comprises:

creating a first set of merged pages having salient content from a first number of page images of the image-based document merged into one image;

creating a second set of merged pages having salient content from a second number of page images of the image-based document merged into one image, the second number being twice the first number;

creating a third set of merged pages having salient content from a third number of page images of the image-based document merged into one image, the third number being twice the second number; and

wherein selecting the page images comprises selecting the first, second or third sets based on a speed control input.

40. The article of manufacture defined in claim 38 wherein identifying the salient content comprises:

computing a histogram of each page of the image-based document; and

finding features based on information from the histogram.

41. The article of manufacture defined in claim 22 wherein page images are displayed, as a function of an input control, according to a merge level indicative of a number of pages of the image-based document per merged image and a rate indicative of the number of merged page images per second based on the merge level.

42. The article of manufacture defined in claim 41 wherein the page images are displayed in order forward or backward based on a direction indication.