Image segmentation and warping for specialized display viewing

• US 7,278,117 B2
• Filed: 07/02/2002
• Issued: 10/02/2007
• Est. Priority Date: 07/02/2002
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Gargi, Ullas
• Primary Examiner(s): Lo, Weilun
• Assistant Examiner(s): THERIAULT, STEVEN B
• Application Number:
  US10/186,694
• Publication Number:
  US 20040004641A1
• Time in Patent Office: 1,918 Days
• Field of Search: 382/176, 382/282, 382/306, 709/230, 715/864, 345/667
• US Class Current: 715/864
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

A method, program product and system for warping image data, the method comprising in one embodiment: receiving image data for an image; segmentation of the image into a plurality of regions based on regional classification of the image; obtaining a set of warping rules; and warping the image based on the classification of each region and the warping rules.

Claims

1. A method for modifying an image by a server, comprising:

obtaining image data for the image;

determining whether the image is a graphics heavy image or a text heavy image;

obtaining first set of warping rules in response to determining that the image is a graphics heavy image;

obtaining a second set of warping rules in response to determining that the image is a test heavy image;

segmenting the image into a plurality of regions based on regional classification of the image, the segmentation performed in response to a request by a client;

classifying each region of the image into a data type of a plurality of data types, wherein each data type has a corresponding warping rule within the obtained first or second set of warping rules, and wherein multiple elements in a region are operable to be classified under a single data type;

warping each region of the segmented image using the corresponding warping rule for the data type of that region; and

transmitting the warped image data to the client, wherein the warped image data represents a modified version of the image.

2. The method as defined in claim 1, further comprising:

receiving a user interaction with a warped image; and

translating the interaction to the image.

3. The method as defined in claim 2, further comprising passing the interaction back to an originating application that originated the image.

4. The method as defined in claim 1, wherein the plurality of regions are indexed in a hierarchical data structure.

5. The method as defined in claim 4, wherein the classifications include at least two of a bitmap, graphic, text word, and text line.

6. The method as defined in claim 1, wherein the segmentation step includes the step of correlating each region to a set of coordinates.

7. The method as defined in claim 1, further comprising obtaining at least one warping rule prior to warping the segmented image.

8. A method for displaying a warped image, comprising:

receiving image data far an image with classification of regions, wherein each region has been classified into a data type of a plurality of data types, wherein multiple elements in a region are operable to be classified under a single data type;

obtaining rules for warping the image so the image can be displayed on a display with different display characteristics, wherein each data type has a corresponding rule for warping, and the obtained rules are the corresponding rules for warping for the data types that the regions are classified in;

warping the image based on classification of each region and the corresponding rules for warping for the data types that the regions are classified in, wherein warping the image includes applying a scaling algorithm to the image to obtain at least one scaled region within the image and moving the at least one scaled region to a predetermined location in the warped image; and

displaying the warped image.

9. The method as defined in claim 8, further comprising:

receiving a user interaction with the warped image; and

translating the interaction to the image.

10. The method as defined in claim 9, further comprising passing the interaction in the image back to an originating application that originated the image data.

11. The method as defined in claim 8, wherein a warping rule obtained in said obtaining rules for warping step comprises splitting at least one region classified as a text block into a plurality of lines, and splitting text lines into a plurality of words.

12. The method as defined in claim 11, wherein the words are scaled and rearranged within the warped image, but maintaining adjacent words consecutive.

13. The method as defined in claim 10, wherein the obtaining rules for warping step comprises obtaining a different set of rules based on whether the image is a web page or a text document.

14. The method as defined in claim 10, wherein the obtaining rules for warping step comprises obtaining a different set of rules based on whether the image is a graphics heavy web page or a text heavy web page.

15. The method as defined in claim 10, wherein the obtaining rules for warping step comprises obtaining a set of rules via selection by a user.

16. The method as defined in claim 10, wherein the obtaining rules for warping step comprises obtaining rules from a user profile.

17. The method as defined in claim 10, wherein the obtaining rules for warping step comprises selecting rules based on an available bandwidth designation.

18. A method for warping image data comprising:

receiving the image data for an image;

determining whether the image is a graphics heavy image or a text heavy image;

obtaining a first set of warping rules in response to determining that the image is a graphics heavy image;

obtaining a second set of warping rules in response to determining that the image is a text heavy image;

segmenting the image into a plurality of regions based on regional classification of the image;

classifying each region of the image into a data type of a plurality of data types, wherein each data type has a corresponding warping rule, and wherein multiple elements in a region are operable to be classified under a single data type;

displaying the image on a display with different display charactetistics, wherein the different display characteristics are based upon the determination of whether the image is a graphics heavy image or a text heavy image, wherein each data type has a corresponding rule for warping within the obtained first or second set of warping rules, and the corresponding warping rules are the corresponding rules for warping for the data types that the regions are classified in; and

warping the image based on the classification of each region and the corresponding warping rules.

19. The method as defined in claim 18, further comprising displaying the warped image.

20. The method as defined in claim 18, wherein the segmentation of the image into a plurality of regions based on regional classification of the image step includes correlating each region to a set of coordinates.

27. The program product as defined in claim 20, wherein the segmentation of the image into a plurality of regions based on regional classification of the image step includes correlating each region to a set of coordinates.

21. A program product comprising memory encoded with machine readable program code for causing a server to

determine whether an image is a graphics heavy image or a text heavy image;

obtain a first set of warping rules in response to determining that the image is a graphics heavy image;

obtain a second set of warping rules in response to determining that the image is a text heavy image;

perform segmentation of the image into a plurality of regions based on regional classification of the image in response to a request by a client;

classify each region of the image into a data type of a plurality of data types, wherein each data type has a corresponding warping rule within the obtained first or second set of warping rules, and wherein multiple elements in a region are operable to be classified under a single data type;

warp the segmented image, wherein each data type of the segmented image is warped according to the corresponding warping rule; and

transmit the warped image data to the client, wherein the warped image data represents a modified version of the image.

22. The program product as defined in claim 21, wherein the code causes the server to receive a user interaction with a warped image, translate the interaction to the image.

23. The program product as defined in claim 22, wherein the code causes the server to pass the interaction in the image back to an originating application that originated the image.

24. The program product as defined in claim 21, wherein the segmentation of the image into a plumlity of regions based on regional classification of the image step comprises determining a classification for each of a plurality of regions in the image based on a data type for data in the region.

25. The program product as defined in claim 24, wherein the plurality of regions are indexed in a hierarchical data structure.

26. The program product as defined in claim 25, wherein the classifications include at least two of a bitmap, graphic, text word, and text line.

28. A program product for client-side processing of an image, the product comprising memory encoded with machine readable program code for causing a machine to

receive image data for a segmented image with classification of regions, wherein each region of the image is classified into a data type of a plurality of data types, and wherein multiple elements in a region are operable to be classified under a single data type;

obtain rules for warping the image so the image can be displayed with different display characteristics, wherein each data type contains corresponding rules for warping and the rules are obtained based on an available bandwidth; and

warp the image based on the classification of each region and the rules for warping corresponding to the data types that the regions are classified in.

29. The program product as defined in claim 28, wherein the code also causes the machine to receive user interaction with the warped image and translate the interaction to the image.

30. The program product as defined in claim 29, wherein the code also causes the machine to pass the interaction in the image back to an originating application that originated the image data.

31. The program product as defined in claim 28, wherein a warping rule obtained in said obtaining rules for warping step comprises applying an image scaling algorithm to all images to obtain scaled images.

32. The program product as defined in claim 31, wherein a warping rule obtained in said obtaining rules for warping step comprises moving at least one scaled image to a predetermined location in the warped image.

33. The program product as defined in claim 28, wherein the obtaining rules for warping step comprises obtaining a set of rules via selection by a user.

34. The program product as defined in claim 28, wherein the obtaining rules for warping step comprises obtaining rules from a user profile.

35. A program product comprising memory encoded with machine readable program code for causing a machine to

determine whether an image is a graphics heavy image or a text heavy image;

obtain a first set of warping rules in response to determining that the image is a graphics heavy image;

obtain a second set of warping rules in response to determining that the image is a text heavy image;

perform segmentation of the image into a plurality of regions based on regional classification of the image;

classify each region of the image into a data type of a plurality of data types, wherein each data type has a corresponding warping rule, and wherein multiple elements in a region are operable to be classified under a single data type;

display the image on a display with different display characteristics, wherein the different display characteristics are based upon the determination of whether the image is a graphics heavy image or a text heavy image; and

warp the image based on the classification of each segment and the corresponding warping rule corresponding to the data type within the obtained first or second set of warping rules.

36. The program product as defined in claim 35, wherein the segmentation of the image into a plurality of regions based on regional classification of the image step includes correlating each region to a set of coordinates.

37. The program product as defined in claim 35, further comprising code for:

receiving a user interaction with a warped image; and

translating the interaction to the image.

38. The program product as defined in claim 37, further comprising code for passing the interaction in the image back to an originating application that originated the image.

39. The program product as defined in claim 35, wherein the segmentation of the image into a plurality of regions based on regional classification of the image step comprises determining a classification for each of a plurality of regions in the image based on a data type for data in the region.

40. The program product as defined in claim 39, wherein the plurality of regions are indexed in a hierarchical data structure.

41. A system comprising:

a component for obtaining image data for an image;

a component for determining whether the image is a graphics heavy image or a text heavy image;

a component for obtaining a first set of warping rules in response to determining that the image is a graphics heavy image and for obtaining a second set of warping rules in response to determining that the image is a text heavy image;

a component for performing segmentation of the image into a plurality of regions based on regional classification of the image, wherein the segmentation includes classifying each region of the image into a data type of a plurality of data types, wherein each data type has a corresponding warping rule within the obtained first or second set or warping rules, and wherein multiple elements in a region are operable to be classified under a single data type, the segmentation performed in response to a request by another party;

a component for warping the segmented image using the corresponding warping rules for the data types that the regions are classified in; and

a component for transmitting the warped image data.

42. The system as defined in claim 41, further comprising:

a component for receiving a user interaction with a warped image; and

a component for translating the interaction to the image.

43. The system as defined in claim 41, wherein the segmentation component determines a classification for each of a plurality of regions in the image based on a data type for data in the region.

44. The system as defined in claim 43, wherein the plurality of regions are indexed in a hierarchical data structure.

45. A client comprising a component for

receiving image data for an image with classification of regions, wherein each region is further classified into a data type of a plurality of data types, wherein each data type has a corresponding warping rule, which are obtained based on the amount of bandwidth available to the client, and wherein multiple elements in a region are operable to be classified under a single data type;

and

a component for warping the image based on the classification of each region and the warping rules corresponding to the data types that the regions are classified in, so the warped image has display charaeleristies that are different than those of the image data.

46. The client as defined in claim 45, further comprising a component for displaying the warped image.

47. The client as defined in claim 45, further comprising:

a component for receiving a user interaction with the warped image; and

a component for translating the interaction to the image.

48. The client as defined in claim 45, wherein the obtaining rules component obtains a different set of rules based on whether the image is a web page or a text document.

49. The client as defined in claim 45, wherein the obtaining rules component obtains rules from a user profile.

50. A system for warping image data comprising

means for receiving image data;

means for performing segmentation of the image into a plurality of regions based on regional classification of the image, wherein each region is further classified into a data type of a plurality of data types, wherein each data type has a corresponding warping rule, and wherein multiple elements in a region are operable to be classified under a single data type;

means for obtaining a set of warping rules for warping the image so the image can be displayed on a display with different display charaeleristies; and

means for warping the image based on the classification of each region and the corresponding warping rules for the data types that the regions are classified in, wherein warping the image includes applying a scaling algorithm to the image to obtain at least one sealed region within the image and moving the at least one scale region to a predetermined location within the warped image.