Systems and methods for enhanced image adaptation
First Claim
1. A method for enhanced image adaptation for a display device having a target area T, the method comprising:
- modeling an image with respect to a plurality of visual attentions to generate a respective set of attention objects for each attention of the visual attentions;
assigning a respective attention value to each attention object, the attention value indicating a relative importance of image information represented by the attention object as compared to other ones of the attention objects;
indicating a respective weight for each of the visual attentions;
for each visual attention, adjusting assigned attention values of corresponding attention objects based on corresponding visual attention weight, the adjusting comprising;
normalizing the assigned attention values to a unit interval; and
computing adjusted attention values according to;
wherein AVi is an adjusted attention value for attention object i derived from a visual attention model k, wk is a weight of the visual attention model k, and AVik is a normalized attention value of attention object AOi detected for the visual attention model k;
for each of one or more image adaptation schemes, determining a respective objective measure of information fidelity (IF) of a region R of the image as a function of a resource constraint of the display device and as a function of a weighted sum of IF of a value of each attention object (AOi) in the region R;
selecting a substantially optimal adaptation scheme from among the one or more image adaptation schemes, each image adaptation scheme ranked as a function of, at least, its determined objective measure of information fidelity IF with respect to the region R; and
adapting the image via the substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area T.
2 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods for adapting images for substantially optimal presentation by heterogeneous client display sizes are described. In one aspect, an image is modeled with respect to multiple visual attentions to generate respective attention objects for each of the visual attentions. For each of one or more image adaptation schemes, an objective measure of information fidelity (IF) is determined for a region R of the image. The objective measures are determined as a function of a resource constraint of the display device and as a function of a weighted sum of IF of each attention object in the region R. A substantially optimal adaptation scheme is then selected as a function of the calculated objective measures. The image is then adapted via the selected substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area of the client display.
-
Citations
49 Claims
-
1. A method for enhanced image adaptation for a display device having a target area T, the method comprising:
-
modeling an image with respect to a plurality of visual attentions to generate a respective set of attention objects for each attention of the visual attentions; assigning a respective attention value to each attention object, the attention value indicating a relative importance of image information represented by the attention object as compared to other ones of the attention objects; indicating a respective weight for each of the visual attentions; for each visual attention, adjusting assigned attention values of corresponding attention objects based on corresponding visual attention weight, the adjusting comprising; normalizing the assigned attention values to a unit interval; and computing adjusted attention values according to; wherein AVi is an adjusted attention value for attention object i derived from a visual attention model k, wk is a weight of the visual attention model k, and AVik is a normalized attention value of attention object AOi detected for the visual attention model k; for each of one or more image adaptation schemes, determining a respective objective measure of information fidelity (IF) of a region R of the image as a function of a resource constraint of the display device and as a function of a weighted sum of IF of a value of each attention object (AOi) in the region R; selecting a substantially optimal adaptation scheme from among the one or more image adaptation schemes, each image adaptation scheme ranked as a function of, at least, its determined objective measure of information fidelity IF with respect to the region R; and adapting the image via the substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area T. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
and wherein ROIi is a spatial size of AOi.
-
-
7. A method as recited in claim 1, wherein selecting the substantially optimal adaptation scheme further comprises searching a region of the image for an attention object set that represents higher IF after adaptation as compared to collective IF values of other attention object sets after adaptation.
-
8. A method as recited in claim 1, wherein selecting the substantially optimal adaptation scheme further comprises calculating the substantially optimal adaptation scheme via integer programming and use of a branch and bound algorithm.
-
9. A method as recited in claim 1, wherein selecting the substantially optimal adaptation scheme is based on the following:
wherein I is a set of AOs, ROI is a spatial size of AOi, and MPSi is a minimal perceptible size beyond which AOi is not to be reduced.
-
10. A method as recited in claim 1, wherein adapting the image via the substantially optimal adaptation scheme to generate an adapted image further comprises iteratively scaling the region R to a region r comprising a determined set of optimal attention objects (Iopt) by extending and/or contracting the region while keeping Iopt valid.
-
11. A method as recited in claim 1, wherein adapting the image further comprises scaling the region R to a region r comprising a determined set of optimal attention objects (Iopt) according to the following scaling equation:
wherein MPSi is the Minimal Perceptible Size of AOi, wherein ROIi is a spatial area of AOi in the image; and wherein the sealing equation is solved such that Iopt remains valid and such that the region r is scaled to a largest region within target area T.
-
12. A method as recited in claim 1, wherein adapting the image further comprises seating the region R comprising a determined set of optimal attention objects (Iopt) such that Iopt remains valid and such that the region R is scaled to a highest resolution to fit within target area T.
-
13. A method as recited in claim 1, wherein the weighted sum of information fidelity (IF) of each (i) attention object (AO) in the region R is based on:
-
wherein u(x) is defined as
andwherein ROIi is a spatial size of AOi, wherein size(x) calculates the spatial size, and rR denotes a ratio of image scaling down.
-
-
14. A method as recited in claim 13, wherein the ratio of image scaling down is computed as follows:
-
and wherein WidthT, HeightT, WidthR, and HeightR represent widths and heights of target area T and solution region R, respectively.
-
-
15. A computer-readable medium storing computer-executable instructions for enhanced image adaptation for a display device having a target area T, the computer-executable instructions comprising instructions for:
-
modeling an image with respect to a plurality of visual attentions to generate a respective set of attention objects for each attention of the visual attentions; for each of one or more image adaptation schemes, determining a respective objective measure of information fidelity (IF) of a region R of the image as a function of, at least; a resource constraint of the display device; and a weighted sum of an information fidelity of each attention object (AOi) in the region R based on; wherein IFR corresponds to the weighted sum, ROIi is a spatial size of the attention object AOi, AVi is a value of the attention object AOi, and IFAOi is the information fidelity of the attention object AOi; selecting a substantially optimal adaptation scheme from among the one or more image adaptation schemes, each image adaptation scheme ranked as a function of, at least, its determined objective measure of information fidelity IF with respect to the region R; and adapting the image via the substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area T. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
wherein AVi is an adjusted attention value for attention object i derived from visual attention model k, wk is the weight of the visual attention model k, and AVik is a normalized mention value of attention object AOi detected in the visual attention model k.
-
-
22. A computer-readable medium as recited in claim 15, wherein the instructions for selecting the substantially optimal adaptation scheme further comprise instructions for searching a region of the image for an attention object set that represents higher IF after adaptation as compared to collective IF values of other attention object sets after adaptation.
-
23. A computer-readable medium as recited in claim 15, wherein the instructions for selecting the substantially optimal adaptation scheme further comprise instructions for calculating the substantially optimal adaptation scheme via integer programming and use of a branch and bound algorithm.
-
24. A computer-readable medium as recited in claim 15, wherein the instructions for selecting the substantially optimal adaptation scheme is based on the following:
wherein I is a set of AOs, ROI is a spatial size of AOi, and MPSi is a minimal perceptible size beyond which AOi is not to be reduced.
-
25. A computer-readable medium as recited in claim 15, wherein the instructions for adapting the image further comprise instructions for iteratively scaling the region R to a region r comprising a determined set of optimal attention objects (Iopt) by extending and/or contracting the region while keeping Iopt valid.
-
26. A computer-readable medium as recited in claim 15, wherein the instructions for adapting the image further comprise instructions for scaling the region R to a region r comprising a determined set of optimal attention objects (Iopt) according to the following scaling equation:
wherein MPSi is the Minimal Perceptible Size of AOi, wherein ROIi is a spatial area of AOi in the image; and wherein the scaling equation is solved such that Iopt remains valid and such that the region r is scaled to a largest region within target area T.
-
27. A computer-readable medium as recited in claim 15, wherein the instructions for adapting the image further comprise instructions for scaling the region R comprising a determined set of optimal attention objects (Iopt) such that Iopt remains valid and such that the region R is scaled to a highest resolution to fit within target area T.
-
28. A computer-readable medium as recited in claim 15, wherein the weighted sum of information fidelity (IF) of each (i) attention object (AO) in the region R is based on:
-
wherein u(x) is defined as
andwherein ROIi is a spatial size of AOi, wherein size(x) calculates the spatial size, and rR denotes a ratio of image scaling down.
-
-
29. A computer-readable medium as recited in claim 28, wherein the ratio of image scaling down is computed as follows:
-
and wherein WidthT, HeightT, WidthR, and HeightR represent widths and heights of target area T and solution region R, respectively.
-
-
30. A computing device for enhanced image adaptation for a display device having a target area T, the computing device comprising:
-
a processor; and a memory coupled to the processor, the memory comprising computer-program executable instructions executable by the processor for, at least; modeling an image with respect to a plurality of visual attentions to generate a respective set of attention objects for each attention of the visual attentions; for each of one or more image adaptation schemes, determining a respective objective measure of information fidelity (IF) of a region R of the image as a function of, at least; a resource constraint of the display device; and a weighted sum of an information fidelity of each attention object (AOi) in the region R; selecting a substantially optimal adaptation scheme from among the one or more image adaptation schemes, each image adaptation scheme ranked as a function of, at least, its determined objective measure of information fidelity IF with respect to the region R; and adapting the image via the substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area T, the adapting comprising; scaling the region R to a region r comprising a determined set of optimal attention objects (Iopt) according to the following scaling equation; wherein MPSi is a minimum perceptible size of AOi, and ROIi is a spatial area of AOi in the image; and wherein the scaling equation is solved such that Iopt remains valid and such that the region r is scaled to a largest region within the target area T. - View Dependent Claims (31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44)
wherein AVi is an adjusted attention value for attention object i derived from visual attention model k, wk is the weight of the visual attention model k, and AVik is a normalized attention value of attention object AOi detected in the visual attention model k.
-
-
37. A computing device as recited in claim 30, wherein the weighted sum of information fidelity (IF) of each (i) attention object (AO) in the region R is based on:
-
and wherein ROIi is a spatial size of AOi.
-
-
38. A computing device as recited in claim 30, wherein the instructions for selecting the substantially optimal adaptation scheme further comprise instructions for searching a region of the image for an attention object set that represents higher IF after adaptation as compared to collective IF values of other attention object sets after adaptation.
-
39. A computing device as recited in claim 30, wherein the instructions for selecting the substantially optimal adaptation scheme further comprise instructions for calculating the substantially optimal adaptation scheme via integer programming and use of a branch and bound algorithm.
-
40. A computing device as recited in claim 30, wherein the instructions for selecting the substantially optimal adaptation scheme is based on the following:
wherein I is a set of AOs, ROI is a spatial size of AOi, and MPSi is a minimal perceptible size beyond which AOi is not to be reduced.
-
41. A computing device as recited in claim 30, wherein the instructions for adapting the image further comprise instructions for iteratively scaling the region R to a region r comprising a determined set of optimal attention objects (Iopt) by extending and/or contracting the region while keeping Iopt valid.
-
42. A computing device as recited in claim 30, wherein the instructions for adapting the image further comprise instructions for scaling the region R comprising a determined set of optimal attention objects (Iopt) such that Iopt remains valid and such that the region R is scaled to a highest resolution to fit within target area T.
-
43. A computing device as recited in claim 30, wherein the weighted sum of information fidelity (IF) of each (i) attention object (AO) in the region R is based on:
-
wherein u(x) is defined as wherein ROIi is a spatial size of AOi, wherein size(x) calculates the spatial size, and rR denotes a ratio of image scaling down.
-
-
44. A computing device as recited in claim 43, wherein the ratio of image scaling down is computed as follows:
wherein WidthT, HeightT, WidthR, and HeightR represent widths and heights of target area T and solution region R, respectively.
-
45. A computer-implemented system for enhanced image adaptation for a display device having a target area T, the system comprising one or more program modules configured to, at least:
-
model an image with respect to a plurality of visual attentions to generate a respective set of attention objects for each attention of the visual attentions; determine, for each of one or more image adaptation schemes, a respective objective measure of information fidelity (IF) of a region R of the image as a function of, at least; a resource constraint of the display device; and a weighted sum of an information fidelity of each attention object (AOi) in the region R; select a substantially optimal adaptation scheme from among the one or more image adaptation schemes, each image adaptation scheme ranked as a function of, at least, its determined objective measure of information fidelity IF with respect to the region R, and the selecting based on the following; wherein I is a set of valid attention objects, IFI is an information fidelity of I, AVi is an attention value of AOi, rI is a scaling ratio for an image corresponding to I, size(ROIi) is a spatial size of a region ROIi corresponding to AOi, MPSi is a minimal perceptible size of AOi, and the function u(x) has the value 1 when x is nonnegative and 0 otherwise; and adapt the image via the substantially optimal adaptation scheme to generate an adapted image on an output display as a function of at least the target area T. - View Dependent Claims (46, 47, 48, 49)
-
Specification