RENDERING PERSONALIZED TEXT ON CURVED IMAGE SURFACES

• US 20120146991A1
• Filed: 12/10/2010
• Published: 06/14/2012
• Est. Priority Date: 12/10/2010
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University)), Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Bala, Raja, Fan, Zhigang, Eschbach, Reiner, Ding, Hengzhou, Allebach, Jan P., Bouman, Charles A.
• Granted Patent: US 8,619,074 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

As set forth herein, a computer-implemented method facilitates replacing text on cylindrical or curved surfaces in images. For instance, the user is first asked to perform a multi-click selection of a polygon to bound the text. A triangulation scheme is carried out to identify the pixels. Segmentation and erasing algorithms are then applied. The ellipses are estimated accurately through constrained least squares fitting. A 3D framework for rendering the text, including the central projection pinhole camera model and specification of the cylindrical object, is generated. These parameters are jointly estimated from the fitted ellipses as well as the two vertical edges of the cylinder. The personalized text is wrapped around the cylinder and subsequently rendered.

Claims

1. A computer-implemented method for placing personalized text onto a curved surface in an image, comprising:

receiving user input pertaining to a location of an existing text string in the image;

defining bounding polygon according to user input;

segmenting the existing text string;

performing connected component analysis on the text string to identify connected components in the existing text string;

detecting at least one of upper and lower extreme edge pixels of text characters represented by the connected components;

identifying an edge at each side of the curved surface;

calculating 3D curved surface geometry and camera geometry using the upper and lower extreme edge pixels of the text characters and left and right edges;

erasing the existing text string by replacing text pixels with background-colored pixels; and

inserting a personalized text string into the location of the erased text according to the calculated 3D curved surface geometry and camera geometry.

2. The method according to claim 1, wherein the user input comprises a plurality of points on the image selected by the user using a graphical user interface (GUI), and further comprising defining the bounding polygon as a function of the user-selected points.

3. The method according to claim 1, further comprising segmenting the existing text string according to pixel color.

4. The method according to claim 1, wherein the curved surface is cylindrical and the upper and lower extreme edge pixels of the text characters fall on an ellipse within the image.

5. The method according to claim 4, wherein calculating 3D curved surface geometry and camera geometry comprises generating a 3D central projection pinhole camera model.

6. The method according to claim 5, wherein the 3D central projection pinhole camera model is defined as:

[E, l_L, l_R]=M(f, {right arrow over (n)}, {right arrow over (s)})

where E is a projected ellipse, l_L is a left edge of the cylinder, l_R is a right edge of the cylinder, M is the model that maps a 3D object to its 2D projection on an image plane, f is a focal length from a point of origin to an image plane, {right arrow over (n)} is a normal vector, and {right arrow over (s)} is a shift vector.

7. The method according to claim 6, further comprising:

building a cost function of the 3D geometry parameters f, {right arrow over (n)}, {right arrow over (s)} that describes a degree to which the projected ellipse and left and right cylinder edges match the measured ellipse and left and right cylinder images obtained from image data; and

calculating 3D geometry parameters f, {right arrow over (n)}, {right arrow over (s)} that minimize the said cost function.

8. The method according to claim 7 wherein the cost function C comprises a first component C₁ that describes a degree to which the projected ellipse obtained from the model M matches a measured ellipse computed from extreme edge pixels of text characters in the image, and a second component C₂ that describes a degree to which the projected left and right edges l_L and l_R obtained from the model M match the left and right edges identified within the image.

9. The method of claim 8 wherein the first component C₁ is determined as a function of:

a distance between center points of the projected ellipse and the measured ellipse;

a difference between the lengths of semi-minor axes of the projected ellipse and the measured ellipse;

a difference between the lengths of semi-major axes of the projected ellipse and the measured ellipse; and

an angle between the semi major axes of the projected ellipse and the measured ellipse.

10. The method according to claim 9 wherein the first component C₁ is computed as a function of distances from the detected extreme edge pixels of text characters to the projected ellipse E.

11. The method according to claim 8, wherein the second component C₂ is determined as a function of distances between endpoints of projected and identified edge lines bounding the edges of the cylinder.

12. A system that facilitates replacing an existing text string on a curved surface in an image with a personalized text string, comprising:

a computer-readable medium that stores computer-executable instructions;

a processor that executes the computer-executable instructions, the instructions comprising;

receiving user input pertaining to a location of an existing text string in the image;

defining bounding polygon according to user input;

segmenting the existing text string;

performing connected component analysis on the text string to identify connected components in the existing text string;

detecting upper and lower extreme edge pixels of text characters represented by the connected components;

identifying an edge at each side of the curved surface;

calculating 3D curved surface geometry and camera geometry using the upper and lower extreme edge pixels of the text characters and identified edges;

erasing the existing text string by replacing text pixels with background-colored pixels; and

inserting a personalized text string into the location of the erased text according to the calculated 3D curved surface geometry and camera geometry.

13. The system according to claim 12, wherein the user input comprises a plurality of points on the image selected by the user using a graphical user interface (GUI), and the instructions further comprising defining the bounding polygon as a function of the user-selected points.

14. The system according to claim 12, the instructions further comprising segmenting the existing text string according to at least one of pixel color and pixel luminescence.

15. The system according to claim 12, wherein the curved surface is cylindrical.

16. The system according to claim 12, wherein calculating 3D curved surface geometry and camera geometry comprises generating a 3D central projection pinhole camera model.

17. The system according to claim 16, wherein the 3D central projection pinhole camera model is defined as:

[E, l_L, l_R]=M(f, {right arrow over (n)}, {right arrow over (s)})

where E is a projected ellipse, l_L is a left edge of the curved surface, l_R is a right edge of the curved surface, M is the model, f is a focal length from a point of origin to an image plane, {right arrow over (n)} is a normal vector, and {right arrow over (s)} a shift vector.

18. The system according to claim 17, the instructions further comprising building a cost function that describes a degree to which an output ellipse matches a measured ellipse in the image.

19. The system according to claim 18, wherein:

the cost function C comprises a first component C₁ and a second component C₂;

wherein the first component is determined as a function of;

a distance between center points of the output ellipse and the measured ellipse;

a difference between the lengths of semi-minor axes of the output ellipse and the measured ellipse;

a difference between the lengths of semi-major axes of the output ellipse and the measured ellipse; and

an angle between the semi major axes of the output ellipse and the measured ellipse; and

wherein the second component is determined as a function of distances between endpoints of edge lines bounding the edges of the curved surface.

20. A computer-implemented method for placing personalized text onto a curved surface in an image, comprising:

bounding a region on a curved surface in the image;

segmenting an existing text string in the bounded region;

detecting upper and lower extreme edge pixels of text characters represented by connected components in the bounded region;

identifying an edge at each side of the curved surface;

calculating 3D curved surface geometry and camera geometry;

replacing existing text pixels in the bounded region with background-colored pixels; and

inserting a personalized text string into the bounded region according to the calculated 3D curved surface geometry and camera geometry.

21. The method according to claim 20, wherein the curved surface is cylindrical.