Rendering personalized text on curved image surfaces

• US 8,619,074 B2
• Filed: 12/10/2010
• Issued: 12/31/2013
• Est. Priority Date: 12/10/2010
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Bala, Raja, Fan, Zhigang, Eschbach, Reiner, Ding, Hengzhou, Allebach, Jan P., Bouman, Charles A.
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US12/964,820
• Publication Number:
  US 20120146991A1
• Time in Patent Office: 1,117 Days
• Field of Search: 345/419, 345/421, 345/427, 345/441, 345/467, 382/104, 382/154, 382/295
• US Class Current: 345/419
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

As set forth herein, a computer-implemented method facilitates replacing text on cylindrical or curved surfaces in images. For instance, the user is first asked to perform a multi-click selection of a polygon to bound the text. A triangulation scheme is carried out to identify the pixels. Segmentation and erasing algorithms are then applied. The ellipses are estimated accurately through constrained least squares fitting. A 3D framework for rendering the text, including the central projection pinhole camera model and specification of the cylindrical object, is generated. These parameters are jointly estimated from the fitted ellipses as well as the two vertical edges of the cylinder. The personalized text is wrapped around the cylinder and subsequently rendered.

Claims

1. A computer-implemented method for placing personalized text onto a curved surface in an image, comprising:

receiving user input pertaining to a location of an existing text string in the image;

defining a bounding polygon according to user input;

segmenting the existing text string;

performing connected component analysis on the text string to identify connected components in the existing text string;

detecting at least one of upper and lower extreme edge pixels of text characters represented by the connected components;

identifying an edge at each side of the curved surface;

via a processor, calculating 3D curved surface geometry and camera geometry using the upper and lower extreme edge pixels of the text characters and left and right edges, wherein calculating the 3D curved surface geometry and camera geometry comprises generating a 3D central projection pinhole camera model defined as;

[E,l_L,l_R]=M(f,{right arrow over (n)},{right arrow over (s)})

where E is a projected ellipse, l_L is a left edge of the cylinder, l_R is a right edge of the cylinder, M is the model that maps a 3D object to its 2D projection on an image plane, f is a focal length from a point of origin to an image plane, {right arrow over (n)} is a normal vector, and {right arrow over (s)} is a shift vector;

erasing the existing text string by replacing text pixels with background-colored pixels; and

inserting a personalized text string into the location of the erased text according to the calculated 3D curved surface geometry and camera geometry to personalize the image, and outputting the personalized image.

2. The method according to claim 1, wherein the user input comprises a plurality of points on the image selected by the user using a graphical user interface (GUI), and further comprising defining the bounding polygon as a function of the user-selected points.

3. The method according to claim 1, further comprising segmenting the existing text string according to pixel color.

4. The method according to claim 1, wherein the curved surface is cylindrical and the upper and lower extreme edge pixels of the text characters fall on an ellipse within the image.

5. The method according to claim 1, further comprising:

building a cost function of the 3D geometry parameters f, {right arrow over (n)}, {right arrow over (s)} that describes a degree to which the projected ellipse and left and right cylinder edges match the measured ellipse and left and right cylinder images obtained from image data; and

calculating 3D geometry parameters f,{right arrow over (n)}, {right arrow over (s)} that minimize the said cost function.

6. The method according to claim 5 wherein the cost function C comprises a first component C₁that describes a degree to which the projected ellipse obtained from the model M matches a measured ellipse computed from extreme edge pixels of text characters in the image, and a second component C₂that describes a degree to which the projected left and right edges l_Land l_Robtained from the model M match the left and right edges identified within the image.

7. The method of claim 6 wherein the first component C₁is determined as a function of:

a distance between center points of the projected ellipse and the measured ellipse;

a difference between the lengths of semi-minor axes of the projected ellipse and the measured ellipse;

a difference between the lengths of semi-major axes of the projected ellipse and the measured ellipse; and

an angle between the semi major axes of the projected ellipse and the measured ellipse.

8. The method according to claim 7 wherein the first component C₁is computed as a function of distances from the detected extreme edge pixels of text characters to the projected ellipse E.

9. The method according to claim 6, wherein the second component C₂is determined as a function of distances between endpoints of projected and identified edge lines bounding the edges of the cylinder.

10. A system that facilitates replacing an existing text string on a curved surface in an image with a personalized text string, comprising:

a non-transitory computer-readable medium that stores computer-executable instructions;

a processor that executes the computer-executable instructions, the instructions comprising;

receiving user input pertaining to a location of an existing text string in the image;

defining a bounding polygon according to user input;

segmenting the existing text string;

performing connected component analysis on the text string to identify connected components in the existing text string;

detecting upper and lower extreme edge pixels of text characters represented by the connected components;

identifying an edge at each side of the curved surface;

calculating 3D curved surface geometry and camera geometry using the upper and lower extreme edge pixels of the text characters and identified edges, wherein calculating 3D curved surface geometry and camera geometry comprises generating a 3D central projection pinhole camera model defined as;

[E,l_L,l_R]=M(f,{right arrow over (n)},{right arrow over (s)})

where E is a projected ellipse, l_L, is a left edge of the curved surface, l_Ris a right edge of the curved surface, M is the model,f is a focal length from a point of origin to an image plane, {right arrow over (n)} is a normal vector, and {right arrow over (s)} is a shift vector;

erasing the existing text string by replacing text pixels with background-colored pixels; and

inserting a personalized text string into the location of the erased text according to the calculated 3D curved surface geometry and camera geometry.

11. The system according to claim 10, wherein the user input comprises a plurality of points on the image selected by the user using a graphical user interface (GUI), and the instructions further comprising defining the bounding polygon as a function of the user-selected points.

12. The system according to claim 10, the instructions further comprising segmenting the existing text string according to at least one of pixel color and pixel luminescence.

13. The system according to claim 10, wherein the curved surface is cylindrical.

14. The system according to claim 10, the instructions further comprising building a cost function that describes a degree to which an output ellipse matches a measured ellipse in the image.

15. The system according to claim 14, wherein:

the cost function C comprises a first component C₁and a second component C₂;

wherein the first component is determined as a function of;

a distance between center points of the output ellipse and the measured ellipse;

a difference between the lengths of semi-minor axes of the output ellipse and the measured ellipse;

a difference between the lengths of semi-major axes of the output ellipse and the measured ellipse; and

an angle between the semi major axes of the output ellipse and the measured ellipse; and

wherein the second component is determined as a function of distances between endpoints of edge lines bounding the edges of the curved surface.

16. A computer-implemented method for placing personalized text onto a curved surface in an image, comprising:

bounding a region on a curved surface in the image;

segmenting an existing text string in the bounded region;

detecting upper and lower extreme edge pixels of text characters represented by connected components in the bounded region;

identifying an edge at each side of the curved surface;

via a processor, calculating 3D curved surface geometry and camera geometry by generating a 3D central projection pinhole camera model defined as;

[E,l_L,l_R]=M(f,{right arrow over (n)},{right arrow over (s)})

where E is a projected ellipse, l_Lis a left edge of the curved surface, l_Ris a right edge of the curved surface, M is the model, f is a focal length from a point of origin to an image plane, {right arrow over (n)} is a normal vector, and {right arrow over (s)} is a shift vector;

replacing existing text pixels in the bounded region with background-colored pixels; and

inserting a personalized text string into the bounded region according to the calculated 3D curved surface geometry and camera geometry to personalize the image, and outputting the personalized image.

17. The method according to claim 16, wherein the curved surface is cylindrical.