Systems and methods for image or video personalization with selectable effects

• US 9,058,757 B2
• Filed: 08/13/2012
• Issued: 06/16/2015
• Est. Priority Date: 08/13/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Bala, Raja, Ding, Hengzhou, Furst, Michael R., Sherwin, Reuven J.
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US13/584,159
• Publication Number:
  US 20140043363A1
• Time in Patent Office: 1,037 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Embodiments relate to systems and methods for image or video personalization with selectable effects. Image data, which can include video sequences or digital still images, can be received in a graphical personalization tool to perform various image processing and related operations to insert personalized objects into the image data. In aspects, the personalized object(s) can be or include graphical inputs such as, for instance, textual information, graphical information, and/or other visual objects. The graphical personalization tool can automatically perform one or more processing stages in the image path, such as identifying key regions in a still image and/or key frames in a video sequence, in which personalized objects will be generated and inserted. Personalized objects can be extended to additional regions of a still image, can be animated across multiple still images, and/or can be extended to additional frames of a video sequence, all on an automated or user-assisted basis.

Claims

1. A method of producing personalized image data for rendering on a display device, comprising:

accessing image data from an image source, wherein the image data comprises a sequence of image frames;

automatically identifying a key image frame of the sequence of image frames, based on smooth region finding operations, detected regions of a specific texture, detected planar surfaces, detected text, or detected specific motion characters;

receiving, via user input, at least one personalized object for insertion into the image data;

inserting the at least one personalized object into the key image frame to produce personalized output image data wherein at least one of the original image data and the personalized object exhibits motion;

extending the incorporation of the at least one personalized object to additional image frames of the image data;

interpolating motion vectors to determine the location and appearance of the at least one personalized object for the additional image frames;

subsequent to interpolating the motion vectors, querying a user to identify any errors in the location or the appearance of the at least one personalized object for the additional image frames;

in response to the querying, receiving user correction input to adjust the location or the appearance of the at least one personalized object for the additional image frames; and

interpolating motion vectors to determine the location and appearance of the at least one personalized object for the additional image frames based on the user correction input.

2. The method of claim 1, wherein the at least one personalized object comprises at least one of image, graphics, text, or video.

3. The method of claim 1, wherein receiving the at least one personalized object comprises performing an image adjustment operation on the at least one personalized object to produce a desired realistic effect.

4. The method of claim 3, wherein the image adjustment operation comprises at least one of performing a perspective adjustment, a contrast adjustment, or a sizing adjustment of the at least one personalized object.

5. The method of claim 1, wherein receiving the personalized object comprises receiving user correction input to adjust the at least one personalized object.

6. The method of claim 5, wherein the adjustment is made to at least one of perspective, location, size, contrast, or color of the personalized object.

7. The method of claim 1, further comprising receiving a set of exemplar personalized objects to present to the user for selecting the at least one personalized object.

8. The method of claim 1, wherein the display device comprises at least one of a desktop computer, laptop computer, smartphone, or tablet.

9. The method of claim 1, wherein identifying the key image frame of the sequence of image frames is further based on edge detection operations.

10. The method of claim 1, wherein querying the user to identify any errors in the location or the appearance of the at least one personalized object for the additional image frames comprises presenting a graphical interface where the user can view the personalized output image data and identify specific frames in which the location or the appearance of the at least one personalized object is incorrect.

11. The method of claim 10, wherein the graphical interface allows the user to drag corners of the at least one personalized object to give the text a correct perspective on a planar surface.

12. A system, comprising:

an interface to an image source providing image data; and

a processor, communicating with the image source via the interface, the processor being configured to—

access the image data from the image source, wherein the image data comprises a sequence of image frames,

automatically identify a key image frame of the sequence of image frames, based on smooth region finding operations, detected regions of a specific texture, detected planar surfaces, detected text, or detected specific motion characters,

receive, via user input, at least one personalized object for insertion into the image data,

insert the at least one personalized object into the key image frame to produce personalized output image data wherein at least one of the original image data and the personalized object exhibits motion,

extending the incorporation of the at least one personalized object to additional image frames of the image data;

interpolating motion vectors to determine the location and appearance of the at least one personalized object for the additional image frames;

subsequent to interpolating the motion vectors, querying a user to identify any errors in the location or the appearance of the at least one personalized object for the additional image frames;

in response to the querying, receiving user correction input to adjust the location or the appearance of the at least one personalized object for the additional image frames; and

interpolating motion vectors to determine the location and appearance of the at least one personalized object for the additional image frames based on the user correction input.

13. The system of claim 12, wherein the at least one personalized object comprises at least one of graphics, text, or video.

14. The system of claim 12, wherein receiving the at least one personalized object comprises performing an image adjustment operation on the at least one personalized object to produce a desired realistic effect.

15. The system of claim 14, wherein the image adjustment operation comprises at least one of performing a perspective adjustment, a contrast adjustment, or a sizing adjustment of the at least one personalized object.

16. The system of claim 12, wherein receiving the personalized object comprises receiving user correction input to adjust the at least one personalized object.

17. The system of claim 16, wherein the adjustment is made to at least one of perspective, location, size, contrast, or color of the personalized object.

18. The system of claim 12, wherein the processor is further configured to receive a set of exemplar personalized objects to present for user selection as the at least one personalized object.

19. The system of claim 12, further comprising a display device for display of the personalized output image data.

20. The system of claim 19, wherein the display device comprises at least one of a desktop computer, laptop computer, smartphone, or tablet.

21. The system of claim 12, wherein identifying the key image frame of the sequence of image frames is further based on edge detection operations.

22. The system of claim 12, wherein querying the user to identify any errors in the location or the appearance of the at least one personalized object for the additional image frames comprises presenting a graphical interface where the user can view the personalized output image data and identify specific frames in which the location or the appearance of the at least one personalized object is incorrect.

23. The system of claim 22, wherein the graphical interface allows the user to drag corners of the at least one personalized object to give the text a correct perspective on a planar surface.