Systems and methods for interactive image caricaturing by an electronic device

• US 10,379,734 B2
• Filed: 02/23/2013
• Issued: 08/13/2019
• Est. Priority Date: 02/23/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zou, Xuan, Ling, Fan, Bi, Ning, Zhang, Lei
• Primary Examiner(s): Coleman, Stephen P
• Application Number:
  US14/764,149
• Publication Number:
  US 20160011769A1
• Time in Patent Office: 2,362 Days
• Field of Search: None
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

A method for interactive image caricaturing by an electronic device is described. The method includes detecting at least one feature location of an image. The method further includes generating, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line. The method additionally includes obtaining a gesture input. The method also includes determining at least one caricature action based on the at least one gesture input. The method further includes generating a caricature image based on the image mesh, the at least one caricature action and the image.

Claims

1. A method for interactive image caricaturing by an electronic device, comprising:

detecting at least one feature location of an image;

generating, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component in the image;

obtaining at least one gesture input;

determining at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

generating a caricature image based on the image mesh, the at least one caricature action and the image.

2. The method of claim 1, wherein the image is a face image.

3. The method of claim 1, wherein the at least one gesture input comprises a touch input.

4. The method of claim 1, further comprising displaying at least one of the image and the caricature image.

5. The method of claim 1, wherein generating a caricature image comprises applying the at least one caricature action to the image.

6. The method of claim 5, wherein the at least one caricature action provides shifting information for the at least one vertex of the image mesh and provides the pixel map that maps the image vertex pixel to the corresponding caricature image vertex pixel based on the shifting information, and wherein generating the caricature image comprises interpolating a location of other pixels in the caricature image based on the pixel map.

7. The method of claim 1, further comprising validating the at least one gesture input.

8. The method of claim 7, wherein validating the at least one gesture input comprises determining whether a motion of the gesture input is greater than a motion threshold.

9. The method of claim 1, further comprising maintaining a caricature action list.

10. The method of claim 9, further comprising adding the at least one caricature action to the caricature action list.

11. The method of claim 1, wherein the at least one gesture input comprises multiple gesture inputs.

12. The method of claim 11, further comprising determining a motion of the gesture inputs relative to one another.

13. The method of claim 11, further comprising determining that the gesture inputs are horizontally aligned.

14. The method of claim 11, further comprising determining that the gesture inputs are vertically aligned.

15. An electronic device for interactive image caricaturing, comprising:

feature location detection circuitry configured to detect at least one feature location of an image;

image mesh generation circuitry coupled to the feature location detection circuitry, wherein the image mesh generation circuitry is configured to generate, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component in the image;

sensing circuitry configured to obtain at least one gesture input;

caricature action determination circuitry coupled to the sensing circuitry, wherein the caricature action determination circuitry is configured to determine at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

caricature image generation circuitry coupled to the image mesh generation circuitry and the caricature action determination circuitry, wherein the caricature image generation circuitry is configured to generate a caricature image based on the image mesh, the at least one caricature action and the image.

16. The electronic device of claim 15, wherein the image is a face image.

17. The electronic device of claim 15, wherein the at least one gesture input comprises a touch input.

18. The electronic device of claim 15, further comprising a display that is coupled to the caricature image generation circuitry and configured to display at least one of the image and the caricature image.

19. The electronic device of claim 15, wherein generating a caricature image comprises applying the at least one caricature action to the image.

20. The electronic device of claim 19, wherein the at least one caricature action provides shifting information for the at least one vertex of the image mesh and provides the pixel map that maps the image vertex pixel to the corresponding caricature image vertex pixel based on the shifting information, and wherein generating the caricature image comprises interpolating a location of other pixels in the caricature image based on the pixel map.

21. The electronic device of claim 15, further comprising gesture input validation circuitry coupled to the sensing circuitry, wherein the gesture input validation circuitry is configured to validate the at least one gesture input.

22. The electronic device of claim 21, wherein validating the at least one gesture input comprises determining whether a motion of the gesture input is greater than a motion threshold value.

23. The electronic device of claim 15, wherein the electronic device is configured to maintain a caricature action list.

24. The electronic device of claim 23, wherein the electronic device is configured to add the at least one caricature action to the caricature action list.

25. The electronic device of claim 15, wherein the at least one gesture input comprises multiple gesture inputs.

26. The electronic device of claim 25, further comprising gesture input validation circuitry configured to determine a motion of the gesture inputs relative to one another.

27. The electronic device of claim 25, further comprising gesture input validation circuitry configured to determine that the gesture inputs are horizontally aligned.

28. The electronic device of claim 25, further comprising gesture input validation circuitry configured to determine that the gesture inputs are vertically aligned.

29. A non-transitory tangible computer-readable medium having instructions thereon, the instructions comprising:

code for causing an electronic device to detect at least one feature location of an image;

code for causing the electronic device to generate, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component in the image;

code for causing the electronic device to obtain at least one gesture input;

code for causing the electronic device to determine at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

code for causing the electronic device to generate a caricature image based on the image mesh, the at least one caricature action and the image.

30. The computer-readable medium of claim 29, wherein the image is a face image.

31. The computer-readable medium of claim 29, wherein the at least one gesture input comprises a touch input.

32. The computer-readable medium of claim 29, wherein the instructions further comprise code for causing the electronic device to display at least one of the image and the caricature image.

33. The computer-readable medium of claim 29, wherein the code for causing the electronic device to generate a caricature image comprises code for causing the electronic device to apply the at least one caricature action to the image.

34. The computer-readable medium of claim 29, wherein the instructions further comprise code for causing the electronic device to validate the at least one gesture input.

35. The computer-readable medium of claim 29, wherein the at least one gesture input comprises multiple gesture inputs.

36. The computer-readable medium of claim 29, wherein the instructions further comprise code for causing the electronic device to maintain a caricature action list.

37. The computer-readable medium of claim 36, wherein the instructions further comprise code for causing the electronic device to add the at least one caricature action to the caricature action list.

38. An apparatus for interactive image caricaturing, comprising:

means for detecting at least one feature location of an image;

means for generating, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component of the image;

means for obtaining at least one gesture input;

means for determining at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

means for generating a caricature image based on the image mesh, the at least one caricature action and the image.

39. The apparatus of claim 38, wherein the image is a face image.

40. The apparatus of claim 38, wherein the at least one gesture input comprises a touch input.

41. The apparatus of claim 38, further comprising means for displaying at least one of the image and the caricature image.

42. The apparatus of claim 38, wherein the means for generating a caricature image comprises means for applying the at least one caricature action to the image.

43. The apparatus of claim 38, further comprising means for validating the at least one gesture input.

44. The apparatus of claim 38, wherein the at least one gesture input comprises multiple gesture inputs.

45. The apparatus of claim 38, further comprising means for maintaining a caricature action list.

46. The apparatus of claim 45, further comprising means for adding the at least one caricature action to the caricature action list.