Method of automatically producing sketches and cartoon images from movies

• US 6,226,015 B1
• Filed: 02/25/1998
• Issued: 05/01/2001
• Est. Priority Date: 02/25/1998
• Status: Expired due to Term

• Current Assignee: Hanger Solutions, LLC (IP Investments Group LLC)
• Original Assignee: Intel Corporation
• Inventors: Howes, Jason J., Danneels, Gunner D., Nee, Peter A.
• Primary Examiner(s): Brier, Jeffery
• Assistant Examiner(s): GOOD JOHNSON, MOTILEWA
• Application Number:
  US09/030,648
• Time in Patent Office: 1,161 Days
• Field of Search: 345/473
• US Class Current: 345/473
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/02 Non-photorealistic rendering
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation

Abstract

Sketch and cartoon images are automatically generated from video or still images captured in real-time from a digital camera or obtained from a stored file. Steps of the method include calculating luma values for each pixel in a captured or retrieved image, applying noise reduction techniques to the captured or retrieved image to improve its quality, performing an adaptive luma chroma difference computation to find and mark edges of objects in a captured or retrieved image as lines forming objects in a sketch image, and applying noise reduction to the sketch image. The process may be repeated for each frame in a video sequence. Temporal smoothing is applied to the video sequence of frames. Generated sketch images may be colored to produce cartoon images.

Claims

1. A method of generating a sketch image comprising:

performing an adaptive luma chroma difference calculation to find and mark edges of objects in an input image as lines forming objects in the sketch image.

2. The method of claim 1, further comprising accepting the input image from a digital camera.

3. The method of claim 1, further comprising accepting the input image from a storage medium.

4. The method of claim 1, further comprising repeating the performing step for a plurality of input images in sequence to form a stream of sketch images.

5. The method of claim 4, further comprising displaying the stream of sketch images on a display in real-time.

6. The method of claim 1, further comprising coloring the objects of the sketch image to produce a cartoon image.

7. The method of claim 6, further comprising repeating the performing and coloring steps for a plurality of input images in sequence to form a stream of cartoon images.

8. The method of claim 7, further comprising displaying the stream of cartoon images on a display in real-time.

9. A system for generating a sketch image comprising:

a device to obtain a first image; and

a sketch generator coupled to the device to produce a sketch image based on the first image by performing an adaptive luma chroma difference calculation to find and mark edges of objects in the input image as lines forming objects in the sketch image.

10. The system of claim 9, wherein the device is a digital camera.

11. The system of claim 9, further comprising a noise reducer coupled to the sketch generator to reduce noise in the sketch image.

12. The system of claim 11, further comprising a cartoon generator coupled to the noise reducer to generate a cartoon image from the sketch image.

13. A system for generating a sketch image comprising:

means for obtaining a first image; and

means for performing an adaptive luma chroma difference calculation to find and mark edges of objects in the first image as lines forming objects in the sketch image.

14. The system of claim 13, further comprising means for reducing noise in the sketch image.

15. The system of claim 13, further comprising means for generating a cartoon image from the sketch image.

16. An apparatus for generating a sketch image comprising:

a processing unit for executing programming instructions; and

a storage medium having stored therein a plurality of programming instructions to be executed by the processing unit, wherein when executed, the plurality of programming instructions perform an adaptive luma chroma difference calculation to find and mark edges of objects in a first image as lines forming objects in the sketch image.

17. The apparatus of claim 16, wherein the plurality of programming instructions further comprise instructions to generate a cartoon image from the sketch image.

18. The apparatus of claim 16, wherein the plurality of programming instructions further comprise instructions to reduce noise in the first image.

19. The apparatus of claim 16, wherein the plurality of programming instructions further comprise instructions to reduce noise in the sketch image.

20. A machine readable storage medium having stored therein a plurality of machine readable instructions executable by a processor, the machine readable instructions comprising instructions to perform an adaptive luma chroma difference calculation to find and mark edges of objects in a first image as lines forming objects in a sketch image.

21. The machine readable storage medium of claim 20, wherein the instructions further comprise instructions to generate a cartoon image from the sketch image.

22. The machine readable storage medium of claim 20, wherein the instructions further comprise instructions to reduce noise in the first image.

23. The machine readable storage medium of claim 20, wherein the instructions further comprise instructions to reduce noise in the sketch image.

24. A method for automatically generating a sketch image comprising:

determining a luma value for each pixel in a first image;

determining a luma change threshold for each pixel in the first image;

marking edge pixels in the sketch image based on the luma values and luma change thresholds and coloring the edge pixels a first color and

applying noise reduction to each pixel of the sketch image by an in-place weighted average of neighboring pixels.

25. The method of claim 24, further comprising creating, before the determining steps, the sketch image with all pixels colored a second color.

26. The method of claim 24, further comprising translating the first image to a first format having luma values when the first image is in a second format that does not have luma values.

27. The method of claim 24, further comprising capturing the first image by a digital camera.

28. The method of claim 24, further comprising accepting the first image from a storage medium.

29. The method of claim 24, further comprising coloring the objects of the sketch image to produce a cartoon image.

30. The method of claim 29, further comprising repeating the determining, marking and coloring steps for a plurality of first images in sequence to form a stream of cartoon images.

31. The method of claim 30, further comprising displaying the stream of cartoon images on a display in real-time.

32. The method of claim 24, further comprising repeating the determining and marking steps for a plurality of first images in sequence to form a stream of sketch images.

33. The method of claim 32, further comprising displaying the stream of sketch images on a display in real-time.

34. The method of claim 32, further comprising applying temporal smoothing to the stream of sketch images.

35. An apparatus for automatically generating a sketch image comprising:

a processing unit for executing programming instructions; and

a storage medium having stored therein a plurality of programming instructions to be executed by the processing unit, wherein when executed, the plurality of programming instructions determine a luma value for each pixel in a first image, determine a luma change threshold for each pixel in the first image, mark edge pixels in the sketch image based on the luma values and luma change thresholds, apply noise reduction to each pixel of the sketch image by an in-place weighted average of neighboring pixels, and color the edge pixels a first color.

36. A machine readable storage medium having stored therein a plurality of machine readable instructions executable by a processor, the machine readable instructions comprising instructions to determine a luma value for each pixel in a first image, determine a luma change threshold for each pixel in the first image, mark edge pixels in the sketch image based on the luma values and luma change thresholds, apply noise reduction to each pixel of the sketch image by an in-place weighted average of neighboring pixels, and color the edge pixels a first color.