Video image creation system which proportionally mixes previously created image pixel data with currently created data

• US 5,216,755 A
• Filed: 06/20/1991
• Issued: 06/01/1993
• Est. Priority Date: 12/04/1980
• Status: Expired due to Term

• Current Assignee: Quantel Limited (Belden Inc.)
• Original Assignee: Quantel Limited (Belden Inc.)
• Inventors: Taylor, Richard J., Walker, Ian C., Searby, Anthony D., Kellar, Paul R. N.
• Primary Examiner(s): Shaw, Dale M.
• Assistant Examiner(s): Brutman, Laura
• Application Number:
  US07/723,728
• Time in Patent Office: 712 Days
• Field of Search: 364/521, 178/18, 358/22, 358/903, 340/703, 340/706, 340/707, 340/708, 340/710, 340/725, 340/728, 340/751, 340/798, 340/521, 395/132
• US Class Current: 345/591
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory

Abstract

Video image creation in which a raster-scan monitor displays an image comprising realistic, inherently non-aliased strokes created by combining proportions of the values of: (i) pixels defining a simulated brush moving along the image as an artist moves a pen on a tablet in continuous strokes; and (ii) the values of the pixels in the image that are under the brush at the time. The pixel values resulting from such combining replace the corresponding pixel values in the image so that, after an initial portion of a stroke, the pixel values in the image which are under the brush and therefore will be combined with the brush values comprise pixel values which are the cumulative result of a number of previous such combining operations. Manual pressure on the pen controls the proportions used in such combining operations. As the artist presses harder on the pen, the effect seen on the monitor is that more "paint" is applied. Image creation takes place substantially in real time, to retain the feel of conventional drawing or painting while providing the benefits of the electronic medium. The brush position on the image can be defined to sub-pixel resolution to improve the process. In an air brush mode, dwell time rather than movement of the pen initiates new pixel value combining operations.

Claims

1. A video image creation system comprising:

manually operable means for designating variable coordinate locations under manual control;

means for providing current image data pertaining to a patch of picture points allocated to a designated coordinate location; and

processing means for processing the image for each picture point allocated to a designated coordinate location from both current and previously derived image data to derive new image data for the respective picture point comprising proportions of said current and said previously derived image data;

generator means for generating a parameter which determines the proportions of current and previously derived image data processed by said processing means and which is varied on a picture point by picture point basis so that the proportion of current image data relative to previously derived image data at each picture point in the patch corresponds to the projection on said picture point of the appropriate portion of a continuous three dimensional shape representing the distribution around the center of a notional artist'"'"'s implement the placement of which is determined by said designated coordinate location, the said shape having a high central region and decreasing from said region to the edge of the shape.

2. A system as in claim 1 in which said high central region of said continuous three dimensional shape is flat and said parameter is varied on a picture point by picture point basis only at a peripheral portion of said patch.

3. A system as in claim 1 including a source of a blend control signal, wherein said brush processor includes a subsystem responsive to said blend control signal to blend said brush data and said image data in proportions related to said blend control signal.

4. A system as in claim 3, including frame storage means for storing previously derived data for use by the processing means and for storing new data following processing by the processing means.

5. A system as in claim 4, including monitoring means for observing the image creation via the frame storage means.

6. A system as in claim 5, wherein said manually operable means designates coordinate locations to a higher resolution than the resolution of picture points, and wherein the generator means is adapted to provide a parameter variable on a picture point by picture point basis for picture points in the patch depending on the placement of the notional artist'"'"'s implement as determined by the designated coordinate location with reference to said picture points to an accuracy of a fractional part of a picture point.

7. A system as in claim 6, wherein the generator means includes a memory loaded with said parameter values, and including addressing means for addressing said memory to read said parameter values therefrom.

8. A system as in claim 7, wherein the addressing means is adapted to address a number of storage locations corresponding to the number of picture points allocated to a designated coordinate location.

9. A video image creation system comprising manually operable means for designating coordinate locations under manual control to allow creation of the image to be affected substantially in real time, means for providing current image data pertaining to at least one picture point allocated to a designated coordinate location and processing means for processing the image for each picture point allocated to a designated coordinate location from both current and previously derived image data, and generator means for generating a parameter for determining the proportions of current and previously derived image data processed by said processing means, wherein the generator means is adapted to provide a parameter dependent on manual pressure applied to an artist'"'"'s implement related to said manually operable means to vary the proportion of current data used for processing.

10. A system as in claim 9, wherein the processing means comprises an arithmetic processor.

11. A system as in claim 10, wherein the arithmetic processor includes at least one adder, subtractor and multiplier.

12. A system as in claim 11, wherein the means for providing current image data is adapted to output data pertaining to picture intensity.

13. A system as in claim 12, wherein the means for providing current image data is adapted to output data pertaining to picture color.

14. A system as in claim 13, wherein the means for providing current image data comprises a plurality of memories at least some of which have an output dependent on image color.

15. A system as in claim 14, wherein separate processing means are provided for each of the current image data memories.

16. A system as in claim 15, wherein separate frame storage means are provided for said previously derived image data respectively cooperating with each of the processing means.

17. A system as in claim 16, wherein printing means are provided for providing a hard copy of the created image.

18. A method comprising:

designating variable coordinate locations under manual control;

providing current image data from digital computing circuits for a number of picture points forming a patch of picture points allocated to a designated coordinate location;

using digital computing circuits to process the image for each picture point allocated to a designated coordinate location from both current and previously derived image data to derive new image data for the respective picture point comprising proportions of said current and said previously derived image data; and

using digital computing circuits to generate a parameter which determines the proportions of current and previously derived image data processed by said processing step and which is varied on a picture point by picture point basis for the picture points forming at least a peripheral part of said patch so that the proportion of current image data relative to previously derived image data at picture points forming the patch corresponds to the relative distribution of a notional artist'"'"'s implement the placement of which is determined by said designated coordinate location, said distribution being high in a central region of said patch and progressively decreasing from said central region to the edge of the patch.

19. A method as in claim 18, in which said step of providing variable coordinate locations under manual control comprises providing said coordinate locations to allow creation of the image to be affected substantially in real time wherein, for simulating brushes and pencils having no temporal nature, said manual control step is such that if a manually operable means is held on a location no additional processing occurs with time after the processing initiated upon the a designation of that location and, for simulating air brush, said manual control step is such that if said manually operable means is held on location a train of processing operations is initiated.

20. A method as in claim 18, including using at least one frame store to store previously derived data for use by the processing step and to store new data following processing by the processing step to thereby facilitate the creation of an image by said processing step.

21. A method as in claim 18, wherein the said step of designating coordinate locations designates coordinate locations to a higher resolution than the resolution of picture points, and wherein the generating step is adapted to provide a parameter variable on a picture point by picture point basis for picture points forming the patch depending on the placement of the notional artist'"'"'s implement as determined by the designated coordinate location with reference to said picture points to an accuracy of a fractional part of a picture point.

22. A method as in claim 18 in which said parameter is substantially constant in said high central region.

23. A system comprising:

a store for image data defining an image;

an operator controlled device for drawing a continuous stroke designating a succession of positions defining a stroke in said image;

a source of brush data; and

a brush processor responsive to each designation of a position in the image to blend the brush data with image data from the store related to the designated position and to store the blend in place of the image data used in the blend;

wherein the image data used in a blend in response to a given designated position comprise image data that resulted from a blend in response to previously designated positions in the stroke.

24. A system as in claim 23 including a source of a blend control signal, wherein said brush processor includes a subsystem responsive to said blend control signal to blend said brush data and said image data in proportions related to said blend control signal.

25. A system as in claim 24, in which said image data stored in said stored in said store comprise pixel data, said brush processor blends the brush data with pixel data from pixels within a circular patch of said image, the location of which is determined by the designated position, and said blend control signal comprises respective blend control values for the respective pixels in said patch.

26. A system as in claim 24 in which said brush processor is selectively operable in a first mode in which the processor blends only in response to a new designation which is spaced in distance along the stroke from the previous designation or in a second mode in which the processor blends in response to a new designation which is spaced in time from the previous designation, whether of not it is spaced also in distance.

27. A system as claimed in claim 26, in which said image data stored in said store comprise pixel data, said brush processor blends the brush data with pixel data from pixels within a circular patch of said image, the location of which is determined by the designated position, and said blend control signal comprises respective blend control values for the respective pixels in said patch.

28. A system as in claim 27, in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

29. A system as in claim 27 in which said image data stored in said store comprise pixel data and said device for designating image positions comprises a device for designating image positions to a spatial resolution greater than the pixel resolution of said pixel data such that a designated position comprises a sub-pixel fraction, and selecting blend control signals in dependence upon the sub-pixel fraction comprised in the designated position.

30. A system as in claim 29 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

31. A system as in claim 26 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

32. A system as in claim 24, in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

33. A system as in claim 32 in which said image data stored in said store comprise pixel data and said device for designating image positions comprises a device for designating image positions to a spatial resolution greater than the pixel resolution of said pixel data such that a designated position comprises a sub-pixel fraction, and selecting blend control signals in dependence upon the sub-pixel fraction comprised in the designated position.

34. A system as in claim 24 in which said source of brush data supplies brush data for picture points which are in a circular area and said source of a blend control signal supplies a blend control signal which varies at least within a peripheral annulus of said circular area.

35. A system as in claim 34 in which the blend control signal varies only within said annulus of said circular area.

36. A system as in claim 23 in which said brush processor is selectively operable in a first mode in which the processor blends only in response to a new designation which is spaced in distance along the image from the previous designation or in a second mode in which the processor blends only in response to a new designation which is spaced in time from the previous designation.

37. A system as in claim 36 in which said image data stored in said store comprise pixel data and said device for designating image positions comprises a device for designating image positions to a spatial resolution greater than the pixel resolution of said pixel data such that a designated position comprises a sub-pixel fraction, and selecting blend control signals in dependence upon the sub-pixel fraction comprised in the designated position.

38. A system as in claim 37 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

39. A system as in claim 36 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

40. A system as in claim 23 in which said image data stored in said store comprise pixel data and said device for designating image positions comprises a device for designating image positions to a spatial resolution greater than the pixel resolution of said pixel data such that a designated position comprises a sub-pixel fraction, and selecting blend control signals in dependence upon the sub-pixel fraction comprised in the designated position.

41. A system as in claim 40 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

42. A system as in claim 23 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

43. A system as in claim 23 in which said device for designating positions in said image comprises a pen and a tablet.

44. A system as in claim 43 in which said pen is pressure sensitive and for each designated position produces a respective blend control signal having a value profile related to manual pressure exerted on the pen at the time, and said brush processor comprises a subsystem responsive to the respective blend control signals for the designated positions to blend said brush data and image data in proportions related to the blend control signal.

45. A system as in claim 23 including a source of a blend control signal comprising a pressure sensitive pen which for each designated position produces a respective blend control signal having a parameter varying as a function of the pressure applied to the pen, said blend control signal being supplied to said brush processor to control the proportions of brush data and image data used to form said blend.

46. A system as in claim 45 in which said image data comprises image data for respective pixels of an image, for each designated position said brush data comprises brush data for a respective patch of said image, and said source of a blend control signal comprises a source of respective blend control values for the pixels of the respective patch of said image, wherein for at least some of the pixels in said patch said blend control values vary from one designated position to another in response to variations in pressure on the pen.

47. A system comprising:

a computer-implemented source of brush data;

a computer-implemented source of image data;

a source of a blend control signal which has a selected parameter derived in response to manual pressure; and

a brush processor blending said brush data with said image data in proportions related to said blend control signal.

48. A system as in claim 47 including an operator controlled device for designating a succession of image positions, and in which said image data define an image and said brush processor blends said brush data with image data from image positions related to image positions designated by said device and said brush processor is selectively operable in a first mode in which the brush processor blends said brush data with image data only in response to a new designation which is spaced in distance along the image from the previous designation or in a second mode in which the processor blends in response to a new designation which is spaced either in distance or in time from the previous designation.

49. A system as in claim 48 in which said source of image data comprises a store which stores the image data in the form of pixel data and stores the data resulting from said blending in place of the image data used in the blending and said device for designating image positions comprises a device for designating image positions to a spatial resolution which is greater than the pixel resolution of said pixel data.

50. A system as in claim 49 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

51. A system as in claim 48 in which said device designates positions which comprise positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the results of the blending substantially in real time.

52. A system as in claim 47 including a device for designating a succession of image positions, wherein said image data define an image and said brush processor blends said brush data with image data from image positions related to image positions designated by said device and said source of image data comprises a store which stores the image data in the form of pixel data and stores the data resulting from said blending in place of the image data used in the blending and said device for designating image positions comprises a device for designating image positions to a spatial resolution greater than the pixel resolution of said pixel data.

53. A system as in claim 52 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

54. A system as in claim 47 in which said device designates positions which comprise positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to allow for a display of the results of the blending substantially in real time.

55. A system as in claim 47 in which said source of brush data produces brush data for a first circular area and said source of a blend control signal produces a blend control signal varying such that the proportion of the brush data blended with the image data decreases in moving toward the circumference of the first circular area from a second circular area which is concentric with and smaller that said first circular area.

56. A system comprising:

a store for image data defining an image;

an operator controlled device for designating a succession of image positions;

a source of brush data; and

a brush processor responsive to each designation to blend the brush data with image data from the store related to the designated position;

wherein said brush processor is selectively operable in a first mode in which the processor blends only in response to a new designation which is spaced in distance along the image from the previous designation or in a second mode in which the processor blends in response to a new designation which is spaced in time from the previous designation, whether or not it is spaced also in distance.

57. A system as in claim 56 in which said image data stored in said store comprise pixel data and said device for designating image positions comprises a device for designating image positions to a spatial resolution greater than the pixel resolution of said pixel data such that a designated position comprises a sub-pixel fraction, and selecting blend control signals in dependence upon the sub-pixel fraction comprised in the designated position.

58. A system as in claim 57 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to store data resulting from said blending in the store in place of the image data used in the blending substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

59. A system as in claim 56 in which said device designates positions at pixel spacing in an artist designated stroke, and said brush processor operates substantially in real time to store data resulting from said blending in the store in place of the image data used in the blending substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending substantially in real time.

60. A system comprising:

a store for pixel data defining an image;

a device for designating a succession of image positions to a spatial resolution greater than the pixel resolution, wherein a designated position can comprise a sub-pixel fraction;

a source of brush data; and

a brush processor responsive to each designation of an image position to blend the brush data with pixel data from the store related to the designated image position and any sub-pixel fraction of the designated position, and to store the blend in the store in place of the pixel data used in the blend.

61. A system as in claim 60 in which said brush processor operates substantially in real time to allow for a display of the image data from the store in a manner showing the results of the blending at a rate which gives the appearance that the display simultaneously shows a stroke painted across the image by an artist using said device for designating.

62. A system comprising:

a store for image data defining an image;

a device for designating a succession of image positions;

source of brush data; and

a processor responsive to each designation to blend the brush data with image data from the store related to the designated position and to store the blend in the store in place of the image data used in the blend;

said processor operating substantially in real time to allow for a display of the image data from the store data in a manner showing the results of the blending at a rate which gives the appearance that the display instantaneously shows a stroke painted across the image by an artist using said device for designating.

63. An image creation system comprising:

a pressure sensitive pen and a tablet generating position information identifying positions which the pen assumes in a continuous stroke on the tablet and pressure information related to pressure on the pen during the stroke;

a source of brush data comprising pixel values for pixels in a patch in which the pixel values rise at least over a selected distance inwardly from the periphery of the patch;

an image store comprising storage for image data comprising pixel values of pixels defining an image;

a processor receiving said position and pressure information and, in response to successive pen positions identified by the position information, carrying out a corresponding succession of processing operations in each of which the processor retrieves from the store image data for pixels at positions in the image identified by the position information, processes the retrieved image data with brush data to combine the image and brush data into processed image data in proportions which depend on the pressure information and stores the processed image data in said store in place of the image data used in the processing operation;

wherein each processing operation after a starting portion of the stroke comprises processing image data resulting from a number of previous processing operation during the stroke; and

a raster-scan monitor which receives image data from said store and displays an image corresponding to the received image data.

64. An image creation process comprising the steps of:

moving a pressure sensitive pen on a tablet and generating position information identifying positions which the pen assumes in a continuous stroke on the tablet and pressure information related to a pressure on the pen during the stroke;

providing brush data comprising pixel values for pixels in a patch in which the pixel values rise at least over a selected distance inwardly from the periphery of the patch;

providing an image store comprising storage for image data comprising pixel values of pixels defining an image;

in response to successive pen positions identified by the position information, carrying out a corresponding succession of processing operations in each of which the processor retrieves from the store image data for pixels at positions in the image identified by the position information, processes the retrieved image data with brush data to combine the image and brush data into processed image data in proportions which depend on the pressure information and stores the processed image data in said store in place of the image data used in the processing operation;

wherein each processing operation after an initial portion of the stroke comprises processing image data resulting from a number of previous processing operation during the stroke; and

displaying on a raster-scan monitor an image derived from image data in said store.