Image identification apparatus and method of identifying images

• US 6,744,915 B1
• Filed: 09/08/2000
• Issued: 06/01/2004
• Est. Priority Date: 09/09/1999
• Status: Active Grant

• Current Assignee: Sony Europe (Amsterdam) BV (Sony Group Corp.)
• Original Assignee: Sony United Kingdom Limited (Sony Group Corp.)
• Inventors: Barton, Mark, Thorpe, Jonathan
• Primary Examiner(s): Mehta, Bhaves M.
• Assistant Examiner(s): DESIRE, GREGORY M
• Application Number:
  US09/657,756
• Time in Patent Office: 1,362 Days
• Field of Search: 382/119-122, 382/156-158, 382/179, 382/181, 382/186-190, 382/203
• US Class Current: 382/161
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

An image identification apparatus for identifying an image from a sequence of spatial samples generated in response to a hand drawn representation of the image, the image identification apparatus comprising a segment processor arranged in operation to determine from the spatial samples stroke data representative of strokes performed in drawing the hand drawn representation of the image, a stroke pre-processor which is arranged in operation to generate parameter data from the sequence of spatial samples and the determined stroke data of the hand drawn representation, and an image identifier coupled to the stroke pre-processor which is arranged in operation to identity the image from the parameter data. The image identification apparatus finds application in many fields, including identifying hand drawn images for generating a story board from pre-generated images.

Claims

1. An image identification apparatus for identifying an image from a sequence of spatial samples generated in response to a hand drawn representation of said image, said image identification apparatus comprising:

a drawing processor having a drawing tool for use in creating said hand drawn representation of said image, wherein said sequence of spatial samples is generated in accordance with movement of said drawing tool,

a segment processor arranged in operation to determine from said spatial samples stroke data representative of strokes performed in drawing said hand drawn representation of said image,

a stroke pre-processor which is arranged in operation to generate parameter data from said sequence of spatial samples and the determined stroke data of the hand drawn representation, and

an image identifier coupled to said stroke pre-processor which is arranged in operation to identify said image from the parameter data,

wherein said segment processor operates to determine said stroke data by

generating from said spatial samples relative direction information representative of a relative direction of said drawing tool,

determining from said relative direction information a number of changes of direction of said tool, and

determining said strokes of the hand drawn representation in accordance with the number of changes of direction.

2. An image identification apparatus as claimed in claim 1, wherein said segment processor operates to

generate from said relative direction information a rate of change of direction of said tool, and

determine a number of peaks in the rate of change of direct ion of the toot, said number of peaks being indicative of said number of changes of direction.

3. An image identification apparatus as claimed in claim 1 wherein said segment processor has a filter through which said relative direction information is passed, said number of times said tool changes direction being determined from said filtered relative direction information.

4. An image identification apparatus as claimed in claim 1 wherein said sequence of spatial samples are provided with a temporal reference, and said segment processor operates to determine said stroke data by

generating from said temporally referenced spatial samples relative speed information from a number of said samples representative of a relative speed of said drawing tool with respect to time,

determining from said relative speed information a number of changes of speed of said tool changes, and

determining said strokes of the hand drawn representation in accordance with the number of changes of speed.

5. An image identification apparatus as claimed in claim 4, wherein said segment processor operates to

generate from said relative speed information a rate of change of the speed of said tool,

identify when the rate of change of speed reaches zero, said number of times the drawing tool changes speed being determined from which of the identified zeros in the rate of change of speed corresponds to minimum speeds.

6. An image identification apparatus as claimed in claim 4, wherein said segment processor has a filter through which said relative speed information is passed, said number of times said tool changes speed being determined from said filtered relative speed information.

7. An image identification apparatus as claimed in claim 1, wherein said segment processor operates to determine the strokes performed in drawing said hand drawn representation by combining said relative direction information and said relative speed information.

8. An image identification apparatus as claimed in claim 1 wherein said stroke pre-processor operates to

determine a start and end point for each of said stokes,

form an outer boundary of said hand drawn representation from said start and end points of said strokes,

normalise said hand drawn representation from said outer boundary with respect to a pre-defined reference unit by proportionally adapting said spatial samples, and

generate said parameter data from said normalised hand drawn representation.

9. An image identification apparatus as claimed in claim 8, wherein said parameter data includes at least one of a number of said strokes, a centre of gravity of said hand drawn representation, a mean vector of said hand drawn representation, a normalised vector of said hand drawn representation, the length of said hand drawn representation, and the balance of said hand drawn representation.

10. An image identification apparatus as claimed in claim 1 wherein said image identifier is arranged to access a data store storing a plurality of images, and to indicate which of said plurality of images corresponds to said hand drawn representation in accordance with said parameter data.

11. An image identification apparatus as claimed in claim 10, wherein said image identifier is a neural network which is trained with said parameter data corresponding to said pre-stored images, and is arranged in operation to generate a reference indication representative of which of said pre-stored images said parameter data corresponds.

12. An image identification apparatus as claimed in claim 11 wherein said neural network is a multi-layer perceptron, the number of inputs of an input layer corresponding to the number of items of parameter data, each of said items of parameter data being fed to a corresponding one of said inputs.

13. An image identification apparatus as claimed in claim 1 wherein said image identifier operates to feed data representative of said indicated image to a visual display means connectable to said apparatus to provide a representation of said image.

14. A method of identifying an image from a sequence of spatial samples generated in response to a hand drawn representation of said image, said method comprising the steps of

resolving, from said samples, strokes performed in creating said hand drawn representation of said image,

generating parameter data by associating said spatial samples with the resolved strokes of the hand drawn representation, and

identifying said image from stroke parameter data,

wherein the step of resolving said strokes performed in drawing said hand drawn representation, comprises the steps of

generating from said spatial samples relative direction information representative of a relative direction of a tool used to draw said hand drawn representation, and

determining from said relative direction information a number of times said tool changes direction, the strokes performed in generating said hand drawn representation being resolved in accordance with the number of changes of direction.

15. A method of identifying an image as claimed in claim 14, wherein the step of determining trials said relative direction information a number of times and tool changes direction, comprises the steps of

generating from said relative direction information a rate of change of direction of said tool, and the step of determining the number of tines the drawing tool changes direction comprises the step of

determining a number of peaks in the rate of change of direction of the tool.

16. A method of identifying an image as claimed in claim 14, wherein the step of resolving the strokes performed in drawing said hand drawn representation, comprises the step of

filtering said relative direction information.

17. A method of identifying an image as claimed in claim 14, wherein spatial samples are provided with a temporal reference, and the step of resolving the strokes performed in drawing said hand drawn representation, comprises the step of

generating from said temporally referenced spatial samples relative speed information representative of a relative speed of a tool used to draw said hand drawn representation with respect to time,

determining from said relative speed information a number of changes of speed of said tool, the strokes performed in generating said hand drawn representation being resolved in accordance with the number of changes of speed.

18. A method of identifying an image as claimed in claim 17, wherein the step of determining from said relative direction information a number of changes of speed of said tool, comprises the steps of

generating from said relative speed information a rate of change of the speed of said tool, and

identifying a number of points at which the rate of change of speed reaches zero, and the step of determining the number of times the drawing tool changes speed comprises the step of determining which of the identified zero points correspond to minimum speeds.

19. A method of identifying an image as claimed in claim 17 wherein the step of resolving the strokes performed in drawing said hand drawn representation, comprises, before the step of determining the man her of speed changes the step of

filtering said relative speed information.

20. A method of identifying an image as claimed in claim 14, wherein the step of resolving the strokes performed in drawing said hand drawn representation, comprises the step of

combining the changes speed of said tool and the changes of direction of said tool.

21. A method of identifying an image as claimed in claim 14 wherein the step of generating said parameter data from said spatial samples and the resolved strokes of the hand drawn representation, comprises the steps of

determining start and end points of each of said strokes,

forming an outer boundary from said start and end points of said strokes,

normalising said hand drawn representation from said outer boundary with reference to a pre-defined reference unit by proportionally adapting said spatial samples, and

generating said parameter data from said normalised hand drawn representation.

22. A method of identifying an image as claimed in claim 21, wherein said parameter data include at least one of said number of strokes, a centre of gravity of said hand drawn representation, a mean vector of said hand drawn representation, a normalised vector of said hand drawn representation, the length of said hand drawn representation, and the balance of said hand drawn representation.

23. A method of identifying an image as claimed in claim 14 wherein the step of identifying said image from the parameter data comprises the steps of

generating a reference indication from said parameter data, and

selecting an image from a plurality of images in accordance with said reference indication.

24. A method of identifying an image as claimed in claim 14 wherein the step of generating said reference indication from said parameter data, comprises the step of

applying the parameter data to a neural network which has been trained to said generate reference indication in accordance with which of said plurality of images most closely corresponds to said hand drawn representation of said image.

25. A method of identifying an image as claimed in claim 24, wherein said neural network is a multi-layer perceptron, the number of inputs of all input layer corresponding to the number of items of parameter data, each of said items of parameter data being fed to a corresponding one of said inputs.

26. A method of identifying an image as claimed in claim 14, wherein said image identifier operates to fed data representative of said indicated image to said visual display means which provides a representation of said indicated image.

27. A method of producing a story board representative of a sample production, comprising performing the steps of the method of identifying an image as claimed in claim 14, wherein the pre-stored images correspond to desired images of said production.

28. A computer program arranged to defined the steps of the method of identifying an image as claimed claim 14.

29. A computer readable carrier having stored thereon a computer program which is arranged to perform the steps of the method of identifying an image as claimed in claim 14, when loaded on to a computer.

30. An image identification apparatus for identifying an image from a hand drawn representation of said image, said image identification apparatus comprising

a drawing processor which is arranged in use to produce said hand drawn representation and operates to generate temporally referenced spatial sample, in accordance with the hand drawn representation of said image,

a segment processor coupled to the drawing processor which operates to determine from said temporally referenced spatial samples stroke data representative of strokes performed in drawing said hand drawn representation of said image,

a stroke pre-processor coupled to said segment processor which operates to generate stroke parameter data by associating the temporally referenced spatial samples with the determined stroke data of the hand drawn representation, and

an image identifier coupled to said stroke pre-processor which operates to identity said image from the stroke parameter data,

wherein said segment processor operates to determine said stroke data by

generating from said spatial samples relative direction information representative of a relative direction of a drawing tool used to generate the spatial samples,

determining from said relative direction information a number of changes of direction of said drawing tool, and

determining said strokes of the hand drawn representation in accordance with the number of changes of direction.