Image recognition/reproduction method and apparatus

• US 6,463,176 B1
• Filed: 07/15/1997
• Issued: 10/08/2002
• Est. Priority Date: 02/02/1994
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Iijima, Katsumi
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US08/893,929
• Time in Patent Office: 1,911 Days
• Field of Search: 382/115, 382/116, 382/118, 382/190, 382/203, 382/215, 382/300, 382/243, 382/241, 382/217-218, 382/113, 382/253, 382/195, 358/400, 358/401, 395/121, 395/118, 395/951, 395/955, 395/135, 345/442, 345/421
• US Class Current: 382/195
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

An image recognition method comprising an extraction step of extracting local feature elements in an input image; a conversion step of arranging the local feature elements extracted at said extraction step on each of cells of predetermined lattice pattern composed of a plurality of cells having predetermined sizes and figures while substantially maintaining relative position relationship between a plurality of extracted local feature elements which are near each other; and a matching step of estimating a resemblance between a plurality of input images by matching the plurality of local feature elements arranged on the lattice pattern at said conversion step.

Claims

1. An image recognition method comprising:

an extraction step of extracting local feature elements in an input image;

a conversion step of assembling the local feature elements extracted at said extraction step and generating array data from the local feature elements on the basis of a predetermined lattice pattern; and

a matching step of estimating a resemblance between the local feature elements array data and a reference plurality of local feature elements array data stored in a memory by computing a differential value and selecting model array data for which the differential value is less than a predetermined threshold value.

2. The method according to claim 1, wherein said extraction step extracts a local feature element with respect to each of a plurality of resolutions of an image.

3. The method according to claim 2, wherein said conversion step includes the steps of:

generating a pair composed of a prescribed local feature element and position information indicative thereof in which the pair composed of a prescribed local feature element and position information indicative thereof is scaled at scaling ratios corresponding to respective ones of the resolutions; and

selecting a pair composed of a scaled prescribed local feature element and position information indicative thereof, this pair being such that the distance between a pair composed of the scaled prescribed local feature element and position information indicative thereof and a pair composed of a local feature element, whose resolution corresponds to the scaling ratio, extracted at said extraction step and position information indicative thereof is less than a prescribed distance.

4. The method according to claim 1, wherein said local feature elements include an element having the shape of an L-type intersection.

5. The method according to claim 1, wherein said local feature elements include an element having the shape of an arc of a prescribed curvature.

6. The method according to claim 1, wherein said local feature elements include an element having the shape of an arrow-type intersection.

7. The method according to claim 1, wherein said local feature elements include an element having the shape of a circle.

8. The method according to claim 1, wherein said local feature elements include an element having the shape of a T-type intersection.

9. The method according to claim 1, wherein said local feature elements include an element which is color.

10. The method according to claim 1, wherein said local feature elements include an element which is spatial frequency.

11. The method according to claim 1, wherein said local feature elements include an element which is luminance.

12. An image reproduction method comprising:

a reproduction step of reproducing an image;

an extracting step of extracting feature elements of a reproduced image and discriminating local feature elements;

a structural information extracting step of extracting structural information of the feature elements corresponding to the local feature elements discriminated at said extracting step including a change in motion of an object of interest; and

a curve generating step of smoothly interpolating between the local feature elements corresponding to the feature elements extracted at said extracting step on the basis of the structural information.

13. The method according to claim 12, wherein said reproduction step further includes a step of combining a prescribed background image and the interpolated image to reproduce an image.

14. The method according to claim 12, wherein said local feature elements include an element having the shape of an L-type intersection.

15. The method according to claim 12, wherein said local feature elements include an element having the shape of an arc of a prescribed curvature.

16. The method according to claim 12, wherein said local feature elements include an element having the shape of an arrow-type intersection.

17. The method according to claim 12, wherein said local feature elements include an element having the shape of a circle.

18. The method according to claim 12, wherein said local feature elements include an element having the shape of a T-type intersection.

19. The method according to claim 12, wherein said local feature elements include an element which is color.

20. The method according to claim 12, wherein said local feature elements include an element which is spatial frequency.

21. The method according to claim 12, wherein said local feature elements include an element which is luminance.

22. An image reproduction method comprising:

an extraction step of extracting a feature element immediately prior to occurrence of an occlusion in which at least part of a feature element of an object is hidden behind another feature element;

a second feature-element generation step of generating a second feature element by using a plurality of feature elements immediately prior to occurrence of the occlusion extracted at said extraction step; and

a step of reproducing an image on the basis of the second feature element generated by said second-feature-element generation step.

23. An image recognition apparatus comprising:

extracting means for extracting local feature elements in an input image;

conversion means for assembling the local feature elements extracted by said extracting means and generating array data from the local feature elements on the basis of a predetermined lattice pattern; and

matching means for estimating a resemblance between the local feature elements array data and a reference plurality of local feature elements array data stored in a memory by computing a differential value and selecting model array data for which the differential value is less than a predetermined threshold value.

24. The apparatus according to claim 23, wherein said extracting means extracts a local feature element with respect to each of a plurality of resolutions of an image.

25. The apparatus according to claim 24, wherein said selecting means:

generates a pair composed of a prescribed local feature element and position information indicative thereof in which the pair composed of a prescribed local feature element and position information indicative thereof is scaled at scaling ratios corresponding to respective ones of the resolutions; and

selects a pair composed of a scaled prescribed local feature element and position information indicative thereof, this pair being such that the distance between a pair composed of the scaled prescribed local feature element and position information indicative thereof and a pair composed of a local feature element, whose resolution corresponds to the scaling ratio, extracted at said extraction step and position information indicative thereof is less than a prescribed distance.

26. The method according to claim 23, wherein said local feature elements include an element having the shape of an L-type intersection.

27. The method according to claim 23, wherein said local feature elements include an element having the shape of an arc of a prescribed curvature.

28. The method according to claim 23, wherein said local feature elements include an element having the shape of an arrow-type intersection.

29. The method according to claim 23, wherein said local feature elements include an element having the shape of a circle.

30. The method according to claim 23, wherein said local feature elements include an element having the shape of a T-type intersection.

31. The method according to claim 23, wherein said local feature elements include an element which is color.

32. The method according to claim 23, wherein said local feature elements include an element which is spatial frequency.

33. The method according to claim 23, wherein said local feature elements include an element which is luminance.

34. An image reproduction apparatus comprising:

reproducing means for reproducing an image;

extracting means for extracting feature elements of a reproduced image and discriminating local feature elements;

structural information extracting means for extracting structural information of the feature elements corresponding to the local feature elements discriminated by said extracting means including a change in motion of an object of interest; and

curve generating means for smoothly interpolating between the local feature elements corresponding to the feature elements extracted by said extracting means on the basis of the structural information.

35. The apparatus according to claim 34, wherein said reproducing means further combines a prescribed background image and the interpolated image to reproduce an image.

36. The apparatus according to claim 34, wherein said local feature elements include an element having the shape of an L-type intersection.

37. The apparatus according to claim 34, wherein said local feature elements include an element having the shape of an arc of a prescribed curvature.

38. The apparatus according to claim 34, wherein said local feature elements include an element having the shape of an arrow-type intersection.

39. The apparatus according to claim 34, wherein said local feature elements include an element having the shape of a circle.

40. The apparatus according to claim 34, wherein said local feature elements include an element having the shape of a T-type intersection.

41. The apparatus according to claim 34, wherein said local feature elements include an element which is color.

42. The apparatus according to claim 34, wherein said local feature elements include an element which is spatial frequency.

43. The apparatus according to claim 34, wherein said local feature elements include an element which is luminance.

44. An image reproduction apparatus comprising:

extracting means for extracting a feature element immediately prior to occurrence of an occlusion in which at least part of a feature element of an object is hidden behind another feature element;

second-feature-element generating means for generating a second feature element by using a plurality of feature elements immediately prior to occurrence of the occlusion extracted by said extracting means; and

means for reproducing an image on the basis of the second feature element generated by said second-feature-element generating means.