Method, apparatus, computer program, computer system, and computer-readable storage medium for representing and searching for an object in an image

• US 20050238256A1
• Filed: 04/08/2005
• Published: 10/27/2005
• Est. Priority Date: 07/05/1999
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Bober, Miroslaw Z.
• Granted Patent: US 7,162,105 B2
• Time in Patent Office: Days
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/752 Contour matching

Abstract

A method of representing an object appearing in a still or video image, by processing signals corresponding to the image, comprises deriving a plurality of numerical values associated with features appearing on the outline of an object starting from an arbitrary point on the outline and applying a predetermined ordering to said values to arrive at a representation of the outline.

Claims

1-24. -24. (canceled)

25. A method of searching for an object in an image or sequence of images by processing signals corresponding to the image or images, the method comprising inputting a query in the form of a two-dimensional outline, deriving a descriptor of said outline, obtaining a descriptor of objects in stored images and comparing said query descriptor with each descriptor for a stored object, and selecting and displaying at least one result corresponding to an image containing an object for which the comparison indicates a degree of similarity between the query and said object, wherein the stored and query descriptors are derived from a curvature scale space representation of the outline of the object, wherein the peak co-ordinate values of the CSS representation are ordered on the basis of the peak height values of the peak co-ordinates.

26. A method as claimed in claim 25 wherein the values are ordered starting from the greatest value.

27. A method as claimed in claim 26 wherein the values are ordered in decreasing size.

28. A method as claimed in claim 25 wherein the values are ordered starting from the smallest value.

29. A method as claimed in claim 25 wherein a factor is derived for the query outline and for each stored outline, and the comparison is made using the predetermined ordering only or the predetermined ordering and some other ordering depending on said factors, wherein the factors are derived by deriving a plurality of numerical values associated with features appearing on the outline of an object to represent said outline and deriving a factor indicating the reliability of said representation using a relationship between at least two of said values.

30. A method as claimed in claim 29 wherein the factor is based on the ratio between two of said values.

31. A method as claimed in claim 30 wherein the ratio is of the two greatest values.

32. A method as claimed in claim 29 wherein the numerical values corresponds to peak coordinate values in a CSS representation.

33. A method as claimed in claim 32 wherein said peak coordinate values are ordered on the basis of the peak height values.

39. An apparatus adapted to implement a method as claimed in claim 25.

40. A computer program for implementing a method as claimed in claim 25.

41. A computer system programmed to operate according to a method as claimed in claim 25.

42. A computer-readable storage medium storing computer executable process steps for implementing a method as claimed in claim 25.

34. A method for searching for an object appearing in an image, comprising:

receiving an input including at least one object having an outline;

determining a curvature scale space representation for said input outline to generate peak coordinates for the curvature scale space representation;

ordering said peak coordinates based on peak height value to generate a shape descriptor for said input outline;

comparing said shape descriptor for said input outline to a shape descriptor for an image stored in memory to find at least one substantially matching image; and

outputting said at least one substantially matching image.

35. A system for searching for an object appearing in an image, comprising:

an input device for inputting at least one object having an outline;

a controller and a detector for receiving said input object having an outline and determining a curvature scale space representation for said input outline to generate peak coordinates for the curvature scale space representation;

wherein said controller to order said peak coordinates based on peak height value to generate a shape descriptor for said input outline;

a comparator for comparing said shape descriptor for said input outline to a shape descriptor for an image stored in memory to find at least one substantially matching image; and

a display for displaying said at least one substantially matching image.

36. A method for searching an image applied to a comparator in an search system as claimed in claim 35, comprising:

receiving a first shape descriptor for an object in said image, said shape descriptor including a plurality of peak coordinates for a curvature scale space (CSS) representation of an outline of said object, said peak coordinates being ordered based on peak height values, said peak coordinates of said CSS representation are generated by smoothing said outline in a plurality of stages; and

comparing said first shape descriptor for said object to a second shape descriptor for an image stored in memory to find at least one substantially matching image, wherein each of said peak coordinates ordered based on peak height is compared to corresponding coordinate of said second shape descriptor in said memory.

37. A method for searching an image applied to a controller in a search system as claimed in claim 35, comprising:

generating peak coordinates for a curvature scale space (CSS) representation of an outline of an object in said image by smoothing said outline in a plurality of stages;

ordering said peak coordinates based on peak height values to generate a shape descriptor for said object;

transmitting said shape descriptor to a comparator to search at least one substantially matching image; and

receiving, from said comparator, and displaying said one substantially matching image.

38. A machine-readable medium having stored thereon a plurality of executable instructions, the plurality of instructions comprising instructions to:

receive an input including at least one object having an outline;

determine a curvature scale space representation for said input outline to generate peak coordinates for the curvature scale space representation;

order said peak coordinates based on peak height value to generate a shape descriptor for said input outline;

compare said shape descriptor for said input outline to a shape descriptor for an image stored in memory to find at least one substantially matching image; and

display said at least one substantially matching image.