Method and apparatus for representing and searching for an object using shape

• US 7,761,438 B1
• Filed: 04/26/2000
• Issued: 07/20/2010
• Est. Priority Date: 04/26/2000
• Status: Expired due to Fees

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Bober, Miroslaw Z.
• Primary Examiner(s): To; Baoquoc N
• Application Number:
  US09/559,415
• Time in Patent Office: 3,737 Days
• Field of Search: 707/2, 707/3, 707/4, 707/5, 707/6, 707/10
• US Class Current: 707/706
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56

Abstract

A method of representing an object appearing in a still or video image for use in searching, wherein the object appears in the image with a first two-dimensional outline, by processing signals corresponding to the image, comprises deriving a view descriptor of the first outline of the object and deriving at least one additional view descriptor of the outline of the object in a different view, and associating the two or more view descriptors to form an object descriptor.

Claims

1. A method of searching for a query object in an image or sequence of images by processing signals corresponding to the images using a processor, the method comprising:

providing a plurality of stored images or sequences of images, each image or sequence of images containing a two-dimensional projection of a three-dimensional object in a perspective view;

for each image or sequence of images, providing an object descriptor, each object descriptor including a plurality of perspective view descriptors, each perspective view descriptor representing the outline of the shape of a two-dimensional projection of said three-dimensional object from a different perspective view of said three-dimensional object, including a perspective view not in the image or sequence of images;

inputting a query in the form of a query outline which is the outline of the shape of a two-dimensional projection of a three-dimensional query object in a perspective view;

deriving a query object descriptor of the query object using the query outline;

comparing said query object descriptor with at least one of said object descriptors;

selecting and displaying at least one result, which is an image or sequence of images containing an object for which comparison between the object descriptor and the query object descriptor indicates a degree of similarity between the query object and said object, wherein said object matches the query object but the selected and displayed image or sequence of images does not include said object in the same perspective view as the perspective view of the query object in said query.

2. A method as claimed in claim 1 wherein a query is input in the form of two or more two-dimensional outlines of an object, and wherein a query view descriptor is derived for each of said outlines, and wherein the step of comparing comprises comparing each of said query view descriptors with each view descriptor in each stored object descriptor to derive a plurality of view-similarity values.

3. A method as claimed in claim 2 wherein the view-similarity values are analyzed to derive object similarity values.

4. A method as claimed in claim 1, wherein at least some of the object descriptors include view-independent descriptors which are related to shape or size of the object, and wherein the method comprises inputting a view-independent query value and the step of comparing compares the query value with the view-independent descriptors for the stored object descriptors.

5. The method of claim 1, further comprising:

deriving an object descriptor for an object in an image by;

deriving a view descriptor of a first outline of a three-dimensional object in the image,

deriving at least one additional view descriptor of the outline of the object in a different perspective view from the perspective view in the image, and

associating the two or more view descriptors to form the object descriptor.

6. The method of claim 1, wherein said selecting and displaying includes selecting and displaying an image representation of an object having a different perspective view from perspective view of said query object based on said query object matching with at least two view descriptors including a view descriptor not representing perspective view of the object in the image representation.

7. The method of claim 1, wherein each said view descriptor is a different representation of the object from a different perspective view of the three-dimensional object.

8. The method of claim 1, wherein the step of comparing comprises comparing said query object descriptor with each of the plurality of perspective view descriptors.