Collation/retrieval system, collation/retrieval server, image feature extraction apparatus, collation/retrieval method, and program

• US 9,842,285 B2
• Filed: 07/25/2012
• Issued: 12/12/2017
• Est. Priority Date: 07/29/2011
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Ishiyama, Rui
• Primary Examiner(s): Woldemariam, Aklilu
• Application Number:
  US14/235,292
• Publication Number:
  US 20140153831A1
• Time in Patent Office: 1,966 Days
• Field of Search: 382115, 382124, 382125, 382190, 382278, 345179, 345419
• CPC Class Codes: G06Q 30/0185 Product, service or business identity fraud
• CPC Class Codes: G06Q 30/0278 Product appraisal
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns

Abstract

The present invention is a collation/retrieval system collating a product manufactured by or delivered from a producer or a distributor with a product to be collated comprising: a storage unit that stores an image feature of a predetermined collation area of the product determined in advance at a position relative to a reference section common in every product; a to-be-collated product feature extraction unit that receives an image of the product to be collated and detecting the reference section of the product from the received image to extract an image feature of the collation area determined by reference to the reference section; and a collation unit that collates the stored image feature with the image feature of the collation area of the product to be collated.

Claims

1. A collation/retrieval system collating a product manufactured by or delivered from a producer or a distributor with a product to be collated wherein:

a collation area is a partial area on a surface of the product; and

a reference section is a product identifying label on the product which is common among products, comprising;

a storage device that stores a surface pattern image feature of the collation area determined in advance at a position relative to the reference section; and

at least one processor configured to;

receive an image of the product to be collated;

detect the reference section in the product to be collated from the received image;

extract a surface pattern image feature of the collation area in the product to be collated as determined by reference to the detected reference section; and

collate the stored surface pattern image feature with the surface pattern image feature of the collation area of the product to be collated,

wherein the at least one processor is further configured to;

store in advance a feature of the reference section and the position of the collation area in the storage device;

detect the reference section based on the feature of the reference section;

normalize the position and a size of the collation area by referring to the reference section to extract the surface pattern image feature of the collation area; and

perform at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and a range of the collation area by a relative value determined by reference to a size of the reference section.

2. The collation/retrieval system according to claim 1, wherein the at least one processor is further configured to:

receive an image of the product manufactured by or delivered from a producer or a distributor;

detect the reference section from the image received;

determine the collation area by reference to the reference section detected from the image received; and

extract the surface pattern image feature of the determined collation area to register the extracted surface pattern image feature of the detected collation area in the storage device.

3. The collation/retrieval system according to claim 1,

wherein the storage device stores a plurality of surface pattern image features of collation areas extracted from a plurality of products to be retrieved and management information corresponding to the plurality of products to be retrieved in a mutually associated manner;

and

wherein the at least one processor identifies the stored surface pattern image feature from the plurality of surface pattern image features as having a highest similarity to the surface pattern image feature of the collation area of the product to be collated to output management information of the product.

15. The collation/retrieval system of claim 1 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.

4. An image feature extraction apparatus wherein:

a collation area is a partial area on a surface of a product; and

a reference section is a product identifying label on the product which is common among products, comprising a feature extraction unit that;

stores in advance a feature of the reference section common among products and a position of the collation area determined by reference to the reference section;

detects the reference section in an image of a product to be collated based on the feature of the reference section;

normalizes the position and a size of the collation area by referring to the detected reference section;

extracts a surface pattern image feature of the collation area based on the normalized position and size of the collation area; and

performs at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and the range of the collation area by a relative value determined by reference to a size of the reference section.

5. The image feature extraction apparatus according to claim 4, further comprising:

a transmission unit that transmits the surface pattern image feature of the collation area to a collation server.

16. The image feature extraction apparatus of claim 4 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.

6. A collation/retrieval method performed by at least one processor, comprising:

setting a predetermined section of a product comprising a product identifying label on the product to a reference section, wherein the product identifying label is the same among products;

setting a partial area on a surface of the product existing at a standard position relative to the reference section to a collation area;

storing a feature of the reference section and the position of the collation area relative to the reference section;

detecting the reference section from a captured image of the product based on the feature of the reference section;

normalizing the position and a size of the collation area by referring to the detected reference section;

extracting a surface pattern image feature of the collation area based on the normalized position and size of the collation area;

performing at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and the range of the collation area by a relative value determined by reference to a size of the reference section; and

using the surface pattern image feature in the collation area of the product for collation and retrieval of the product.

7. The collation/retrieval method according to claim 6, further comprising:

capturing a first image comprising at least the reference section common among products and the predetermined collation area of the product determined in advance at a position relative to the reference section;

detecting the reference section from the captured first image to extract the surface pattern image feature of the collation area determined by reference to the detected reference section;

storing the extracted surface pattern image feature of the collation area;

capturing a second image comprising at least the reference section of the product to be collated and the predetermined collation area of the product to be collated determined by reference to the reference section;

detecting the reference section of the product to be collated from the captured second image to extract the surface pattern image feature in the second image of the collation area determined by reference to the reference section; and

collating the stored image feature with the surface pattern image feature of the collation area of the product to be collated.

8. The collation/retrieval method according to claim 6, wherein the reference section comprises a trademark, a logotype, a name of manufacturer, a manufacturing company, a brand name, a product name, and a product number attached to, printed, stamped, or branded on the product.

9. The collation/retrieval method according to claim 6 comprising:

storing a plurality of surface pattern image features of collation areas extracted from a plurality of products to be retrieved and management information corresponding to the plurality of products to be retrieved in a mutually associated manner, and

identifying a stored surface pattern image feature from the plurality of stored surface pattern image features as having a highest similarity to the surface pattern image feature of the collation area of the product to be retrieved to output management information of the product to be retrieved.

10. The collation/retrieval method according to claim 6, wherein the product is an industrial product comprising a leather product, a metal workpiece, a molten resin, and/or a ceramic portion.

17. The collation/retrieval method of claim 6 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.

11. A collation/retrieval system collating a product manufactured by or delivered from a producer or a distributor with a product to be collated wherein:

a collation area is a partial area on a surface of the product; and

a reference section is a product identifying label on the product which is common among products, comprising;

a storage unit that stores a minute image feature of the collation area determined in advance at a position relative to the reference section;

a product feature extraction unit;

a to-be-collated product feature extraction unit that receives an image of the product to be collated, detects the reference section in the product to be collated from the received image and extracts a minute image feature of the collation area in the product to be collated as determined by reference to the reference section; and

a collation unit that collates the stored image feature with the minute image feature of the collation area of the product to be collated, wherein the product feature extraction unit and the to-be-collated product feature extraction unit each;

store in advance a feature of the reference section and the position of the collation area;

detect the reference section based on the feature of the reference section;

normalize the position and a size of the collation area by referring to the reference section to extract the surface pattern image feature of the collation area;

and perform at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and the range of the collation area by a relative value determined by reference to a size of the reference section.

18. The collation/retrieval system of claim 11 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.

12. A collation/retrieval system collating a product manufactured by or delivered from a producer or a distributor with a product to be collated wherein:

a collation area is a partial area on a surface of the product; and

a reference section is a product identifying label on the product which is common among products, comprising;

a memory device that stores a surface pattern image feature of the collation area determined in advance at a position relative to the reference section; and

a microprocessor programmed to;

store in advance a feature of the reference section and the position of the collation area;

receive an image of the product to be collated;

detect the reference section in the product to be collated from the received image based on the feature of the reference section;

normalize the position and a size of the collation area by referring to the detected reference section;

perform at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and the range of the collation area by a relative value determined by reference to a size of the reference section;

extract a surface pattern image feature of the collation area in the product to be collated as determined based on the normalized position and size of the collation area; and

collate the stored surface pattern image feature with the surface pattern image feature of the collation area of the product to be collated.

19. The collation/retrieval system of claim 12 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.

13. An image feature extraction apparatus wherein:

a collation area is a partial area on a surface of a product; and

a reference section is a product identifying label on the product which is common among products, comprising;

a microprocessor programmed to;

Store in advance a feature of the reference section and a position of the collation area determined by reference to the reference section;

detect the reference section in an image of a product to be collated based on the feature of the reference section;

normalize the position and size of the collation area by referring to the detected reference section;

perform at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and the range of the collation area by a relative value determined by reference to a size of the reference section; and

extract a surface pattern image feature of the collation area based on the normalized position and size of the collation area.

20. The image feature extraction apparatus of claim 13 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.

14. A collation/retrieval method using a microprocessor, the microprocessor to execute:

setting a predetermined section of a product comprising a product identifying label on the product to a reference section, wherein the product identifying label is the same among products;

setting a partial area on a surface of the product existing at a standard position relative to the reference section to a collation area;

storing a feature of the reference section and the position of the collation area relative to the reference section;

detecting the reference section from a captured image of the product based on the feature of the reference section;

normalizing the position and a size of the collation area by referring to the detected reference section;

performing at least one of an identification of the collation area by a distance and angle from the reference section and a designation of the position and the range of the collation area by a relative value determined by reference to a size of the reference section;

extracting the surface pattern image feature of the collation area based on the normalized position and size of the collation area; and

using the surface pattern image feature in the collation area of the product for collation and retrieval of the product.

21. The collation/retrieval method of claim 14 wherein the product identifying label comprises at least one of a trademark, a logotype, a name of a manufacturer, a brand name, a product name, and a product number.