ARTICLE MANAGEMENT SYSTEM, INFORMATION PROCESSING APPARATUS, AND CONTROL METHOD AND CONTROL PROGRAM OF INFORMATION PROCESSING APPARATUS

• US 20150379366A1
• Filed: 02/17/2014
• Published: 12/31/2015
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: IWAMOTO, Kota, HATTORI, Wataru, NOMURA, Toshiyuki, HIGA, Kyota, OHASHI, Keishi
• Granted Patent: US 10,438,084 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06K 17/00 Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
• CPC Class Codes: G06K 7/10366 the interrogation device being adapted for miscellaneous applications
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

An apparatus of this invention is directed to an information processing apparatus that effectively counts, on a type basis, articles of a plurality of types displayed in a depth direction on a display shelf. The information processing apparatus includes a display count acquirer that acquires display count information of articles using article presence/absence sensors provided on the display shelf on which the articles are placed, an article identifier that acquires article identification information capable of identifying the types of articles based on an image acquired by capturing the display shelf, and a display recognizer that recognizes, based on the display count information and the article identification information, display count of each type of the articles.

Claims

1. An information processing apparatus comprising:

a memory storing instructions; and

at least one processor configured to process the instructions to;

acquire display count information of articles using article presence/absence sensors provided on a display shelf on which the articles are placed;

acquire article identification information capable of identifying types of articles based on an image acquired by capturing the display shelf; and

recognize, based on the display count information and the article identification information, display count of each type of the articles.

2. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

acquire the display count information of articles included in an article column on the display shelf, in which articles of the same type are arranged from a near side to a far side, and

identify the type of an article in a front of the article column based on the image obtained by capturing a front of the display shelf.

3. The information processing apparatus according to claim 1, wherein each of the article presence/absence sensors comprises:

RFID tags, each having an identifier, which are placed at display positions of articles;

a response receiver that has an antenna for generating, in a region including the RFID tags, an electromagnetic field a strength of which is changed by presence of article, and receives response signals from the RFID tags; and

a memory storing instructions; and

at least one processor configured to process the instructions to;

determine presence/absence of articles based on strengths of the response signals from the RFID tags.

4. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

generate local features of a plurality of feature points from the image acquired by capturing the display shelf;

cluster the plurality of feature points based on coordinates of the plurality of feature points; and

collate the local features of the clustered feature points with a local feature generated based on an image of an article.

5. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

store an arrangement of a detection sensor group corresponding to a shelf allocation of the display shelf, and

link the display count information and the article identification information by referring to the arrangement of the detection sensor group.

6. The information processing apparatus according to claim 1, wherein a sheet identification mark unique to each article presence/absence sensor group is attached to each of the article presence/absence sensors on the front of the display shelf, and wherein the at least one processor is further configured to process the instructions to:

acquire the display count information of each article column from a plurality of article presence/absence sensors together with information of the sheet identification mark,

identify the sheet identification mark included in the captured image and acquire the information of the sheet identification mark, and

link the display count information and the article identification information using the information of the sheet identification mark.

7. The information processing apparatus according to claim 1, wherein a plurality of display shelves exist, and a unique shelf identification mark is attached to the front of each display shelf, and wherein the at least one processor is further configured to process the instructions to:

recognize the shelf identification mark, and

link the display count information and the article identification information using the shelf identification mark.

8. The information processing apparatus according to claim 1, wherein a plurality of display shelves exist, and wherein the at least one processor is further configured to process the instructions to:

store an array of the plurality of articles in at least part of the display shelf,

specify the display shelf based on the array of the articles, and

link the display count information and the article identification information in the specified display shelf.

9. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

collate an article presence/absence pattern identified based on a captured image and an article presence/absence pattern acquired from the article presence/absence sensor, link the article identification information and the article presence/absence information at a display position of the display shelf in which the article presence/absence patterns match, and recognize the display count of the articles.

10. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

collate presence/absence information of a plurality of articles acquired from the article presence/absence sensor and the identified types of a plurality of articles and determine an incorrect arrangement of an article in the article column.

11. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

determine whether the articles have been identified;

determine whether the display count is zero; and

adjust at least one parameter used to identify the articles in response to at least one of;

(1) determining that the articles have not been identified and determining that the display count is not zero and (2) determining that the articles have been identified and determining that the display count is zero.

12. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to process the instructions to:

display the display count of the articles recognized by the display recognizer.

13. A control method of an information processing apparatus, the control method comprising:

acquiring display count information of articles using article presence/absence sensors provided on a display shelf on which the articles are placed;

acquiring article identification information capable of identifying types of articles based on an image acquired by capturing the display shelf; and

recognizing, based on the display count information and the article identification information, display count of each type of the articles.

14. A non-transitory computer readable medium storing a control program of an information processing apparatus for causing a computer to execute a method comprising:

acquiring display count information of articles using article presence/absence sensors provided on a display shelf on which the articles are placed;

acquiring article identification information capable of identifying types of articles based on an image acquired by capturing the display shelf; and

recognizing, based on the display count information and the article identification information, display count of each type of the articles.

15. An article management system comprising:

article presence/absence sensors provided on a display shelf on which articles are placed;

a memory storing instructions; and

at least one processor configured to process the instructions to;

acquire display count information of the articles using the article presence/absence sensors;

capture the display shelf;

acquire article identification information capable of identifying types of articles based on an image acquired by capturing the display shelf and

recognize, based on the display count information and the article identification information, display count of each type of the articles.