Apparatus, method, and program for managing articles

• US 10,217,083 B2
• Filed: 09/09/2016
• Issued: 02/26/2019
• Est. Priority Date: 11/02/2015
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Takahashi, Shihomi, Suzuki, Kaoru, Imoto, Kazunori, Yamaguchi, Osamu
• Primary Examiner(s): Vu, Kim Y
• Assistant Examiner(s): Bloom, Nathan J
• Application Number:
  US15/260,998
• Publication Number:
  US 20170124424A1
• Time in Patent Office: 900 Days
• Field of Search: None
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

According to one embodiment, processing circuitry of an apparatus for managing articles acquires a check image by capturing labels including label information of a character string. The processing circuitry also acquires plan data including a plurality of plan areas indicating installation positions of the labels in the shelf and the label information of the labels assigned to each of the plan areas. The processing circuitry detects a character area of the character string displayed on each of the labels from the check image and recognizes characters in the character areas. The processing circuitry positions and superimposes the check image and the plan data and associates the character areas and the plan areas. The processing circuitry compares the label information of the associated plan areas and the recognition information of the character areas and assigns attributes to the character areas or the plan areas.

Claims

1. An apparatus for managing articles, comprising:

processing circuitry configured to;

acquire a check image by capturing an image of a shelf on which a plurality of articles are arranged and labels which are installed on a front surface of the shelf or front surfaces of the articles and include label information of a character string including one of letters, numbers, and symbols provided to manage the articles;

acquire plan data including a plurality of plan areas indicating installation positions of the labels in the shelf and the label information of the labels assigned to the respective plan areas;

detect a character area of the character string displayed on each of the labels from the check image;

recognize characters in the character areas;

position and superimpose the check image and the plan data, and associate each of the character areas and each of the plan areas;

compare the label information of the associated plan areas and the recognition information of the associated character areas, and assign attributes to the character areas or the plan areas, each of the attributes indicating a state related to correctness or incorrectness of installation of the articles which can be obtained from matching or mismatching between the label information and the recognition information;

obtain a distance between the character area and the plan area;

obtain a matching ratio from a matching degree between the recognition information of the character area and the label information of the plan area;

obtain a detection likelihood indicating the accuracy of characters in the character area;

obtain a recognition score indicating the correctness of the character recognition;

obtain an importance degree of a correction work of installation of the articles based on the distance, the matching ratio, the detection likelihood, and the recognition score; and

assign the importance as an attribute to the character area or the plan area.

2. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

create a work list for each of the character areas or each of the plan areas, the work list including positions of the plan areas, the label information, positions of the character areas, the recognition information and the attributes.

3. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

assign the attributes to the character areas of the check image and output the check image, or assign the attributes to the plan areas of the plan data and output the plan data.

4. The apparatus according to claim 1, wherein each of the plan areas of the plan data is rectangular, and the plan areas are vertically or horizontally arranged side by side, and wherein the processing circuitry is further configured to:

create a plurality of rectangular cells by partitioning the character areas of the check image and associate the plan areas and the character areas by allocating the character areas to the cells and superimposing the plan areas and the cells.

5. The apparatus according to claim 4, wherein the processing circuitry is further configured to:

superimpose the plan areas and the cells such that a vertical frame line and a horizontal frame line partitioning the plan areas are aligned with a vertical frame line and a horizontal frame line partitioning the cells, respectively.

6. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

obtain a matching ratio from a distance between the character area and the plan area and a matching degree between the recognition information of the character area and the label information of the plan area, assign an attribute of “

normal”

when the matching ratio is equal to or higher than a predetermined matching threshold, and assign an attribute of “

wrong installation”

when the matching ratio is lower than the matching threshold.

7. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

when the position of the character area does not match with the position of the plan area, determine that the labels are installed at positions at which the plan areas are not present, and assign an attribute of “

installation position error”

.

8. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

when the character area in the check image cannot be detected by a detection unit, determine that the labels are not installed, and assign an attribute of “

installation forgot”

.

9. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

when a detection likelihood indicating the accuracy of characters in the character area detected by a detection unit is lower than a predetermined detection threshold and a recognition score indicating the correctness of character recognition by a recognition unit is equal to or higher than a predetermined recognition threshold, determine that the labels are installed to be hard to be visible, and assign an attribute of “

installation deficiency”

.

10. A method for managing articles, comprising:

acquiring a check image by capturing an image of a shelf on which a plurality of articles are arranged, and labels which are installed on a front surface of the shelf or front surfaces of the articles and include label information of a character string including one of letters, numbers, and symbols provided to manage the articles;

acquiring plan data including a plurality of plan areas indicating installation positions of the labels in the shelf and the label information of the labels assigned to the plan areas, respectively;

detecting a character area of the character string displayed on each of the labels from the check image;

recognizing characters in the character areas;

positioning and superimposing the check image and the plan data and associating each of the character areas and each of the plan areas;

comparing the label information of the associated plan areas and the recognition information of the associated character areas and assigning attributes to the character areas or the plan areas, wherein each of the attributes indicates a state related to correctness or incorrectness of installation of the articles which can be obtained from matching or mismatching between the label information and the recognition information;

obtaining a distance between the character area and the plan area;

obtaining a matching ratio from a matching degree between the recognition information of the character area and the label information of the plan area;

obtaining a detection likelihood indicating the accuracy of characters in the character area;

obtaining a recognition score indicating the correctness of the character recognition;

obtaining an importance degree of a correction work of installation of the articles based on the distance, the matching ratio, the detection likelihood, and the recognition score; and

assigning the importance as an attribute to the character area or the plan area.

11. A program stored in a non-transitory computer readable medium, causing a computer to perform a method for managing articles, the method comprising:

acquiring a check image by capturing an image of a shelf on which a plurality of articles are arranged, and labels which are installed on a front surface of the shelf or front surfaces of the articles and include label information of a character string including one of letters, numbers and symbols provided to manage the articles;

acquiring plan data including a plurality of plan areas indicating installation positions of the labels in the shelf and the label information of the labels assigned to the plan areas, respectively;

detecting a character area of the character string displayed on each of the labels from the check image;

recognizing characters in the character areas;

positioning and superimposing the check image and the plan data and associating each of the character areas and each of the plan areas;

comparing the label information of the associated plan areas and the recognition information of the associated character areas and assigning attributes to the character areas or the plan areas, wherein each of the attributes indicates a state related to correctness or incorrectness of installation of the articles which can be obtained from matching or mismatching between the label information and the recognition information;

obtaining a distance between the character area and the plan area;

obtaining a matching ratio from a matching degree between the recognition information of the character area and the label information of the plan area;

obtaining a detection likelihood indicating the accuracy of characters in the character area;

obtaining a recognition score indicating the correctness of the character recognition;

obtaining an importance degree of a correction work of installation of the articles based on the distance, the matching ratio, the detection likelihood, and the recognition score; and

assigning the importance as an attribute to the character area or the plan area.