Image processing device for displaying object detected from input picture image

• US 10,930,037 B2
• Filed: 02/24/2017
• Issued: 02/23/2021
• Est. Priority Date: 02/25/2016
• Status: Active Grant

• Current Assignee: Fanuc Corporation
• Original Assignee: Fanuc Corporation
• Inventors: Namiki, Yuta, Warashina, Fumikazu
• Primary Examiner(s): Beard, Charles L
• Application Number:
  US15/441,934
• Publication Number:
  US 20170249766A1
• Time in Patent Office: 1,460 Days
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 30/19133 Interactive pattern learning with a human teacher
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

An image processing device including an object detection unit for detecting one or more images of objects from an input picture image, on the basis of a model pattern of the object, and a detection result display unit for graphically superimposing and displaying a detection result. The detection result display unit includes a first frame for displaying the entire input picture image and a second frame for listing and displaying one or more partial picture images each including an image detected. In the input picture image displayed in the first frame, a detection result is superimposed and displayed on all the detected images, and in the partial picture image displayed in the second frame, a detection result of an image corresponding to each partial picture image is superimposed and displayed.

Claims

1. An image processing device comprising:

a processor for selecting a model pattern of an object, and using the selected model pattern to detect one or more images of a plurality of objects from an input picture image in which the plurality of objects have been captured by a camera; and

a display controlled by the processor for graphically superimposing and displaying a detection result on each image thus detected,

wherein the display includes;

a first frame for displaying the entire input picture image in which the plurality of objects have been captured by the camera; and

a second frame for listing and displaying a plurality of partial picture images each including a single respective one of the plurality of objects separately cropped from the input picture image, such that each of the plurality of objects are displayed separately in the second frame, and wherein;

in the input picture image displayed in the first frame, a detection result determined based on a degree of match between the detected images and the model pattern is superimposed and displayed on all the detected images, in each of the plurality of partial picture images listed and displayed in the second frame, a respective detection result determined based on a degree of match between the each of the plurality of partial picture images and the model pattern is superimposed and displayed on an image corresponding to each of the plurality of partial picture images, and the first frame and the second frame are displayed simultaneously, wherein the display further superimposes and displays, on the partial picture image displayed in the second frame, first label information stored in association with an image corresponding to each partial picture image, the first label information indicates;

a match decision made by the processor when the degree of match between the detected images and the model pattern is larger than or equal to a threshold value, and

a non-match decision made by the processor when the degree of match between the detected images and the model pattern is less than the threshold, and

wherein the display allows an operator to;

perform a confirmation of the match decision and the non-match decision made by the processor by maintaining the first label information, and perform an override of the match decision and the non-match decision made by the processor by manually changing the first label information, and wherein when the operator performs the override, then the processor is configured to perform a learning process to update the model pattern based on the override of the match decision, the override of the non-match decision, the confirmation of the match decision, and the confirmation of the non-match decision as learning data.

2. The image processing device according to claim 1, wherein the display displays picture images in each of the first frame and the second frame with a setting magnification rate for the first frame and the second frame.

3. The image processing device according to claim 1, wherein the display selects one of a plurality of the partial picture images displayed in the second frame, and

in the input picture image displayed in the first frame, a detection result of an image of an object corresponding to the partial picture image selected is displayed in a highlighted manner.

4. The image processing device according to claim 1, wherein the display superimposes, on the input picture image displayed in the first frame, the first label information corresponding to each of the one or more images detected from the input picture image.

5. The image processing device according to claim 1, wherein the image processing device associates a plurality of input picture images with respective detection results of one or more images of the objects detected from each of the plurality of input picture images and storing the plurality of input picture images and the respective detection results, and selects one of the plurality of input picture images, and

the display displays the one of the input picture images selected and one of the respective detection results in association with the one of the input picture images.

6. The image processing device according to claim 5, wherein the image processing device lists and displays the plurality of input picture images upon superimposing one or more images and the respective detection results of the objects detected from the input picture images, and

wherein the display displays the input picture image selected, and enables switching between the input picture image display and the detection result on the basis of an operation performed by an operator.

7. The image processing device according to claim 6, wherein the processor associates second label information with a plurality of input picture images and stores the second label information and the plurality of input picture images, and

the display further superimposes and displays the second label stored in association with each of the plurality of input picture images and allows an operator to change the second label information, and when the operator changes the second label information of one of the input picture images, the first label information respectively corresponding to the images of the plurality of objects stored in association with the input picture image are collectively changed.

8. The image processing device according to claim 1,

wherein the processor adds the first label information to each of the detected images, associating each first label information with each of the detected images to make learning data, and storing the learning data;

and

performs machine learning or statistical processing on the basis of the learning data.