Information processing device, information processing method, and system

• US 9,870,616 B2
• Filed: 03/21/2016
• Issued: 01/16/2018
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Kikuchi, Naoki
• Primary Examiner(s): Tran, Trang U
• Application Number:
  US15/075,543
• Publication Number:
  US 20160284088A1
• Time in Patent Office: 666 Days
• Field of Search: 348187, 348188, 348180
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

An information processing device includes an obtainer and a controller. The obtainer obtains a captured image obtained through imaging by an imaging device. The controller generates instruction information for instructing how a subject having plural feature points is to be held up.

Claims

1. An information processing device, comprising:

processing circuitry that

obtains a captured image obtained through imaging a subject by an imaging device,

generates instruction information for instructing how the subject is to be held up,

wherein the instruction information is information indicating a shape of the subject that is supposed to be caught in the captured image when a position and a posture of the subject match the position and the posture corresponding to the instruction information, and

the processing circuitry generates the two or more pieces of corresponding instruction information having a one-to-one correspondence with two or more positions and postures of the subject, the two or more positions and postures being determined in advance correspondingly to an area, wherein the two or more pieces of instruction information indicate mutually different shapes.

2. The information processing device according to claim 1, wherein the instruction information is information for instructing a position and a posture of the subject with respect to the imaging device.

3. The information processing device according to claim 1, wherein the processing circuitry sequentially generates two or more corresponding pieces of instruction information, for each of plural areas that divide an image area of the captured image.

4. The information processing device according to claim 3, wherein when the relation between the instruction information and the subject caught in the captured image satisfies the predetermined condition, the processing circuitry obtains feature point data in the captured image and positions of the feature points caught in the captured image, and generates next instruction information.

5. The information processing device according to claim 4, wherein

the instruction information is information indicating a shape of the subject that is supposed to be caught in the captured image when the position and the posture of the subject match the position and the posture corresponding to the instruction information, and

the processing circuitry determines that the predetermined condition is satisfied, when the number of feature points extracted from the captured image is equal to or greater than a first threshold, and a difference between a shape indicated by the instruction information and a shape of the subject caught in the captured image is less than a second threshold.

6. The information processing device according to claim 4, wherein the processing circuitry performs notification to prompt the position and posture of the subject to be changed when the relation between the instruction information and the subject caught in the captured image does not satisfy the predetermined condition.

7. The information processing device according to claim 4, wherein the processing circuitry sequentially generates the two or more pieces of instruction information corresponding to another area when two or more sets of feature point data having a one-to-one correspondence with the two or more pieces of instruction information corresponding to any one area of the plural areas have been obtained.

8. The information processing device according to claim 4, wherein the processing circuitry calculates, based on plural sets of feature point data, a correction parameter for correcting the captured image.

9. The information processing device according to claim 8, wherein the processing circuitry calculates the correction parameter based on the plural sets of feature point data having a one-to-one correspondence with all of the instruction information, when, for each area of the plural areas, the two or more sets of feature point data having a one-to-one correspondence with the two or more pieces of instruction information corresponding to the area have been obtained.

10. The information processing device according to claim 8, wherein every time the two or more sets of feature point data having a one-to-one correspondence with the two or more pieces of instruction information corresponding to any one area of the plural areas are obtained, the processing circuitry calculates the correction parameter based on the obtained two or more sets of feature point data.

11. The information processing device according to claim 3, wherein

the instruction information is information indicating a shape of the subject that is supposed to be caught in the captured image when the position and the posture of the subject match the position and the posture corresponding to the instruction information, and

the processing circuitry sequentially generates, for each area of the plural areas, the two or more pieces of instruction information corresponding to the area, such that a shape of the subject indicated by at least one of the two or more pieces of instruction information corresponding to that area extends over that area and another area adjacent to that area.

12. The information processing device according to claim 1, wherein the processing circuitry performs control of displaying, on a display, a superimposed image including the captured image and the instruction information superimposed on the captured image.

13. The information processing device according to claim 12, wherein every time the instruction information is generated, the processing circuitry changes the superimposed image to be displayed on the display by performing control of displaying, on the display, the superimposed image including the captured image and the generated instruction information superimposed on the captured image.

14. The information processing device according to claim 1, wherein

the instruction information is information for instructing a position and a posture of the subject with respect to the imaging device, and

the processing circuitry sequentially generates two or more corresponding pieces of instruction information, for each of plural areas that divide an image area of the captured image.

15. The information processing device according to claim 1, wherein

the instruction information is information for instructing a position and a posture of the subject with respect to the imaging device,

the instruction information is information indicating a shape of the subject that is supposed to be caught in the captured image when a position and a posture of the subject match the position and the posture corresponding to the instruction information, and

the processing circuitry sequentially generates two or more corresponding pieces of instruction information, for each of plural areas that divide an image area of the captured image.

18. The information processing device of claim 1, wherein the processing circuitry further

displays a superimposed image including the captured image and the instruction information,

determines whether a relation between the instruction information and the subject caught in the captured image satisfies a predetermined condition, by determining at least whether a number of feature points extracted from the captured image is equal to or greater than a first threshold, and

notifies a user to change how the subject is held up when determining that the predetermined condition is not satisfied.

16. An information processing method, comprising:

obtaining a captured image obtained through imaging a subject by an imaging device;

displaying, on a display, a superimposed image including the captured image and instruction information superimposed on the captured image, the instruction information for instructing how the subject is to be held up, wherein the instruction information is information indicating a shape of the subject that is supposed to be caught in the captured image when a position and a posture of the subject match the position and the posture corresponding to the instruction information, and

generating the two or more pieces of corresponding instruction information having a one-to-one correspondence with two or more positions and postures of the subject, the two or more positions and postures being determined in advance correspondingly to an area, wherein the two or more pieces of instruction information indicate mutually different shapes.

17. A system including an imaging device, an information processing device, and a display, the system comprising:

processing circuitry that

displays, on the display, a superimposed image including a captured image obtained through imaging a subject by the imaging device, and instruction information superimposed on the captured image, the instruction information being for instructing how the subject is to be held up,

wherein the instruction information is information indicating a shape of the subject that is supposed to be caught in the captured image when a position and a posture of the subject match the position and the posture corresponding to the instruction information, and

the processing circuitry generates the two or more pieces of corresponding instruction information having a one-to-one correspondence with two or more positions and postures of the subject, the two or more positions and postures being determined in advance correspondingly to an area, wherein the two or more pieces of instruction information indicate mutually different shapes.