Information processing device and position information acquisition method

• US 10,447,996 B2
• Filed: 09/26/2016
• Issued: 10/15/2019
• Est. Priority Date: 10/01/2015
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Ikenoue, Shoichi, Suwa, Yoshihiko
• Primary Examiner(s): Perungavoor, Sathyanaraya V
• Assistant Examiner(s): Dhillon, Peet
• Application Number:
  US15/754,778
• Publication Number:
  US 20180288400A1
• Time in Patent Office: 1,114 Days
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G01C 3/06 Use of electric means to obtain final indication
• CPC Class Codes: G01C 3/085 with electronic parallax measurement
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

When a device is started up in the first time, for example, an initial image and various kinds of information are displayed and setting inputs are received. In parallel with the above, a photographed image is acquired and calibration is performed. At this time, first, if there is no need for measures by a user such as a direction of a camera is corrected, the calibration is performed. If the measures by the user are needed or the calibration is ended unsuccessfully, the stage number n is advanced by one, and the calibration is repeated while presenting information on the measures in the order of a low burden on the user.

Claims

1. An information processing device that acquires position information on an object from an image photographed by a stereo camera and performs information processing, comprising:

a calibration unit configured to detect corresponding points from stereo images photographed from view points of left and right by the stereo camera and thereby perform calibration of the stereo camera; and

a position information acquisition unit configured to correct the stereo images obtained by photographing a moving picture of the object by the stereo camera and further detect the corresponding points to thereby derive a parallax on the basis of a result of the calibration, and acquire and output the position information on the object on the basis of the parallax,

wherein the calibration unit repeats stepwise processing for presenting prescribed information and retrying the calibration in the order that is set on the basis of a low burden on a user in accordance with an unsuccess of the calibration and ends an operation in a stage in which the calibration is successful.

2. The information processing device according to claim 1, further comprising:

an initial processing unit configured to perform initial processing other than the calibration, wherein

the calibration unit tries first calibration without presenting information on the calibration while the initial processing unit presents information for the initial processing.

3. The information processing device according to claim 1, wherein

the calibration unit allows a display device to instantly display at least either of the stereo images photographed by the stereo camera as the prescribed information in any stage.

4. The information processing device according to claim 1, wherein the calibration unit determines by means other than the calibration whether a processing start is right or wrong in the first calibration and, when determining that the processing start is right, does not present information on the calibration but tries the calibration and, when determining that the processing start is wrong, presents the prescribed information set as a next stage and tries the calibration.

5. The information processing device according to claim 4, wherein the calibration unit acquires an attitude of the stereo camera from a measured value of an acceleration sensor installed within the stereo camera and, when the attitude is present in a prescribed range, determines that the processing start of the calibration is right.

6. The information processing device according to claim 4, wherein the calibration unit subjects the stereo images to face detection processing and, when a face is detected, determines that the processing start of the calibration is right.

7. The information processing device according to claim 4, wherein the calibration unit subjects the stereo images to face recognition processing and, when a registered face is not detected, determines that the processing start of the calibration is wrong.

8. The information processing device according to claim 4, wherein the calibration unit allows the position information acquisition unit to acquire the position information on the basis of the result of the calibration performed previously and, when a normal position information is not obtained, determines that the processing start of the calibration is wrong.

9. The information processing device according to claim 1, wherein the calibration unit presents an instruction to the user for taking measures assumed to be necessary as the prescribed information in a final stage.

10. A position information acquisition method performed by an information processing device that acquires position information on an object from an image photographed by a stereo camera and performs information processing, comprising:

detecting corresponding points from stereo images photographed from view points of left and right by the stereo camera and thereby performing calibration of the stereo camera; and

correcting the stereo images obtained by photographing a moving picture of the object by the stereo camera and further detecting the corresponding points to thereby derive a parallax on the basis of a result of the calibration, and acquiring and outputting the position information on the object on the basis of the parallax,

wherein the performing the calibration repeats stepwise processing for presenting prescribed information and retrying the calibration in the order that is set on the basis of a low burden on a user in accordance with an unsuccess of the calibration and ends an operation in a stage in which the calibration is successful.

11. A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer that acquires position information on an object from an image photographed by a stereo camera and performs information processing, causes the computer to carry out actions, comprising:

detecting corresponding points from stereo images photographed from view points of left and right by the stereo camera and thereby performing calibration of the stereo camera; and

correcting the stereo images obtained by photographing a moving picture of the object by the stereo camera and further detecting the corresponding points to thereby derive a parallax on the basis of a result of the calibration, and acquiring and outputting the position information on the object on the basis of the parallax,

wherein performing the calibration repeats stepwise processing for presenting prescribed information and retrying the calibration in the order that is set on the basis of a low burden on a user in accordance with an unsuccess of the calibration and ends an operation in a stage in which the calibration is successful.