Information processing apparatus, method for controlling information processing apparatus, image processing system and non-transitory computer-readable storage medium

• US 10,198,868 B2
• Filed: 05/19/2016
• Issued: 02/05/2019
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Yamaguchi, Hiroichi
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Imperial, Jed-Justin
• Application Number:
  US15/158,869
• Publication Number:
  US 20160350976A1
• Time in Patent Office: 992 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 by influencing the exposure time

Abstract

An information processing apparatus comprises a luminance value obtaining unit for respectively obtaining first and second luminance values of first and second captured images of a physical space, wherein the first and second captured images are respectively captured by first and second image capturing units; a parameter obtaining unit for respectively obtaining first and second tracking parameters of automatic exposure control corresponding to a change in brightness of the physical space for the first and second image capturing units; and a setting unit for setting a first image capturing parameter for the first image capturing unit based on the first luminance value and the obtained first tracking parameter and a second image capturing parameter for the second image capturing unit based on the second luminance value and the obtained second tracking parameter.

Claims

1. An information processing apparatus comprising:

a luminance value obtaining unit configured to obtain a first luminance value of a first captured image of an external environment and a second luminance value of a second captured image of the external environment, wherein the first captured image is captured by a first image capturing unit for obtaining position and orientation of a head-mounted type display apparatus and the second captured image is captured by a second image capturing unit for capturing an image of the external environment on which a computer graphics image is superimposed;

a parameter obtaining unit configured to obtain a first tracking parameter and a second tracking parameter corresponding to a change in brightness of the external environment, the first tracking parameter indicating a first time difference from when the brightness of the external environment changes until a start of automatic exposure control by the first image capturing unit to track the change in brightness of the external environment, the second tracking parameter indicating a second time difference from when the brightness of the external environment changes until a start of automatic exposure control by the second image capturing unit to track the change in brightness of the external environment, and the first time difference from when the brightness of the external environment changes until the start of automatic exposure control by the first image capturing unit for obtaining position and orientation of the head-mounted type display apparatus is less than the second time difference from when the brightness of the external environment changes until the start of automatic exposure control by the second image capturing unit for capturing the image of the external environment on which the computer graphics image is superimposed; and

a setting unit configured to set a first image capturing parameter for the first image capturing unit based on the first luminance value and the obtained first tracking parameter and a second image capturing parameter for the second image capturing unit based on the second luminance value and the obtained second tracking parameter.

2. The information processing apparatus according to claim 1, further comprising:

a detection unit configured to detect an orientation of the first image capturing unit;

a determination unit configured to determine a photometric method based on the orientation of the first image capturing unit; and

an obtaining unit configured to obtain, based on a luminance value of a captured image of the first image capturing unit and on the photometric method, an integrated value of the luminance value,

wherein the setting unit sets the first image capturing parameter for the first image capturing unit based on the integrated value of the luminance value and the first tracking parameter.

3. The information processing apparatus according to claim 2,

wherein the detection unit further detects an orientation of a third image capturing unit that is different from the first image capturing unit, and

the information processing apparatus further includes a control unit configured to switch to synchronized control or separate control of the first image capturing unit and the third image capturing unit, based on the orientation of the first image capturing unit and the orientation of the third image capturing unit.

4. The information processing apparatus according to claim 3,

wherein the first image capturing unit is an image capturing unit for obtaining a position and orientation of the information processing apparatus, and

the second image capturing unit is an image capturing unit for experiencing mixed reality, and

the third image capturing unit is an image capturing unit for obtaining a position and orientation of the information processing apparatus.

5. The information processing apparatus according to claim 1, further comprising:

a determination unit configured to determine a photometric method; and

an obtaining unit configured to obtain, based on a luminance value of a captured image of the second image capturing unit and on the photometric method, an integrated value of the luminance value,

wherein the setting unit sets the second image capturing parameter for the second image capturing unit based on the integrated value of the luminance value and the second tracking parameter.

6. The information processing apparatus according to claim 1,

wherein the first tracking parameter corresponding to the first image capturing unit is different from the second tracking parameter corresponding to the second image capturing unit.

7. The information processing apparatus according to claim 1,

wherein the setting unit sets the first image capturing parameter after a predetermined time corresponding to the first tracking parameter has elapsed and sets the second image capturing parameter after a predetermined time corresponding to the second tracking parameter has elapsed.

8. The information processing apparatus according to claim 1,

wherein at least one of the first image capturing parameter and the second image capturing parameter is at least one of a shutter speed and a gain.

9. The information processing apparatus according to claim 1,

wherein the information processing apparatus is the head-mounted type display apparatus.

10. An image processing system comprising an information processing apparatus and an image processing apparatus that is wire-connected or wirelessly-connected to the information processing apparatus, the information processing apparatus including:

a luminance value obtaining unit configured to obtain a first luminance value of a first captured image of an external environment and a second luminance value of a second captured image of the external environment, wherein the first captured image is captured by a first image capturing unit for obtaining position and orientation of a head-mounted type display apparatus and the second captured image is captured by a second image capturing unit for capturing an image of the external environment on which a computer graphics image is superimposed;

a parameter obtaining unit configured to obtain a first tracking parameter and a second tracking parameter corresponding to a change in brightness of the external environment, the first tracking parameter indicating a first time difference from when the brightness of the external environment changes until a start of automatic exposure control by the first image capturing unit to track the change in brightness of the external environment, the second tracking parameter indicating a second time difference from when the brightness of the external environment changes until a start of automatic exposure control by the second image capturing unit to track the change in brightness of the external environment, and the first time difference from when the brightness of the external environment changes until the start of automatic exposure control by the first image capturing unit for obtaining position and orientation of the head-mounted type display apparatus is less than the second time difference from when the brightness of the external environment changes until the start of automatic exposure control by the second image capturing unit for capturing the image of the external environment on which the computer graphics image is superimposed; and

a setting unit configured to set a first image capturing parameter for the first image capturing unit based on the first luminance value and the obtained first tracking parameter and a second image capturing parameter for the second image capturing unit based on the second luminance value and the obtained second tracking parameter.

11. A method for controlling an information processing apparatus, comprising:

obtaining a first luminance value of a first captured image of an external environment and a second luminance value of a second captured image of the external environment, wherein the first captured image is captured by a first image capturing unit for obtaining position and orientation of a head-mounted type display apparatus and the second captured image is captured by a second image capturing unit for capturing an image of the external environment on which a computer graphics image is superimposed;

obtaining a first tracking parameter and a second tracking parameter corresponding to a change in brightness of the external environment, the first tracking parameter indicating a first time difference from when the brightness of the external environment changes until a start of automatic exposure control by the first image capturing unit to track the change in brightness of the external environment, the second tracking parameter indicating a second time difference from when the brightness of the external environment changes until a start of automatic exposure control by the second image capturing unit to track the change in brightness of the external environment, and the first time difference from when the brightness of the external environment changes until the start of automatic exposure control by the first image capturing unit for obtaining position and orientation of the head-mounted type display apparatus is less than the second time difference from when the brightness of the external environment changes until the start of automatic exposure control by the second image capturing unit for capturing the image of the external environment on which the computer graphics image is superimposed; and

setting a first image capturing parameter for the first image capturing unit based on the first luminance value and the obtained first tracking parameter and a second image capturing parameter for the second image capturing unit based on the second luminance value and the obtained second tracking parameter.

12. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute a method for controlling an information processing apparatus, comprising:

obtaining a first luminance value of a first captured image of an external environment and a second luminance value of a second captured image of the external environment, wherein the first captured image is captured by a first image capturing unit for obtaining position and orientation of a head-mounted type display apparatus and the second captured image is captured by a second image capturing unit for capturing an image of the external environment on which a computer graphics image is superimposed;

obtaining a first tracking parameter and a second tracking parameter corresponding to a change in brightness of the external environment, the first tracking parameter indicating a first time difference from when the brightness of the external environment changes until a start of automatic exposure control by the first image capturing unit to track the change in brightness of the external environment, the second tracking parameter indicating a second time difference from when the brightness of the external environment changes until a start of automatic exposure control by the second image capturing unit to track the change in brightness of the external environment, and the first time difference from when the brightness of the external environment changes until the start of automatic exposure control by the first image capturing unit for obtaining position and orientation of the head-mounted type display apparatus is less than the second time difference from when the brightness of the external environment changes until the start of automatic exposure control by the second image capturing unit for capturing the image of the external environment on which the computer graphics image is superimposed; and

setting a first image capturing parameter for the first image capturing unit based on the first luminance value and the obtained first tracking parameter and a second image capturing parameter for the second image capturing unit based on the second luminance value and the obtained second tracking parameter.