Information processing apparatus, information processing method, and storage medium

• US 10,078,918 B2
• Filed: 10/18/2016
• Issued: 09/18/2018
• Est. Priority Date: 10/20/2015
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Koga, Kazuhiro
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US15/296,889
• Publication Number:
  US 20170109937A1
• Time in Patent Office: 700 Days
• Field of Search: 345633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

An information processing apparatus configured to generate an image to be displayed on a display device including an image capturing unit includes, an image acquisition unit configured to acquire a captured image obtained by the image capturing unit, an estimation unit configured to estimate a position and an orientation of the image capturing unit based on the captured image, a first generation unit configured to generate an image of a virtual object based on the estimated position and orientation of the image capturing unit, a moving velocity acquisition unit configured to acquire a moving velocity of the image capturing unit, a second generation unit configured to generate an image representing the moving velocity, and a display control unit configured to cause the display device to display a composite image obtained by combining the image of the virtual object and the image generated by the second generation unit.

Claims

1. An information processing apparatus configured to generate an image to be displayed on a display device including an image capturing unit, the information processing apparatus comprising:

an image acquisition unit configured to acquire a captured image obtained by the image capturing unit;

an estimation unit configured to estimate a position and an orientation of the image capturing unit based on the captured image;

a first generation unit configured to generate an image of a virtual object based on the estimated position and orientation of the image capturing unit;

a moving velocity acquisition unit configured to acquire a current moving velocity of the image capturing unit and a limit of the moving velocity;

a second generation unit configured to generate an image representing the current moving velocity in contradistinction to the limit of the moving velocity; and

a display control unit configured to cause the display device to display a composite image obtained by combining the image of the virtual object and the image representing the current moving velocity in contradistinction to the limit of the moving velocity.

2. The information processing apparatus according to claim 1, further comprising a determination unit configured to determine the threshold, based on an image capturing parameter of the image capturing unit.

3. The information processing apparatus according to claim 2, wherein the image capturing parameter is a shutter speed of the image capturing unit.

4. The information processing apparatus according to claim 2, wherein the determination unit is configured to determine the threshold based on association information in which a measurement value of the image capturing parameter and the threshold are associated.

5. The information processing apparatus according to claim 4, further comprising a presentation unit configured to generate warning information when the measurement value exceeds the threshold, and present the warning information to a user monitoring the display device.

6. The information processing apparatus according to claim 5, wherein the presentation unit further combines an image representing the warning information with the composite image.

7. The information processing apparatus according to claim 4, wherein the presentation unit is configured to present the warning information to the user through voice.

8. The information processing apparatus according to claim 1, wherein the composite image is an image obtained by combining the captured image, the image of the virtual object generated by the first generation unit, and the image generated by the second generation unit.

9. The information processing apparatus according to claim 1, wherein the display device is an optical see-through or video see-through display device.

10. The information processing apparatus according to claim 1, wherein the moving velocity acquisition unit is a gyro sensor.

11. The information processing apparatus according to claim 1, wherein the display device is mounted to a part of a human body.

12. A system comprising:

a display device; and

an information processing apparatus connected to the display device,

wherein the display device includes;

an image capturing unit configured to capture an image of a real space;

a first transmission unit configured to transmit the image captured by the image capturing unit to the information processing apparatus; and

a display unit configured to receive the image transmitted from the information processing apparatus and display the image, and

wherein the information processing apparatus includes;

an image acquisition unit configured to acquire the captured image from the display device;

an estimation unit configured to estimate a position and an orientation of the image capturing unit based on the captured image;

a first generation unit configured to generate an image of a virtual object based on the estimated position and orientation of the image capturing unit;

a moving velocity acquisition unit configured to acquire a current moving velocity of the image capturing unit and a limit of the moving velocity;

a second generation unit configured to generate an image representing the current moving velocity in contradistinction to the limit of the moving velocity; and

a second transmission unit configured to transmit, to the display device, a composite image obtained by combining the image of the virtual object and the image representing the current moving velocity in contradistinction to the limit of the moving velocity;

a moving velocity acquisition unit configured to acquire a current moving velocity of the image capturing unit and a limit of the moving velocity;

a second generation unit configured to generate an image representing the current moving velocity in contradistinction to the limit of the moving velocity; and

a display control unit configured to cause the display device to display a composite image obtained by combining the image of the virtual object and the image representing the current moving velocity in contradistinction to the limit of the moving velocity.

13. An information processing method for generating an image to be displayed on a display device including an image capturing unit, the information processing method comprising:

acquiring a captured image obtained by the image capturing unit;

estimating a position and an orientation of the image capturing unit based on the captured image;

generating an image of a virtual object based on the estimated position and orientation of the image capturing unit;

acquiring a current moving velocity of the image capturing unit and a limit of the moving velocity;

generating an image representing the current moving velocity in contradistinction to the limit of the moving velocity; and

causing the display device to display a composite image obtained by combining the image of the virtual object and the image representing the current moving velocity in contradistinction to the limit of the moving velocity.

14. A storage medium storing a program for causing a computer to perform an information processing method for generating an image to be displayed on a display device including an image capturing unit, the information processing method comprising:

acquiring a captured image obtained by the image capturing unit;

estimating a position and an orientation of the image capturing unit based on the captured image;

generating an image of a virtual object based on the estimated position and orientation of the image capturing unit;

acquiring a current moving velocity of the image capturing unit and a limit of the moving velocity;

generating an image representing the current moving velocity in contradistinction to the limit of the moving velocity; and

causing the display device to display a composite image obtained by combining the image of the virtual object and the image representing the current moving velocity in contradistinction to the limit of the moving velocity.