IMAGE PROCESSING SYSTEM AND CONTROL METHOD FOR THE SAME

• US 20150371444A1
• Filed: 06/04/2015
• Published: 12/24/2015
• Est. Priority Date: 06/18/2014
• Status: Abandoned Application

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Hara, Kazutoshi, Yamaguchi, Hiroichi, Takemoto, Kazuki
• Application Number:
  US14/730,667
• Publication Number:
  US 20150371444A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An image processing system includes an image processing apparatus that is wearable by a user and is configured to capture real space and display real space video. The system generates mixed reality video obtained by superimposing virtual object video on the real space video; identifies a display area of a real object that is included in the real space video; measures a distance between the image processing apparatus and the real object. In addition, the system performs notification for causing a user who is wearing the image processing apparatus to recognize existence of the real object, if the display area of the real object is hidden by the virtual object video, and the distance between the image processing apparatus and the real object is less than a predetermined distance.

Claims

1. An image processing system including an image processing apparatus that is wearable by a user and is configured to capture real space and display real space video, the system comprising:

a generation unit configured to generate mixed reality video obtained by superimposing virtual object video on the real space video;

an identification unit configured to identify a display area of a real object that is included in the real space video;

a measurement unit configured to measure a distance between the image processing apparatus and the real object; and

a notification unit configured to perform notification for causing a user who is wearing the image processing apparatus to recognize existence of the real object, if the display area of the real object is hidden by the virtual object video, and the distance between the image processing apparatus and the real object is less than a predetermined distance.

2. The image processing system according to claim 1, wherein the notification unit performs the notification, if all of the display area of the real object is hidden by the virtual object video, and the distance between the image processing apparatus and the real object is less than the predetermined distance.

3. The image processing system according to claim 1, wherein the notification unit causes the image processing apparatus to display a warning indicating that the real object exists, or sets the virtual object video to be displayed translucently or as a contour.

4. The image processing system according to claim 1, further comprising a velocity determination unit configured to determine a relative velocity of the real object relative to the image processing apparatus, based on a temporal change in the distance measured by the measurement unit,

wherein the notification unit performs the notification, if the display area of the real object is hidden by the virtual object video, the distance between the image processing apparatus and the real object is less than the predetermined distance, and the relative velocity determined by the velocity determination unit is greater than or equal to a predetermined velocity.

5. The image processing system according to claim 1, wherein the measurement unit is a depth sensor.

6. The image processing system according to claim 1, further comprising an image sensing unit for a left eye and an image sensing unit for a right eye,

wherein the measurement unit computes the distance based on a parallax between an image obtained by the image sensing unit for the left eye and an image obtained by the image sensing unit for the right eye.

7. The image processing system according to claim 1, wherein the measurement unit computes the distance based on a geographical position of the image processing apparatus and a geographical position of the real object that are acquired from an external apparatus.

8. The image processing system according to claim 1, further comprising a disable setting unit for disabling the notification by the notification unit.

9. A method for controlling an image processing system including an image processing apparatus that is wearable by a user and is configured to capture real space and display real space video, the method comprising:

generating mixed reality video obtained by superimposing virtual object video on the real space video;

identifying a display area of a real object that is included in the real space video;

measuring a distance between the image processing apparatus and the real object; and

performing notification for causing a user who is wearing the image processing apparatus to recognize existence of the real object, if the display area of the real object is hidden by the virtual object video, and the distance between the image processing apparatus and the real object is less than a predetermined distance.

10. An image processing system including an image processing apparatus that is wearable by a user and is configured to capture real space and display real space video, the system comprising:

a generation unit configured to generate mixed reality video obtained by superimposing virtual object video on the real space video;

an identification unit configured to identify a display area of a real object that is included in the real space video;

a measurement unit configured to measure a distance between an identified real object that is moving and a user who is wearing the image processing apparatus; and

a notification unit configured to perform notification for causing the user who is wearing the image processing apparatus to recognize existence of the real object that is moving, if the display area of the real object that is moving is hidden by the virtual object video, and the distance measured by the measurement unit is less than a predetermined distance.

11. The image processing system according to claim 10, wherein the notification unit performs the notification, if all of the display area of the real object that is moving is hidden by the virtual object video, and the distance measured by the measurement unit is less than the predetermined distance.

12. The image processing system according to claim 10, wherein the notification unit causes the image processing apparatus to display a warning indicating that the real object exists, or sets the virtual object video to be displayed translucently or as a contour.

13. The image processing system according to claim 10, further comprising a velocity determination unit configured to determine a relative velocity of the real object relative to the user, based on a temporal change in the distance measured by the measurement unit,

wherein the notification unit performs the notification, if the display area of the real object that is moving is hidden by the virtual object video, the distance measured by the measurement unit is less than the predetermined distance, and the relative velocity determined by the velocity determination unit is greater than or equal to a predetermined velocity.

14. The image processing system according to claim 10, wherein the measurement unit is a depth sensor.

15. The image processing system according to claim 10, further comprising an image sensing unit for a left eye and an image sensing unit for a right eye,

wherein the measurement unit computes the distance based on a parallax between an image obtained by the image sensing unit for the left eye and an image obtained by the image sensing unit for the right eye.

16. The image processing system according to claim 10, wherein the measurement unit computes the distance based on a geographical position of the image processing apparatus and a geographical position of the real object that are acquired from an external apparatus.

17. The image processing system according to claim 10, further comprising a disable setting unit for disabling the notification by the notification unit.

18. A method for controlling an image processing system including an image processing apparatus that is wearable by a user and is configured to capture real space and display real space video, the method comprising:

generating mixed reality video obtained by superimposing virtual object video on the real space video;

identifying a display area of a real object that is included in the real space video;

measuring a distance between an identified real object that is moving and a user who is wearing the image processing apparatus; and

performing notification for causing the user who is wearing the image processing apparatus to recognize existence of the real object that is moving, if the display area of the real object that is moving is hidden by the virtual object video, and the measured distance is less than a predetermined distance.