MOBILE ELECTRONIC DEVICE, VIRTUAL INFORMATION DISPLAY METHOD AND STORAGE MEDIUM STORING VIRTUAL INFORMATION DISPLAY PROGRAM

• US 20120218257A1
• Filed: 02/21/2012
• Published: 08/30/2012
• Est. Priority Date: 02/24/2011
• Status: Abandoned Application

• Current Assignee: Kyocera Corporation
• Original Assignee: Kyocera Corporation
• Inventors: HISANO, Shuhei
• Application Number:
  US13/400,891
• Publication Number:
  US 20120218257A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

According to an aspect, a mobile electronic device includes a detecting unit, an imaging unit, a display unit, and a control unit. The control unit calculates, based on a first image in which a marker is taken by the imaging unit, a reference position that is a relative position of the real marker when seen from the mobile electronic device. When causing the display unit to display a second image taken by the imaging unit, the control unit calculates a predicted position that is a position at which the marker is predicted to exist in taking the second image, based on a position of the mobile electronic device acquired based on a result detected by the detecting unit in taking the first image, a position of the mobile electronic device acquired based on a result detected by the detecting unit in taking the second image, and the reference position.

Claims

1. A mobile electronic device comprising:

a detecting unit configured to detect a change in a position and attitude of the mobile electronic device;

an imaging unit;

a display unit configured to display an image taken by the imaging unit; and

a control unit configured to;

calculate, based on a first image in which a marker placed at a certain position and having a predetermined size and form is taken by the imaging unit, a reference position that is a relative position of the real marker when seen from the mobile electronic device; and

calculate, when causing the display unit to display a second image taken by the imaging unit, a predicted position that is a position at which the marker is predicted to exist in taking the second image, based on a position of the mobile electronic device acquired based on a result detected by the detecting unit in taking the first image, a position of the mobile electronic device acquired based on a result detected by the detecting unit in taking the second image, and the reference position, and superimpose virtual information corresponding to the marker at a position corresponding to the predicted position of the second image for display.

2. The mobile electronic device according to claim 1,

wherein the control unit is configured to superimpose the virtual information at a position corresponding to the predicted position of the second image in size matched with the predicted position for display when causing the display unit to display the second image.

3. The mobile electronic device according to claim 1, wherein

the control unit is configured to:

calculate a reference attitude that is a relative attitude of the real marker when seen from the mobile electronic device based on the first image; and

calculate, when the causing display unit to display the second image, a predicted attitude of the marker in taking the second image based on a position and attitude of the mobile electronic device acquired based on a result detected by the detecting unit in taking the first image, a position and attitude of the mobile electronic device acquired based on a result detected by the detecting unit in taking the second image, and the reference attitude, and superimpose the virtual information whose attitude is set based on the predicted attitude at a position corresponding to the predicted position of the second image for display.

4. The mobile electronic device according to claim 1,

wherein the control unit is configured to cause the display unit to display a guide indicating a direction in which the predicted position exists when the predicted position is at a position at which the virtual information is not superimposed on the second image.

5. The mobile electronic device according to claim 1, further comprising

an operation unit configured to accept an operation,

wherein the control unit is configured to change the virtual information according to an operation accepted by the operation unit for display.

6. The mobile electronic device according to claim 5,

wherein the control unit is configured to change size of the virtual information according to an operation accepted by the operation unit for display.

7. The mobile electronic device according to claim 5,

wherein the control unit is configured to change a position of the virtual information according to an operation accepted by the operation unit for display.

8. The mobile electronic device according to claim 1, further comprising

a communication unit configured to communicate with an apparatus,

wherein the control unit is configured to acquire the virtual information from the apparatus via communications by the communication unit.

9. The mobile electronic device according to claim 1, wherein

the virtual information is a three-dimensional object created based on three-dimensional model data, and

the control unit is configured to create, when superimposing the virtual information on the second image for display, the virtual information based on the three-dimensional model data acquired in advance.

10. A virtual information display method executed by a mobile electronic device that includes a detecting unit, an imaging unit, and a display unit, the virtual information display method comprising:

taking a first image, in which a marker placed at a certain position and having a predetermined size and form is taken, by the imaging unit;

detecting a first position of the mobile electronic device in taking the first image by the detecting unit;

calculating, based on the first image, a reference position that is a relative position of the real marker when seen from the mobile electronic device;

taking a second image by the imaging unit;

displaying the second image on the display unit;

detecting a second position of the mobile electronic device in taking the first image by the detecting unit;

calculating a predicted position that is a position at which the marker is predicted to exist in taking the second image, based on the first position of the mobile electronic device, the second position of the mobile electronic device, and the reference position; and

superimposing virtual information corresponding to the marker at a position corresponding to the predicted position of the second image for display.

11. A non-transitory storage medium that stores a virtual information display program for causing, when executed by a mobile electronic device that includes a detecting unit, an imaging unit, and a display unit, the mobile electronic device to execute:

taking a first image, in which a marker placed at a certain position and having a predetermined size and form is taken, by the imaging unit;

detecting a first position of the mobile electronic device in taking the first image by the detecting unit;

calculating, based on the first image, a reference position that is a relative position of the real marker when seen from the mobile electronic device;

taking a second image by the imaging unit;

displaying the second image on the display unit;

detecting a second position of the mobile electronic device in taking the first image by the detecting unit;

calculating a predicted position that is a position at which the marker is predicted to exist in taking the second image, based on the first position of the mobile electronic device, the second position of the mobile electronic device, and the reference position; and

superimposing virtual information corresponding to the marker at a position corresponding to the predicted position of the second image for display.