AUTHORING DEVICE AND AUTHORING METHOD

• US 20170316612A1
• Filed: 04/20/2017
• Published: 11/02/2017
• Est. Priority Date: 04/28/2016
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: MOTEKI, Atsunori, YAMAGUCHI, Nobuyasu, YOSHITAKE, Toshiyuki
• Granted Patent: US 10,438,412 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method includes specifying a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point, correcting the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map, and controlling a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

Claims

1. A method executed by a computer, the method comprising:

specifying a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point;

correcting the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map; and

controlling a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

2. The method according to claim 1, wherein the correcting includes:

calculating a first intersection of the capturing direction and an approximate plane based on a part of the plurality of map points defined in the second map, the part of the plurality of map points being located around a specific map point which is associated with the virtual object, and

correcting the position of the virtual object to a position of the first intersection.

3. The method according to claim 2, wherein the correcting includes:

calculating a second intersection of the capturing direction and a plane that passes through a center of gravity of the approximate plane and is perpendicular to the approximate plane when an angle formed by a line normal to the approximate plane and the capturing direction is close to a right angle, and

correcting the position of the virtual object to a position of the second intersection.

4. The method according to claim 1, wherein

the correcting includes determining whether the position of the virtual object is to be corrected based on an error between a position of a first map point of the first map and a second position of a second map point of the second map, and

the first map point and the second map point are corresponding each other and located around the position of the virtual object.

5. The method according to claim 1, wherein the correcting includes determining whether the position of the virtual object is to be corrected, based on a density of map points of the first map, which is located around the position of the virtual object.

6. The method according to claim 1, further comprising:

determining a plurality of keyframes from among frames photographed by the camera, the plurality of keyframes including a first keyframe and a second keyframe that follows the first keyframe; and

generating the second map, based on the first keyframe and the second keyframe.

7. The method according to claim 1, further comprising:

determining a plurality of keyframes from among frames photographed by the camera; and

generating the second map, based on the plurality of keyframes, by bundle adjustment.

8. The method according to claim 1, further comprising:

determining a keyframe from among frames photographed by the camera; and

generating the second map by finely adjusting a position of a map point of the first map based on the keyframe and a frame from among the frames which follows the keyframe.

9. A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process, the process comprising:

specifying a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point;

correcting the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map; and

controlling a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

10. A device comprising:

a memory; and

a processor coupled to the memory and configured to;

specify a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point,

correct the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map, and

control a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

11. The device according to claim 10, wherein the processor is configured to:

calculate a first intersection of the capturing direction and an approximate plane based on a part of the plurality of map points defined in the second map, the part of the plurality of map points being located around a specific map point which is associated with the virtual object, and

correct the position of the virtual object to a position of the first intersection.

12. The device according to claim 11, wherein the processor is configured to:

calculate a second intersection of the capturing direction and a plane that passes through a center of gravity of the approximate plane and is perpendicular to the approximate plane when an angle formed by a line normal to the approximate plane and the capturing direction is close to a right angle, and

correct the position of the virtual object to a position of the second intersection.

13. The device according to claim 10, wherein

the processor is configured to determine whether the position of the virtual object is to be corrected based on an error between a position of a first map point of the first map and a second position of a second map point of the second map, and

the first map point and the second map point are corresponding each other and located around the position of the virtual object.

14. The device according to claim 10, wherein the processor is configured to determine whether the position of the virtual object is to be corrected, based on a density of map points of the first map, which is located around the position of the virtual object.

15. The device according to claim 10, wherein the processor is configured to:

determine a plurality of keyframes from among frames photographed by the camera, the plurality of keyframes including a first keyframe and a second keyframe that follows the first keyframe, and

generate the second map, based on the first keyframe and the second keyframe.

16. The device according to claim 10, wherein the processor is configured to:

determine a plurality of keyframes from among frames photographed by the camera, and

generate the second map, based on the plurality of keyframes, by bundle adjustment.

17. The device according to claim 10, wherein the processor is configured to:

determine a keyframe from among frames photographed by the camera, and

generate the second map by finely adjusting a position of a map point of the first map based on the keyframe and a frame from among the frames which follows the keyframe.