SUPERIMPOSED DISPLAY METHOD AND SUPERIMPOSED DISPLAY APPARATUS

• US 20170132752A1
• Filed: 10/12/2016
• Published: 05/11/2017
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: AOYAGI, Tomohiro, NUMATA, Yojiro
• Granted Patent: US 10,304,159 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 3/147 using affine transformations
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

A computer displays a plurality of edge lines extracted from a shot image of a spatial structure that has been shot by an image shooting apparatus, and a plurality of ridge lines included in a model image represented by model information of the spatial structure, in a manner such that the edge lines and the ridge lines are selectable. Then, the computer accepts a selection instruction indicating an edge line and a ridge line of an overlay target. Finally, in accordance with the accepted selection instruction, the computer displays a superimposition image in which the model image has been superimposed onto the shot image in a manner such that the edge line and the ridge line of the overlay target overlap each other.

Claims

1. A superimposed display method comprising:

displaying, by a computer, a plurality of edge lines extracted from a shot image of a spatial structure that has been shot by an image shooting apparatus, and a plurality of ridge lines included in a model image represented by model information of the spatial structure, in a manner such that the edge lines and the ridge lines are selectable;

accepting, by the computer, a selection instruction indicating an edge line and a ridge line of an overlay target; and

in accordance with the selection instruction, displaying, by the computer, a superimposition image in which the model image has been superimposed onto the shot image in a manner such that the edge line and the ridge line of the overlay target overlap each other.

2. The superimposed display method according to claim 1, wherein

the overlay target includes a plurality of combinations of edge lines and ridge lines, each of the plurality of combinations including an edge line and a ridge line, and

the displaying the superimposition image superimposes the model image onto the shot image in a manner such that an edge line and a ridge line included in each of the plurality of combinations overlap each other.

3. The superimposed display method according to claim 2, wherein

the selection instruction includes a priority for each of the plurality of combinations, and

the displaying the superimposition image sequentially selects the plurality of combinations one by one in accordance with the priority, and superimposes the model image onto the shot image in a manner such that an edge line and a ridge line included in each selected combination overlap each other.

4. The superimposed display method according to claim 1, further comprising:

in accordance with the selection instruction, displaying a line segment linking the edge line and the ridge line of the overlay target before displaying the superimposition image.

5. The superimposed display method according to claim 1, further comprising:

in accordance with the selection instruction,

highlighting the edge line and the ridge line of the overlay target before displaying the superimposition image.

6. The superimposed display method according to claim 5, further comprising:

cancelling the highlighting of the edge line and the ridge line of the overlay target when displaying the superimposition image.

7. The superimposed display method according to claim 1, further comprising:

highlighting the edge line and the ridge line of the overlay target when displaying the superimposition image.

8. A superimposed display method comprising:

displaying, by a computer, a shot image of a spatial structure that has been shot by an image shooting apparatus, and a model image represented by model information of the spatial structure; and

before and after accepting an instruction to display a superimposition image in which the model image has been superimposed onto the shot image in a manner such that a reference position on the shot image and a reference position on the model image overlap each other, switching, by the computer, between a rule that separate change instructions be accepted for the shot image and the model image, and a rule that a common change instruction be accepted for the shot image and the model image, the separate and common change instructions indicating enlargement, reduction, movement, or rotation.

9. A superimposed display apparatus comprising:

a memory; and

a processor operatively coupled to the memory and the processor configured to;

display a plurality of edge lines extracted from a shot image of a spatial structure that has been shot by an image shooting apparatus, and a plurality of ridge lines included in a model image represented by model information of the spatial structure, in a manner such that the edge lines and the ridge lines are selectable;

accept a selection instruction indicating an edge line and a ridge line of an overlay target; and

in accordance with the selection instruction, display a superimposition image in which the model image has been superimposed onto the shot image in a manner such that the edge line and the ridge line of the overlay target overlap each other.

10. A superimposed display apparatus comprising:

a memory; and

a processor operatively coupled to the memory and the processor configured to;

display a shot image of a spatial structure that has been shot by an image shooting apparatus, and a model image represented by model information of the spatial structure; and

before and after accepting an instruction to display a superimposition image in which the model image has been superimposed onto the shot image in a manner such that a reference position on the shot image and a reference position on the model image overlap each other, switch between a rule that separate change instructions be accepted for the shot image and the model image, and a rule that a common change instruction be accepted for the shot image and the model image, the separate and common change instructions indicating enlargement, reduction, movement, or rotation.

11. A non-transitory computer-readable recording medium having stored therein a superimposed display program that causes a computer to execute a process comprising:

displaying a plurality of edge lines extracted from a shot image of a spatial structure that has been shot by an image shooting apparatus, and a plurality of ridge lines included in a model image represented by model information of the spatial structure, in a manner such that the edge lines and the ridge lines are selectable;

accepting a selection instruction indicating an edge line and a ridge line of an overlay target; and

in accordance with the selection instruction, displaying a superimposition image in which the model image has been superimposed onto the shot image in a manner such that the edge line and the ridge line of the overlay target overlap each other.

12. A non-transitory computer-readable recording medium having stored therein a superimposed display program that causes a computer to execute a process comprising:

displaying a shot image of a spatial structure that has been shot by an image shooting apparatus, and a model image represented by model information of the spatial structure; and

before and after accepting an instruction to display a superimposition image in which the model image has been superimposed onto the shot image in a manner such that a reference position on the shot image and a reference position on the model image overlap each other, switching between a rule that separate change instructions be accepted for the shot image and the model image, and a rule that a common change instruction be accepted for the shot image and the model image, the separate and common change instructions indicating enlargement, reduction, movement, or rotation.