Computer-readable recording medium, information processing method, and information processing apparatus

• US 10,681,269 B2
• Filed: 02/06/2017
• Issued: 06/09/2020
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Nakao, Manabu, Yamada, Moyuru, Hada, Yoshiro
• Primary Examiner(s): Perez-Fuentes, Luis
• Application Number:
  US15/425,279
• Publication Number:
  US 20170289447A1
• Time in Patent Office: 1,219 Days
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A non-transitory computer-readable recording medium stores therein an information processing program that causes a computer to execute a process including: acquiring images photographed with a first angle of view and a second angle of view wider than the first angle of view; specifying a position and an attitude of a camera that photographed the image photographed with the first angle of view based on the image photographed with the second angle of view; stitching a plurality of images photographed with the first angle of view to generate a panoramic image; correcting the position on the generated panoramic image, of the image photographed with the first angle of view based on the specified position and attitude of the camera; and mapping the panoramic image to a three-dimensional model by texture mapping based on the corrected position.

Claims

1. A non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process comprising:

acquiring a plurality of high-definition images photographed multiple times by a first camera with a first angle of view and wide-viewing-angle images photographed by a second camera that is an omnidirectional camera with a second angle of view wider than the first angle of view;

specifying a position and an attitude of the first camera that photographed the images photographed with the first angle of view based on the images photographed with the second angle of view by the second camera;

stitching the plurality of images photographed with the first angle of view to generate a panoramic image;

comparing central coordinates on the panoramic image, of the images photographed with the first angle of view, included in the panoramic image to extract the central coordinates of the images photographed with the first angle of view, positioned at edges of the panoramic image as reference points and correcting positions of the extracted reference points; and

mapping the panoramic image to a three-dimensional model by texture mapping based on the corrected position.

2. The non-transitory computer-readable recording medium according to claim 1, wherein

the specifying involves extracting feature points of the image photographed with the second angle of view and assigning reliability based on the extracted feature points to the specified position and attitude of the first camera, and

the central coordinates of the images photographed with the first angle of view are extracted based on the assigned reliability.

3. The non-transitory computer-readable recording medium according to claim 1, wherein

the correcting involves extracting central coordinates of the images photographed with the first angle of view, positioned at four corners of the panoramic image as reference points and correcting positions of the extracted reference points.

4. An information processing method implemented by a computer, the information processing method comprising:

acquiring a plurality of high-definition images photographed multiple times by a first camera with a first angle of view and wide-viewing-angle images photographed by a second camera that is an omnidirectional camera with a second angle of view wider than the first angle of view, using a processor;

specifying a position and an attitude of the first camera that photographed the images photographed with the first angle of view based on the images photographed with the second angle of view, using the processor;

stitching the plurality of images photographed with the first angle of view to generate a panoramic image, using the processor;

comparing central coordinates on the panoramic image, of the images photographed with the first angle of view, included in the panoramic image to extract the central coordinates of the images photographed with the first angle of view, positioned at edges of the panoramic image as reference points and correcting positions of the extracted reference points, using the processor; and

mapping the panoramic image to a three-dimensional model by texture mapping based on the corrected position, using the processor.

5. The information processing method according to claim 4, wherein

the specifying involves extracting feature points of the image photographed with the second angle of view and assigning reliability based on the extracted feature points to the specified position and attitude of the first camera, and

the central coordinates of the images photographed with the first angle of view are extracted based on the assigned reliability.

6. The information processing method according to claim 4, wherein

the correcting involves extracting central coordinates of the images photographed with the first angle of view, positioned at four corners of the panoramic image as reference points and correcting positions of the extracted reference points.

7. An information processing apparatus comprising:

a memory; and

a processor coupled to the memory, wherein the processor executes a process comprising;

acquiring a plurality of high-definition images photographed multiple times by a first camera with a first angle of view and wide-viewing-angle images photographed by a second camera that is an omnidirectional camera with a second angle of view wider than the first angle of view;

specifying a position and an attitude of the first camera that photographed the images photographed with the first angle of view based on the images photographed with the second angle of view;

stitching the plurality of images photographed with the first angle of view to generate a panoramic image;

comparing central coordinates on the panoramic image, of the images photographed with the first angle of view, included in the panoramic image to extract the central coordinates of the images photographed with the first angle of view, positioned at edges of the panoramic image as reference points and correcting positions of the extracted reference points; and

mapping the panoramic image to a three-dimensional model by texture mapping based on the corrected position of the image photographed with the first angle of view.

8. The information processing apparatus according to claim 7, wherein

the specifying involves extracting feature points of the image photographed with the second angle of view and assigns reliability based on the extracted feature points to the specified position and attitude of the first camera, and

the central coordinates of the images photographed with the first angle of view are extracted based on the assigned reliability.

9. The information processing apparatus according to claim 7, wherein

the correcting involves extracting central coordinates of the images photographed with the first angle of view, positioned at four corners of the panoramic image as reference points and corrects positions of the extracted reference points.