Display control system, display control device, display control method, and program

• US 10,742,947 B2
• Filed: 03/30/2015
• Issued: 08/11/2020
• Est. Priority Date: 03/30/2015
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Inventors: Mujibiya, Adiyan, Yamashita, Shogo
• Primary Examiner(s): Chowdhury, Nigar
• Application Number:
  US15/323,968
• Publication Number:
  US 20170201733A1
• Time in Patent Office: 1,961 Days
• Field of Search: 386 43, 348 43
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/349 Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking for viewing without the aid of special glasses using fly-eye lenses H04N13/307
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

Labor of photographing images of an object can be reduced in a case where the images of the object viewed from a viewpoint moving along with an orbit are displayed. Image obtaining means of a display control system obtains a plurality of image data pieces generated by photographing an object from a plurality of photographing positions that are different from one another in photographing directions. Information obtaining means obtains photograph information relating to the photographing positions of the respective image data pieces in a three-dimensional space based on the image data pieces. Image selecting means selects some of the image data pieces based on the photograph information of the respective image data pieces and orbit information relating to an orbit of a viewpoint that moves while changing a viewing direction in the three-dimensional space. Display control means displays, on display means, the image data pieces selected by the image selecting means in an order according to the orbit.

Claims

1. A display control system comprising at least one processor, wherein the at least one processor:

obtains a plurality of image data pieces generated by photographing an object from a plurality of photographing positions that are different from one another in photographing directions;

detects three-dimensional Cartesian coordinates of a plurality of feature points regarding the object based on at least one of the image data pieces;

wherein the feature points are different points on the object;

obtains photograph information relating to the photographing positions of the respective image data pieces in a three-dimensional space based on the image data pieces, the photograph information being obtained using positional relationships of the feature points;

calculates trajectory information relating to a trajectory of a viewpoint that moves while changing a viewing direction in the three-dimensional space based on three-dimensional Cartesian coordinates of a feature point designated from the plurality of feature points;

wherein the trajectory information includes trajectory position information of the viewpoint and viewing direction information of the viewpoint from the trajectory,

wherein the calculation includes calculating a rotation axis perpendicular to a line connecting one of the photographing positions to one of the feature points;

selects some of the image data pieces based on the photograph information of the respective image data pieces and the trajectory information; and

displays, on a display, the selected image data pieces in an order according to the trajectory.

2. The display control system according to claim 1,

wherein the at least one processor calculates the trajectory information about the viewpoint that moves while fixating on the three-dimensional Cartesian coordinates of the designated feature point.

3. The display control system according to claim 1,

wherein the at least one processor determines whether the designated feature point is photographed in the respective image data pieces based on the image data pieces, and

selects some of the image data pieces based on the photograph information of the respective image data pieces, the trajectory information, and a determination result.

4. The display control system according to claim 1,

wherein the photograph information of the respective image data pieces comprises photographing position information of the respective image data pieces,

wherein the trajectory information comprises trajectory position information of the viewpoint, and

wherein the at least one processor selects some of the image data pieces based on distances among positions indicated by the photographing position information of the respective image data pieces and positions indicated by the trajectory position information.

5. The display control system according to claim 4,

wherein the photograph information of the respective image data pieces comprises photographing direction information of the respective image data pieces,

wherein the trajectory information comprises viewing direction information about the viewpoint from the trajectory, and

wherein the at least one processor selects some of the image data pieces based on deviations of directions indicated by the photographing direction information of the respective image data pieces from directions indicated by the viewing direction information, and calculates evaluation values weighing the deviations more than the distances based on the distances and the deviations of the respective image data pieces, and selects some of the image data pieces based on the evaluation values of the respective image data pieces.

6. The display control system according to claim 1,

wherein the photograph information of the respective image data pieces comprises photographing direction information of the respective image data pieces,

wherein the trajectory information comprises viewing direction information about the viewpoint from the trajectory, and

wherein the at least one processor selects some of the image data pieces based on deviations of directions indicated by the photographing direction information of the respective image data pieces from directions indicated by the viewing direction information.

7. The display control system according to claim 1,

wherein the photograph information of the respective image data pieces comprises the photographing position information of the respective image data pieces, and

wherein the at least one processor selects some of the image data pieces such that the distances of the positions indicated by the photographing position information of the selected respective image data pieces are within a predetermined range based on the photographing position information of the respective image data pieces.

8. The display control system according to claim 1,

wherein the at least one processor determines at least either of display positions and display sizes of the selected respective image data pieces such that at least either of a deviation of the display position and a deviation of the display size of the object displayed on the display is within a predetermined range.

9. A display control device comprising at least one processor, wherein the at least one processor:

obtains content stored in a storage that stores a plurality of image data pieces and photograph information in association with one another, the plurality of image data pieces being generated by photographing an object from a plurality of photographing positions that are different from one another in photographing directions, and the photograph information being related to the photographing positions of the respective image data pieces in a three-dimensional space;

detects three-dimensional Cartesian coordinates of a plurality of feature points regarding the object based on at least one of the image data pieces, wherein the photograph information is obtained using positional relationships of the feature points;

wherein the feature points are different points on the object;

calculates trajectory information relating to a trajectory of a viewpoint that moves while changing a viewing direction in the three-dimensional space based on three-dimensional Cartesian coordinates of a feature point designated from the plurality of feature points;

wherein the trajectory information includes trajectory position information of the viewpoint and viewing direction information of the viewpoint from the trajectory,

wherein the calculation includes calculating a rotation axis perpendicular to a line connecting one of the photographing positions to one of the feature points;

selects some of the image data pieces based on the photograph information of the respective image data pieces and the trajectory information; and

display, on a display, the selected image data pieces in an order according to the trajectory.

10. A display control method comprising:

obtaining a plurality of image data pieces generated by photographing an object from a plurality of photographing positions that are different from one another in photographing directions;

detecting three-dimensional Cartesian coordinates of a plurality of feature points regarding the object based on at least one of the image data pieces;

wherein the feature points are different points on the object;

obtaining photograph information relating to the photographing positions of the respective image data pieces in a three-dimensional space based on the image data pieces, the photograph information being obtained using positional relationships of the feature points;

calculating trajectory information relating to a trajectory of a viewpoint that moves while changing a viewing direction in the three-dimensional space based on three-dimensional Cartesian coordinates of a feature point designated from the plurality of feature points;

wherein the trajectory information includes trajectory position information of the viewpoint and viewing direction information of the viewpoint from the trajectory,

wherein the calculation includes calculating a rotation axis perpendicular to a line connecting one of the photographing positions to one of the feature points;

selecting some of the image data pieces based on the photograph information of the respective image data pieces and the trajectory information; and

displaying, on a display, the selected image data pieces in an order according to the trajectory.

11. A non-transitory computer-readable information storage medium having stored thereon a program for causing a computer to:

obtain content stored in a storage that stores a plurality of image data pieces and photograph information in association with one another, the plurality of image data pieces being generated by photographing an object from a plurality of photographing positions that are different from one another in photographing directions, and the photograph information being related to the photographing positions of the respective image data pieces in a three-dimensional space;

detects three-dimensional Cartesian coordinates of a plurality of feature points regarding the object based on at least one of the image data pieces, wherein the photograph information is obtained using positional relationships of the feature points;

wherein the feature points are different points on the object;

calculate trajectory information relating to a trajectory of a viewpoint that moves while changing a viewing direction in the three-dimensional space based on three-dimensional Cartesian coordinates of a feature point designated from the plurality of feature points;

wherein the trajectory information includes trajectory position information of the viewpoint and viewing direction information of the viewpoint from the trajectory,

wherein the calculation includes calculating a rotation axis perpendicular to a line connecting one of the photographing positions to one of the feature points;

select some of the image data pieces based on the photograph information of the respective image data pieces and trajectory information relating to an trajectory of a viewpoint that moves while changing a viewing direction in the three-dimensional space; and

display, on a display, the selected image data pieces in an order according to the trajectory.