INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND PROGRAM

• US 20150145888A1
• Filed: 04/24/2013
• Published: 05/28/2015
• Est. Priority Date: 06/12/2012
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hanai, Yuya
• Application Number:
  US14/405,262
• Publication Number:
  US 20150145888A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/293 Generating mixed stereoscopic images; Generating mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

There is provided an information processing device including an acquisition section configured to acquire a captured image captured by an imaging section, and a data processing section configured to superimposedly display a virtual image generated by changing an input image on the captured image in a display section. The data processing section displays, on the display section, the virtual image generated by changing one of a relative position and a relative angle of the imaging section and the input image, which are virtually set, in a time series.

Claims

1. An information processing device comprising:

an acquisition section configured to acquire a captured image captured by an imaging section; and

a data processing section configured to superimposedly display a virtual image generated by changing an input image on the captured image in a display section,

wherein the data processing section displays, on the display section, the virtual image generated by changing one of a relative position and a relative angle of the imaging section and the input image, which are virtually set, in a time series.

2. The information processing device according to claim 1,

wherein the data processing section generates the virtual image, of which at least one of the display position and the display angle is changed in a time series, by performing image conversion processing to which metadata set in association with each frame of a moving image content of the virtual image is applied.

3. The information processing device according to claim 2,

wherein the data processing section acquires parameters, including a relative position parameter Rpos to be applied for determining the display position of the virtual image and a relative angle parameter Rrot to be applied for determining the display angle of the virtual image, as metadata set in association with each frame of the moving image content of the virtual image, applies the acquired parameters, generates the virtual image, of which at least one of the display position and the display angle is changed in a time series, and superimposedly displays the virtual image on the captured image.

4. The information processing device according to claim 3,

wherein the data processing section calculates a transform parameter that transforms a model view matrix Mmarker corresponding to an initial image of the virtual image into a model view matrix Mdisplay corresponding to a last image of the virtual image, multiplies the relative position parameter Rpos or the relative angle parameter Rrot, which are metadata set corresponding to each virtual image frame, with respect to the calculated transform parameter, calculates offset information to be applied to conversion processing of each virtual image frame, performs conversion processing of the virtual image frame to which the calculated offset information is applied, and generates the moving image of the virtual image of which at least one of the display position and the display angle is changed in a time series.

5. The information processing device according to claim 4,

wherein each of the relative position parameter Rpos and the relative angle parameter Rrot is a value that is sequentially changed in a range of 0 to 1 in each moving image frame from the initial image to the last image of the virtual image.

6. The information processing device according to claim 1,

wherein the data processing section performs blurring process on the virtual image of a boundary portion between the virtual image and the captured image upon the processing of superimposedly displaying the virtual image on the captured image.

7. The information processing device according to claim 1,

wherein the data processing section performs processing of generating moving image content of the virtual image, calculates parameters, including a relative position parameter Rpos to be applied for determining the display position of the virtual image and a relative angle parameter Rrot to be applied for determining the display angle of the virtual image, with respect to frames constituting the moving image content of the virtual image upon processing of generating the moving image content of the virtual image, and stores the calculated parameters in a storage section as metadata corresponding to image frames.

8. The information processing device according to claim 7,

wherein the data processing section sets values of the relative position parameter Rpos and the relative angle parameter Rrot in a range of 0 to 1 according to a subject distance of the virtual image in each moving image frame of the moving image content.

9. The information processing device according to claim 7,

wherein the data processing section sets values, which are sequentially changed in a range of 0 to 1 with respect to each moving image frame from an initial image to a last image of the virtual image, as values of the relative position parameter Rpos and the relative angle parameter Rrot.

10. The information processing device according to claim 9,

wherein the data processing section sets values of the relative position parameter Rpos and the relative angle parameter Rrot with respect to each moving image frame constituting the virtual image in different aspects according to a mode setting.

11. The information processing device according to claim 7,

wherein the data processing section outputs restriction information indicating an angle and a distance of a preset allowable range to a display section upon processing of generating moving image content of the virtual image, and generates moving image content of the virtual image including a virtual object within the angle and the distance of the allowable range.

12. An information processing device comprising:

an imaging section configured to performing image capturing; and

a data processing section configured to generate moving image content of a virtual image, based on a captured image of the imaging section,

wherein the data processing section calculates parameters, including a relative position parameter Rpos to be applied for determining the display position of the virtual image and a relative angle parameter Rrot to be applied for determining the display angle of the virtual image, with respect to frames constituting the moving image content of the virtual image upon processing of generating the moving image content of the virtual image, and stores the calculated parameters in a storage section as metadata corresponding to image frames.

13. The information processing device according to claim 12,

wherein the data processing section sets values of the relative position parameter Rpos and the relative angle parameter Rrot in a range of 0 to 1 according to a subject distance of the virtual image in each moving image frame of the moving image content.

14. An information processing device comprising:

an imaging section configured to performing image capturing;

a display section configured to display a captured image of the imaging section; and

a data processing section configured to superimposedly display a virtual image on the captured image displayed on the display section,

wherein the data processing section acquires, from a server, a moving image of the virtual image, of which at least one of a display position and a display angle is changed in a time series, and superimposedly displays the acquired virtual image on the captured image.

15. An information processing method, which is performed by an information processing device,

wherein the information processing device comprises an acquisition section configured to acquire a captured image captured by an imaging section, and a data processing section configured to superimposedly display a virtual image generated by changing an input image on a display section, and

wherein the data processing section displays, on the display section, the virtual image generated by changing one of a relative position and a relative angle of the imaging section and the input image, which are virtually set, in a time series.

16. An information processing method, which is performed by an information processing device,

wherein the information processing device comprises an imaging section configured to perform image capturing, and a data processing section configured to generate moving image content of a virtual image, based on a captured image of the imaging section, and

wherein the data processing section calculates parameters, including a relative position parameter Rpos to be applied for determining the display position of the virtual image and a relative angle parameter Rrot to be applied for determining the display angle of the virtual image, with respect to frames constituting the moving image content of the virtual image upon processing of generating the moving image content of the virtual image, and stores the calculated parameters in a storage section as metadata corresponding to image frames.

17. A program for causing an information processing device to execute information processing,

wherein the information processing device comprises:

an acquisition section configured to acquire a captured image captured by an imaging section; and

a data processing section configured to superimposedly display a virtual image generated by changing an input image on the captured image in a display section, and

wherein the program causes the data processing section to display, on the display section, the virtual image generated by changing one of a relative position and a relative angle of the imaging section and the input image, which are virtually set, in a time series.

18. A program for causing an information processing device to execute information processing,

wherein the information processing device comprises an imaging section configured to perform image capturing, and a data processing section configured to generate moving image content of a virtual image, based on a captured image of the imaging section, and

wherein the program causes the data processing section to calculate parameters, including a relative position parameter Rpos to be applied for determining the display position of the virtual image and a relative angle parameter Rrot to be applied for determining the display angle of the virtual image, with respect to frames constituting the moving image content of the virtual image upon processing of generating the moving image content of the virtual image, and store the calculated parameters in a storage section as metadata corresponding to image frames.