GENERATION DEVICE AND GENERATION METHOD, AND REPRODUCTION DEVICE AND REPRODUCTION METHOD

• US 20200137417A1
• Filed: 06/12/2018
• Published: 04/30/2020
• Est. Priority Date: 06/26/2017
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: IZUMI, Nobuaki
• Application Number:
  US16/624,524
• Publication Number:
  US 20200137417A1
• Time in Patent Office: Days
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/39 involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices

Abstract

The present disclosure relates to a generation device and a generation method, and a reproduction device and a reproduction method enabled to provide a mapping that is applicable to an arbitrary mapping method and in which high resolution is set in a viewing direction. The generation device includes a normalization unit that converts a first vector that maps a 360-degree image onto a predetermined 3D model into a second vector of a 3D model of a unit sphere. The present disclosure can be applied to, for example, a generation device that generates a 360-degree image from captured images in six directions, and generates an encoded stream in which high resolution is set in a predetermined direction of the 360-degree image, and the like.

Claims

1. A generation device comprising

a normalization unit that converts a first vector that maps a 360-degree image onto a predetermined 3D model into a second vector of a 3D model of a unit sphere.

2. The generation device according to claim 1, further comprising

a correction unit that adds a predetermined offset position vector to the second vector of the 3D model of the unit sphere converted.

3. The generation device according to claim 2, wherein

the correction unit adds the offset position vector that is negative to the second vector.

4. The generation device according to claim 2, wherein

the correction unit adds the offset position vector to the second vector multiplied by a positive constant.

5. The generation device according to claim 1, further comprising

a generation unit that generates the first vector.

6. The generation device according to claim 1, further comprising

a rotation processing unit that rotates a captured image to be converted into the 360-degree image such that a predetermined direction is at an image center.

7. The generation device according to claim 1, wherein

there is a plurality of high resolution directions each being a direction in which high resolution is set for the 360-degree image, and

a number of the normalization units are provided corresponding to a number of the high resolution directions.

8. The generation device according to claim 1, further comprising

a setting unit that determines a high resolution improvement ratio in a high resolution direction that is a direction is which high resolution is set for the 360-degree image.

9. The generation device according to claim 1, further comprising

a setting unit that generates information specifying a high resolution direction that is a direction in which high resolution is set for the 360-degree image.

10. A generation method comprising

converting a first vector of a predetermined 3D model onto which a 360-degree image is mapped into a second vector of a 3D model of a unit sphere.

11. A reproduction device comprising:

a receiving unit that receives a 360-degree image generated by another device; and

a normalization unit that converts a first vector that maps the 360-degree image onto a predetermined 3D model into a second vector of a 3D model of a unit sphere.

12. The reproduction device according to claim 11, further comprising

a correction unit that adds a predetermined offset position vector to the second vector of the 3D model of the unit sphere converted.

13. The reproduction device according to claim 12, wherein

the correction unit adds the offset position vector that is negative to the second vector.

14. The reproduction device according to claim 12, wherein

the correction unit adds the offset position vector to the second vector multiplied by a positive number.

15. The reproduction device according to claim 11, further comprising

a generation unit that generates the first vector.

16. The reproduction device according to claim 11, further comprising

a selection unit that selects one 360-degree image from among a plurality of 360-degree images having different high resolution directions, depending on a gaze direction of a viewer, wherein

the receiving unit receives the 360-degree image selected by the selection unit from the other device.

17. The reproduction device according to claim 11, further comprising

a rotation calculation unit that generates rotation information on the 360-degree image on a basis of information specifying a high resolution direction of the 360-degree image.

18. The reproduction device according to claim 17, further comprising

a drawing unit that generates a display image key rotating a 3D model image onto which the 360-degree image is mapped in accordance with the second vector of the 3D model of the unit sphere on a basis of the rotation information, and perspectively projecting the 3D model image onto a visual field range of a viewer.

19. The reproduction device according to claim 18, wherein

the rotation calculation unit calculates, as movement information, an amount of offset from a center of the unit sphere, and

the drawing unit generates the display image by rotating the 3D model image on a basis of the rotation information, of a viewing position in accordance with the movement information, and perspectively projecting the 3D model image, from the viewing position after offset movement, onto the visual field range of the viewer.

20. A reproduction method comprising:

receiving a 360-degree image generated by another device, and

converting a first vector that maps the 360-degree image onto a predetermined 3D model into a second vector of a 3D model of a unit sphere.