INFORMATION PROCESSING DEVICE AND METHOD

• US 20190313121A1
• Filed: 12/14/2017
• Published: 10/10/2019
• Est. Priority Date: 12/28/2016
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: ESHIMA, MASASHI
• Granted Patent: US 11,019,362 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The present disclosure relates to an information processing device and an information processing method that enable suppression of a reduction in coding efficiency while suppressing a reduction in subjective quality. Information regarding a three-dimensional region is encoded on the basis of a distribution related to overlapping of visual fields of the three-dimensional region to be imaged by a plurality of imaging units, the distribution being specified by using a parameter relating to overlapping of visual fields that are imaging ranges of the plurality of imaging units. The present disclosure can be applied to, for example, an information processing device, an encoding device, and the like.

Claims

1. An information processing device comprising

an encoding unit that encodes information regarding a three-dimensional region on a basis of a distribution related to overlapping of visual fields of the three-dimensional region to be imaged by a plurality of imaging units, the distribution being specified by using a parameter relating to overlapping of visual fields that are imaging ranges of the plurality of imaging units.

2. The information processing device according to claim 1, wherein

the encoding unit further includes a weighting unit that performs weighting of the three-dimensional region by using the parameter.

3. The information processing device according to claim 2, wherein

the weighting unit individually performs weighting of a voxel that is a unit region of a predetermined size.

4. The information processing device according to claim 3, wherein

the weighting unit sets a larger weight for a voxel including a subject.

5. The information processing device according to claim 3, further comprising

a subject determination unit that determines whether or not a subject is included, for each voxel, on a basis of the parameter, wherein

the weighting unit is configured to set a larger weight for a voxel determined by the subject determination unit that the subject is included, than for a voxel determined by the subject determination unit that the subject is not included.

6. The information processing device according to claim 5, wherein

the subject determination unit detects a peak voxel in which the parameter is larger than a first threshold value, detects and holds a voxel in which the parameter is larger than a second threshold value smaller than the first threshold value in a periphery of the peak voxel detected, and determines that the peak voxel and the voxel in the periphery held include the subject.

7. The information processing device according to claim 3 wherein

the weighting unit sets a smaller weight for an unviewable voxel in the three-dimensional region.

8. The information processing device according to claim 7, wherein

the weighting unit sets a smaller weight for the unviewable voxel specified on a basis of a viewable region map indicating a distribution of a viewable region in the three-dimensional region.

9. The information processing device according to claim 3, further comprising

a parameter calculating unit that calculates the parameter for each voxel, wherein

the weighting unit is configured to perform weighting of the three-dimensional region by using the parameter calculated by the parameter calculating unit.

10. The information processing device according to claim 9, wherein

the parameter calculating unit calculates the parameter for each voxel on a basis of a position, a direction, a size, and a shape of a visual field cone indicating the imaging range of each of the imaging units.

11. The information processing device according to claim 10, wherein

the parameter calculating unit calculates the parameter further on a basis of a position in the visual field cone of each voxel.

12. The information processing device according to claim 10, wherein

the parameter calculating unit estimates the visual field cone on a basis of a position and an orientation of each of the imaging units.

13. The information processing device according to claim 12, wherein

the parameter calculating unit estimates the visual field cone further on a basis of an internal parameter of each of the imaging units.

14. The information processing device according to claim 12, wherein

the parameter calculating unit estimates the visual field cone further on a basis of depth information.

15. The information processing device according to claim 2, further comprising

a weight video associating unit that associates a distribution of a weight in the three-dimensional region obtained by the weighting unit with a captured image of the three-dimensional region, wherein

the encoding unit is configured to encode the captured image on a basis of the distribution of the weight in the three-dimensional region associated with the captured image by the weight video associating unit.

16. The information processing device according to claim 15, wherein

the encoding unit controls a bit rate or a coding mode of encoding of the captured image depending on the distribution of the weight in the three-dimensional region.

17. The information processing device according to claim 2, further comprising

a weight audio associating unit that associates a distribution of a weight in the three-dimensional region obtained by the weighting unit with audio in the three-dimensional region, wherein

the encoding unit is configured to encode the audio on a basis of the distribution of the weight in the three-dimensional region associated with the audio by the weight audio associating unit.

18. The information processing device according to claim 17, wherein

the encoding unit controls a bit rate or a coding mode of encoding of the audio depending on the distribution of the weight in the three-dimensional region.

19. The information processing device according to claim 2, further comprising

a weight three-dimensional structure associating unit that associates a distribution of a weight in the three-dimensional region obtained by the weighting unit with three-dimensional structure data indicating a three-dimensional structure of the three-dimensional region, wherein

the encoding unit is configured to encode the three-dimensional structure data on a basis of the distribution of the weight in the three-dimensional region associated with the three-dimensional structure data by the weight three-dimensional structure associating unit.

20. The information processing device according to claim 19, wherein

the encoding unit controls an approximation and a deletion rate of a number of vertices expressing three dimensions, of encoding of the three-dimensional structure data, depending on the distribution of the weight in the three-dimensional region.

21. An information processing method comprising

encoding information regarding a three-dimensional region on a basis of a distribution related to overlapping of visual fields of the three-dimensional region to be imaged by a plurality of imaging units, the distribution being specified by using a parameter relating to overlapping of visual fields that are imaging ranges of the plurality of imaging units.