Techniques for Correction of Visual Artifacts in Multi-View Images

• US 20190005709A1
• Filed: 06/30/2017
• Published: 01/03/2019
• Est. Priority Date: 06/30/2017
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Kim, Jae Hoon, Zhang, Dazhong, Yuan, Hang, Wu, Hsi-Jung, Zhai, Jiefu, Chen, Ming, Zhou, Xiaosong
• Application Number:
  US15/638,587
• Publication Number:
  US 20190005709A1
• Time in Patent Office: Days
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Techniques are disclosed for correcting artifacts in multi-view images that include a plurality of planar views. Image content the planar views may be projected from the planar representation to a spherical projection. Thereafter, a portion of the image content may be projected from the spherical projection to a planar representation. The image content of the planar representation may be used for display. Extensions are disclosed that correct artifacts that may arise during deblocking filtering of the multi-view images.

Claims

1. An image correction method, comprising:

projecting image content of planar views from a multi-view image to a spherical projection,

projecting at least a portion of the image content from the spherical projection to a planar projection, and

displaying the image content projected on the planar projection.

2. The method of claim 1, wherein the portion of the image content is a view window selected from the spherical projection.

8. The method of claim 1, wherein the multi-view image contains a plurality of planar views of image data.

9. The method of claim 1, wherein the multi-view image contains a plurality of planar views of image data and a panoramic view of image data.

3. An image coding method, comprising:

for a multi-view image, projecting image content of a plurality of views of the image to a common spherical projection,

deriving a two dimensional image from the content of the spherical projection, wherein the two dimensional image has regions for each view of the multi-view image surrounded by padding content;

coding the two dimensional image by motion-compensation prediction.

4. The method of claim 3, wherein the coding comprises, for intra-coding, coding a pixel block that contains padded image content using decoded data of another pixel block from the same image as a basis of prediction.

5. The method of claim 3, wherein the coding comprises, for inter-coding, coding a pixel block that contains padded image content using decoded data of a pixel block from a previously-coded image as a basis of prediction.

6. The method of claim 3, further comprising decoding the coded two dimensional image, the decoding including deblocking filtering of recovered padded image content.

7. The method of claim 3, wherein, for one view of the multi-view image, padding content is derived from image data of another view of the multi-view image.

10. A system, comprising:

a pre-processor having an input for a multi-view image to;

project image content of a plurality of views of the multi-view image to a common spherical projection, and

derive a two dimensional image from the content of the spherical projection, wherein the two dimensional image has regions for each view of the multi-view image surrounded by padding content;

a motion compensation prediction video coder having an input for image data output by the pre-processor.

11. The system of claim 10, wherein the video coder comprises:

a pixel block coder having an input for a pixel block from the pre-processor image data and an input for prediction data and an output for coded pixel block data,

a pixel block decoder having an input for the coded pixel block data,

a reference picture store for storing a decoded images obtained from the pixel block decoder, the decoded images containing padding content, and

a predictor, having an input for reference picture data from the reference picture store and an output for the prediction data.

12. The system of claim 11, wherein, for intra-coding of a pixel block, the predictor outputs decoded data of another pixel block from the same image as the pixel block as the prediction data.

13. The system of claim 11, wherein, for intra-coding of a pixel block, the predictor outputs decoded data of a pixel block from a different image as the pixel block as the prediction data.

14. The system of claim 11, further comprising a deblocking filter provided in a communication path between the pixel block decoder and the reference picture store to deblock filter decoded images including the padded image content.

15. A computer readable medium storing program instructions that, when executed by a processing device, cause the device to perform a method, comprising:

for a multi-view image, projecting image content of a plurality of views of the image to a common spherical projection,

deriving a two dimensional image from the content of the spherical projection, wherein the two dimensional image has regions for each view of the multi-view image surrounded by padding content;

coding the two dimensional image by motion-compensation prediction.

16. An image decoding method, comprising:

decoding a coded two dimensional image by motion-compensation, the two dimensional image representing content of multiple views about a common reference point, the two dimensional image having regions for each view of the multi-view image surrounded by padding content;

storing a decoded two dimensional image in a reference picture store for use in decoding of a later-received coded two dimensional image having padding content.

17. The method of claim 16, wherein the decoding comprises, for an intra-coded pixel block of the coded two dimensional image that contains padded image content, decoding the pixel block using decoded data of another pixel block from the same image as a basis of prediction.

18. The method of claim 16, wherein the decoding comprises, for an intra-coded pixel block of the coded two dimensional image that contains padded image content, decoding the pixel block using decoded data of a pixel block from a previously-coded image as a basis of prediction.

19. The method of claim 16, wherein the decoding comprises deblocking filtering the decoded two dimensional image including the padded image content.

20. The method of claim 16, wherein, for one view of the multi-view image, the padding content is derived from image data of another view of the retrieved multi-view image.

21. The method of claim 16, wherein the coded multi-view image and the retrieved multi-view image each contains a plurality of planar views of image data.

22. The method of claim 16, wherein the coded multi-view image and the retrieved multi-view image each contains a plurality of planar views of image data and a panoramic view of image data.

23. A system, comprising:

a motion compensation prediction video decoder having an input for coded image data representing content of multiple views about a common reference point, the two dimensional image having regions for each view of the multi-view image surrounded by padding content;

a post-processor having an input for a decoded multi-view image to derive an image from the decoded multi-view image content without padding content.

24. The system of claim 23, wherein the decoder comprises:

a pixel block decoder having an input for a coded pixel block data from the coded image data and an input for prediction data,

a reference picture store for storing a decoded images obtained from the pixel block decoder, the decoded images containing padding content, and

a predictor, having an input for reference picture data from the reference picture store and an output for the prediction data.

25. The system of claim 24, wherein, for an intra-coded pixel block, the predictor outputs decoded data of another pixel block from the same image as the coded pixel block as the prediction data.

26. The system of claim 24, wherein, for an intra-coded pixel block, the predictor outputs decoded data of a pixel block from the a different image as the coded pixel block as the prediction data.

27. The system of claim 24, further comprising a deblocking filter provided in a communication path between the pixel block decoder and the reference picture store to deblock filter decoded images including the padded image content.

28. A computer readable medium storing program instructions that, when executed by a processing device, cause the device to perform a method, comprising:

decoding a coded two dimensional image by motion-compensation, the two dimensional image representing content of multiple views about a common reference point, the two dimensional image having regions for each view of the multi-view image surrounded by padding content;

storing a decoded two dimensional image in a reference picture store for use in decoding of a later-received coded two dimensional image having padding content.