Image encoding and display

• US 10,257,492 B2
• Filed: 02/25/2015
• Issued: 04/09/2019
• Est. Priority Date: 02/26/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Raghoebardajal, Sharwin Winesh, Bickerstaff, Ian Henry
• Primary Examiner(s): Jiang, Zaihan
• Application Number:
  US15/121,518
• Publication Number:
  US 20160366392A1
• Time in Patent Office: 1,504 Days
• Field of Search: 348 53
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0074 Stereoscopic image analysis
• CPC Class Codes: H04N 21/440245 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 2213/008 Aspects relating to glasses for viewing stereoscopic images

Abstract

An image encoding method includes, for each of left and right images of an image stereo pair, identifying a region of interest (ROI) in an input image, generating multiple representations of that input image such that one representation is a high field of view (FOV) representation including substantially all of the input image and another representation is a low FOV representation having a lower FOV than the high FOV representation and including at least the ROI. The relationship between pixel resolution and image size for the high and low FOV representations is such that, in relation to the ROI, the low FOV representation provides more image detail than the high FOV representation. The method also includes generating one or more multiplexed images by juxtaposing portions of the left and right, high and low FOV representations so as to include all image content thereof in fewer than four multiplexed images.

Claims

1. An image encoding method comprising:

for each of a left and a right image of a stereo pair of images, identifying, in an input image, a region of interest;

generating, by one or more processors, at least two image representations of that input image, such that one of the image representations is a high field of view (FOV) image representation including substantially all of the input image, and another of the image representations is a low FOV image representation having a lower FOV than the high FOV image representation and including at least the region of interest;

in which a relationship between pixel resolution and image size for the high FOV image representation and the low FOV image representation is such that, in relation to the region of interest, the low FOV image representation provides a higher amount of image detail than the high FOV image representation;

generating, by the one or more processors, one or more multiplexed images by juxtaposing at least portions of the left and right, high FOV and low FOV image representations, so as to include all image content of the left and right, high FOV and low FOV image representations in fewer than four multiplexed images; and

performing one or both of transmitting or storing the multiplexed images;

in which generating the one or more multiplexed images comprises generating a first multiplexed image by juxtaposing a whole of a first one of the left and right, high FOV and low FOV image representations next to a whole of a second one of the left and right, high FOV and low FOV image representations, and generating a second multiplexed image by juxtaposing a whole of a third one of the left and right, high FOV and low FOV image representations next to a whole of a fourth one of the left and right, high FOV and low FOV image representations.

2. A method according to claim 1, in which the performing step comprises one or both of transmitting or storing the multiplexed images at a higher image repetition rate than an image repetition rate of the input images.

3. A method according to claim 1, in which performing step comprises one or both of transmitting or storing the multiplexed images as respective left and right dummy 3D images.

4. A non-transitory computer readable storage medium storing computer readable instructions thereon, the instructions, when executed by one or more processors, casing the one or more processors to perform an image encoding method, the method comprising:

for each of a left and a right image of a stereo pair of images, identifying, in an input image, a region of interest;

generating at least two image representations of that input image, such that one of the image representations is a high field of view (FOV) image representation including substantially all of the input image, and another of the image representations is a low FOV image representation having a lower FOV than the high FOV image representation and including at least the region of interest;

in which a relationship between pixel resolution and image size for the high FOV image representation and the low FOV image representation is such that, in relation to the region of interest, the low FOV image representation provides a higher amount of image detail than the high FOV image representation;

generating one or more multiplexed images by juxtaposing at least portions of the left and right, high FOV and low FOV image representations, so as to include all image content of the left and right, high FOV and low FOV image representations in fewer than four multiplexed images; and

performing one or both of transmitting or storing the multiplexed images;

in which generating the one or more multiplexed images comprises generating a first multiplexed image by juxtaposing a whole of a first one of the left and right, high FOV and low FOV image representations next to a whole of a second one of the left and right, high FOV and low FOV image representations, and generating a second multiplexed image by juxtaposing a whole of a third one of the left and right, high FOV and low FOV image representations next to a whole of a fourth one of the left and right, high FOV and low FOV image representations.

5. An image encoding apparatus comprising:

a representation generator configured, for each of a left and a right image of a stereo pair of images, to;

identify, in an input image, a region of interest; and

generate at least two image representations of the input image, such that one of the representations is a high field of view (FOV) image representation including substantially all of the input image, and another of the image representations is a low FOV image representation having a lower FOV than the high FOV image representation and including at least the region of interest;

in which a relationship between pixel resolution and image size for the high FOV image representation and the low FOV image representation is such that, in relation to the region of interest, the low FOV image representation provides a higher amount of image detail than the high FOV image representation;

a multiplexer operatively coupled to the representation generator and configured to generate one or more multiplexed images by juxtaposing at least portions of the left and right, high FOV and low FOV image representations, so as to include all image content of the left and right, high FOV and low FOV image representations in fewer than four multiplexed images; and

an image output operatively coupled to the multiplexer and configured to output the multiplexed images;

in which the representation generator is configured to generate the one or more multiplexed images by generating a first multiplexed image by juxtaposing a whole of a first one of the left and right, high FOV and low FOV image representations next to a whole of a second one of the left and right, high FOV and low FOV image representations, and by generating a second multiplexed image by juxtaposing a whole of a third one of the left and right, high FOV and low FOV image representations next to a whole of a fourth one of the left and right, high FOV and low FOV image representations.

6. The image encoding apparatus of claim 5, wherein the image encoding apparatus is a videogame machine.