Image processing system, image provider server, information processing device, and image processing method, adapted to change in resolution

• US 8,933,936 B2
• Filed: 09/10/2012
• Issued: 01/13/2015
• Est. Priority Date: 12/27/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.), Sony Corporation (Sony Group Corp.)
• Inventors: Inada, Tetsugo, Ohba, Akio, Segawa, Hiroyuki
• Primary Examiner(s): TITCOMB, WILLIAM D
• Application Number:
  US13/608,101
• Publication Number:
  US 20130167031A1
• Time in Patent Office: 855 Days
• Field of Search: 715/716, 715/700, 715/815, 715/848, 345/619, 345/428, 345/698
• US Class Current: 345/428
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612
• CPC Class Codes: H04N 21/4821 using a grid, e.g. sorted out by channel and broadcast time
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection

Abstract

Moving image data is delivered from an image provider server. A hierarchical data generation device decodes the moving image data and generates data for the moving image representing each frame in a plurality of resolutions, by reducing frames included in the moving image in a single or multiple stages. A decoder reads only data for a layer in the hierarchical data for each frame, the layer being determined by a resolution requested for display, and decodes the read data. This produces a series of frames representing frames in a requested resolution. A display device displays the frames so that the moving image is displayed in the requested resolution.

Claims

1. An image processing system comprising an information terminal configured to display an image array in response to a user operation, and a server connected to the information terminal via a network and configured to transmit image array data to the information terminal,

the information terminal comprising:

a user operation information transmission unit configured to receive user operation input for requesting a change in a size of one or more images in the image array subject to display and to transmit information related to the user operation input to the server; and

a displayed image processing unit configured to display the image array by using data for a display screen that is transmitted from the server in response to the user operation input and in which a size of the one or more images in the image array subject to display is changed,

the server comprising;

a hierarchical data generation device configured to generate hierarchical data representing each of the images in the image array subject to display in a plurality of resolutions and hierarchized in the order of resolution; and

a displayed image processing device which comprises;

a decoder unit configured to switch and decode one or more layers in the hierarchical data subject to decoding process in accordance with a change in a display resolution of the one or more images in the image array subject to display in the information terminal, the change being indicated by the user operation input transmitted from the information terminal;

a display screen generation unit configured to generate the data for the display screen including the decoded one or more images in the image array subject to display; and

a data transmission unit configured to transmit the data for the display screen to the information terminal.

2. The image processing system according to claim 1,

wherein the information terminal further comprises a data reception unit configured to receive image data from an image provider server via a network, and

wherein the displayed image processing unit of the information terminal includes, in a display screen the data for which is transmitted from the server, an image produced by decoding said image data, and presents a resultant display screen as an ultimate display screen.

3. The image processing system according to claim 2, wherein the displayed image processing unit further includes, in the display screen, an image produced by decoding image data stored in a storage in the information terminal.

4. The image processing system according to claim 2, wherein the information terminal changes a source of supply of data for an image included in a display screen, depending on information on availability of bandwidth for transfer of image data.

5. The image processing system according to claim 1, wherein the hierarchical data generation device generates hierarchical data for images subject to display in a plurality of information terminals in a centralized manner, and transmits, to each of a plurality of displayed image processing devices comprised in the server, data for a layer subject to decoding by the decoder unit in each of the plurality of displayed image processing devices.

6. The image processing system according to claim 1, wherein

the user operation information transmission unit of the information terminal acknowledges the user operation input requesting a change in the number of concurrently displayed images, and

the display screen generation unit of the server generates the data for a display screen including a plurality of images subject to display decoded by the decoder unit.

7. The image processing system according to claim 1,

wherein the hierarchical data generation device of the server generates items of the hierarchical data, each of which represents each frame in a moving image subject to display, and

wherein the data transmission unit successively transmits the data for a display screen including the each frame generated by the display screen generation unit to the information terminal.

8. The image processing system according to claim 7, wherein the display screen generation unit increases, if a display resolution of a moving image subject to display is less than or equal to a threshold value, a speed of playing back the moving image without changing a rate of generating the data for a display screen, by discarding frames subject to generation at predetermined intervals.

9. The image processing system according to claim 7, wherein if the display resolution is less than or equal to a threshold value as a result of an increase in the number of concurrently displayed moving images, the display screen generation unit discards frames of a moving image outside a predetermined range from the screen'"'"'s center or from a moving image selected by a user at predetermined intervals, and generates the data for a display screen accordingly.

10. The image processing system according to claim 7, wherein the hierarchical data generation device generates hierarchical data including a layer of an image representing frames of a moving image in the original resolution and a layer of an image representing another moving image produced in association with said moving image in a resolution smaller than the original resolution.

11. An image provider server comprising:

a user operation information reception unit configured to receive information related to user operation input requesting a change in a size of one or more images in an image array subject to display from an information terminal connected via a network;

a hierarchical data generation unit configured to generate hierarchical data representing each of the images in the image array subject to display in a plurality of resolutions and hierarchized in the order of resolution;

a decoder unit configured to switch and decode one or more layers in the hierarchical data subject to decoding process in accordance with a change in a display resolution of the one or more images in the image array subject to display in the information terminal, the change being indicated by the information related to user operation input;

a display screen generation unit configured to generate data for a display screen including the decoded one or more images in the image array subject to display; and

a data transmission unit configured to transmit the data for the display screen to the information terminal so as to display the display screen in which a size of the one or more images in the image array subject to display is changed.

12. An information processing device comprising:

a hierarchical data generation unit configured to generate hierarchical data representing, for each of one or more moving images in an image array, each frame of the moving image in a plurality of resolutions and hierarchized in the order of resolution;

an input unit configured to acknowledge user operation input requesting a change in the number of concurrently displayed one or more images in the image array while one or more images in the image array are being displayed;

a decoder unit configured to switch and decode one or more layers in the hierarchical data subject to decoding process when a display resolution of one or more moving images in the image array subject to display is changed in response to the user operation input; and

a display unit configured to generate and display data for a display screen including decoded one or more moving images in the image array.

13. An image processing method implemented by an information terminal configured to display an image array in response to a user operation, and a server connected to the information terminal via a network and configured to transmit image data to the information terminal, the method comprising:

receiving, in the information terminal, user operation input requesting a change in a size of one or more images in the image array subject to display and transmitting the user operation input to the server;

generating, in the server, hierarchical data representing the each of the images in the image array subject to display in a plurality of resolutions and hierarchized in the order of resolution;

switching and decoding, in the server, one or more layers in the hierarchical data subject to decoding process in accordance with a change in a display resolution of one or more images in the image array subject to display in the information terminal, the change being indicated by the user operation input transmitted from the information terminal;

generating, in the server, the data for a display screen including the decoded one or more images in the image array subject to display;

transmitting, from the server, the data for the display screen to the information terminal; and

displaying, in the information terminal, the image array by using data for the display screen which is transmitted from the server in response to the user operation input and in which a size of the one or more images in the image array subject to display is changed.

14. A computer program embedded in a non-transitory computer-readable recording medium, comprising:

an acknowledging module configured to receive, from an information terminal connected via a network, user operation input requesting a change in a size of one or more images in an image array subject to display;

a generation module configured to generate hierarchical data representing each of the images in the image array subject to display in a plurality of resolutions and hierarchized in the order of resolution;

a decoder module configured to switch and decode one or more layers in the hierarchical data subject to decoding process in accordance with a change in a display resolution of the one or more images in the image array subject to display in the information terminal, the change being indicated by the user operation input;

a generation module configured to generate the data for a display screen including the decoded one or more images in the image array subject to display; and

a transmission module configured to transmit the data for the display screen to the information terminal so as to display the display screen in which a size of the one or more images in the image array subject to display is changed in the information terminal.

15. A recording medium encoded with a computer program comprising:

an acknowledging module configured to receive, from an information terminal connected via a network, user operation input requesting a change in a size of one or more images in an image array subject to display;

a generation module configured to generate hierarchical data representing the each of the images in the image array subject to display in a plurality of resolutions and hierarchized in the order of resolution;

a decoder module configured to switch and decode one or more layers in the hierarchical data subject to decoding process in accordance with a change in a display resolution of the one or more images in the image array subject to display in the information terminal, the change being indicated by the user operation input;

a generation module configured to generate the data for a display screen including the decoded one or more images in the image array subject to display; and

a transmission module configured to transmit the data for the display screen to the information terminal so as to display the display screen in which a size of the one or more images in the image array subject to display is changed in the information terminal.