Rendering device and rendering method

• US 9,426,412 B2
• Filed: 05/21/2012
• Issued: 08/23/2016
• Est. Priority Date: 07/13/2011
• Status: Expired due to Fees

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Nishida, Hideshi
• Primary Examiner(s): Li, Tracy
• Assistant Examiner(s): Rahman, Mohammad J
• Application Number:
  US13/819,770
• Publication Number:
  US 20130155185A1
• Time in Patent Office: 1,555 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: G09G 2340/0442 Handling or displaying different aspect ratios, or changing the aspect ratio
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/373 for modifying the size of the graphic pattern
• CPC Class Codes: G09G 5/391 Resolution modifying circuits, e.g. variable screen formats
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G09G 5/395 Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen G09G5/399 takes precedence
• CPC Class Codes: H04N 21/42653 for processing graphics
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 5/44504 Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04N 7/0117 involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407

Abstract

A rendering device provides improved rendering responsiveness in multi-window display for rendering scenarios in which display sizes of images vary over time, while also reducing required memory bandwidth. The device comprises: a scenario processor 101 for interpreting a rendering scenario and calculating for each frame period a scale-down ratio for each of a plurality of pictures; a plurality of decoders 107 for decoding encoded data of a plurality of videos; a plurality of first scalers for scaling-down the decoded pictures using the scale-down ratios calculated by the scenario processor 101; a memory 106 for storing the scaled-down pictures; a plurality of second scalers 113 for reading the scaled-down pictures from the memory and re-scaling the scaled-down pictures to match the scale-down ratios calculated by the scenario processor for a current frame period; and, a composing unit 115 for composing the re-scaled pictures.

Claims

1. A rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the rendering device comprising:

a scenario processor configured to analyze a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

a plurality of acquiring units each configured to acquire an image object for the frame period;

a plurality of first scalers configured to scale-down the acquired image objects;

a memory configured to store therein the scaled-down image objects;

a plurality of second scalers configured to, when the target frame period is a current frame period, read from the memory the scaled-down image objects and re-scale each of the scaled-down image objects to match a scale-down ratio calculated by the scenario processor for the current frame period; and

a composing unit configured to compose each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object, wherein

the rendering scenario includes a scaling function for each of the image objects, the scaling function designating as an argument the scale-down ratio in any of an X-axis direction, a Y-axis direction, and a Z-axis direction,

after n predetermined frame periods have passed, the scenario processor displays each of the image objects at the scale-down ratio designated by the argument of the scaling function,

from when a 1st frame period passes to when an n−

1st frame period passes, the scenario processer changes the scale-down ratio in stages in accordance with the number of passed frame periods, and

the target frame is one of frame periods to which the scale-down ratio which is changed in stages is allocated.

2. A rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the rendering device comprising:

a scenario processor configured to analyze a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

a plurality of acquiring units each configured to acquire an image object for the frame period;

a plurality of first scalers configured to scale-down the acquired image objects;

a memory configured to store therein the scaled-down image objects;

a plurality of second scalers configured to, when the target frame period is a current frame period, read from the memory the scaled-down image objects and re-scale each of the scaled-down image objects to match a scale-down ratio calculated by the scenario processor for the current frame period;

a composing unit configured to compose each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object; and

a plurality of third scalers that are in one-to-one correspondence with the first scalers and are each configured, when a scale-down ratio used by a corresponding first scaler on the acquired image object is smaller than a predetermined scale-down ratio, to create an additionally scaled-down image object, corresponding to the scaled-down image object, using a smaller scale-down ratio than the scale-down ratio used by the corresponding first scaler, and to store both the additionally scaled-down image object and the scaled-down image object in the memory, wherein

each of the additionally scaled-down image objects stored in the memory is read from the memory and re-scaled by the second scalers to create a re-scaled image object, and

each of the scaled-down image objects stored in the memory is only read from the memory and re-scaled by the second scalers to create a re-scaled image object when no additionally scaled-down image object corresponding to the scaled-down image object is stored in the memory.

3. The rendering device in claim 2, wherein

the additionally scaled-down image object is a mipmap of the acquired image object.

4. The rendering device in claim 3, wherein

the predetermined scale-down ratio is 3/4.

5. A rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the rendering device comprising:

a scenario processor configured to analyze a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

a plurality of acquiring units each configured to acquire an image object for the frame period;

a plurality of first scalers configured to scale-down the acquired image objects;

a memory configured to store therein the scaled-down image objects;

a plurality of second scalers configured to, when the target frame period is a current frame period, read from the memory the scaled-down image objects and re-scale each of the scaled-down image objects to match a scale-down ratio calculated by the scenario processor for the current frame period; and

a composing unit configured to compose each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object, wherein

a delay equivalent to a frame period k occurs for combined actions of writing to and reading from the memory each of the scaled-down image objects, and

when one of the acquired image objects is scaled-down by one of the first scalers using a scale-down ratio S1 at a frame period t-k temporally in the past, one of the second scalers reads from the memory the scaled-down image object scaled-down using the ratio S1 and re-scales the scaled-down image object for a current frame period t where the scale-down ratio calculated by the scenario processor is S2, by using a factor of S2/S1.

6. The rendering device in claim 5, wherein

the scale-down ratio used by each of the first scalers is the scale-down ratio calculated by the scenario processor with respect to each of the image objects for the current frame period.

7. The rendering device in claim 5, wherein

the smaller a shape specified by the rendering scenario for each of the image objects, the smaller a scale-down ratio used by each of the first scalers.

8. The rendering device in claim 5, wherein

the larger a distortion in an image object pasting shape specified by the rendering scenario for each of the image objects compared to an original shape of the image object, the smaller a scale-down ratio used by each of the first scalers.

9. The rendering device in claim 5, wherein

the larger a variation over time in an image object pasting position specified by the rendering scenario for each of the image objects, the smaller a scale-down ratio used by each of the first scalers.

10. The rendering device in claim 5, further comprising

a plurality of image feature detection units each configured to detect a human figure image from an image object, wherein

the smaller the human figure image detected in each of the image objects by the feature detection units, the smaller a scale-down ratio used by each of the first scalers.

11. The rendering device in claim 5, wherein

the plurality of image objects include at least one pair of viewpoint images to be used in stereoscopic playback, and

a scale-down ratio used by one of the first scalers on one of the pair of viewpoint images is larger than a scale-down ratio used by another one of the first scalers on the other of the pair of viewpoint images.

12. An integrated circuit that is incorporatable into the rendering device in claim 5, the integrated circuit comprising:

the scenario processor;

the acquiring units;

the first scalers;

the memory;

the second scalers; and

the composing unit.

13. The rendering device in claim 5, being a television device,

wherein

the acquiring units each include:

a tuner configured to acquire a plurality of pieces of encoded data of a video; and

a decoder configured to acquire a plurality of image objects by decoding the acquired plurality of pieces of encoded data of the videos, and

the television device further comprises

a display configured to display the virtual space video composed by the composing unit using the plurality of image objects.

14. A method for controlling a rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the method comprising:

an analyzing step of analyzing a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

an acquiring step of acquiring a plurality of image objects for the frame period;

a first scaling step of scaling-down the acquired image objects;

a storing step of storing the scaled-down image objects in a memory of the rendering device;

a second scaling step of, when the target frame period is a current frame period, reading the scaled-down image objects stored in the memory and re-scaling each of the scaled-down image objects to match a scale-down ratio calculated for the current frame period; and

a composing step of composing each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object, wherein

after n predetermined frame periods have passed, the analyzing step displays each of the image objects at the scale-down ratio designated by an argument of a scaling function,

from when a 1st frame period passes to when an n−

1st frame period passes, the analyzing step changes the scale-down ratio in stages in accordance with the number of passed frame periods, and

the target frame is one of frame periods to which the scale-down ratio which is changed in stages is allocated.