Picture processing apparatus, processing method for use therewith, and program

• US 8,717,504 B2
• Filed: 09/24/2013
• Issued: 05/06/2014
• Est. Priority Date: 01/21/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tsurumi, Shingo
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): HAIEM, SEAN N
• Application Number:
  US14/035,514
• Publication Number:
  US 20140022455A1
• Time in Patent Office: 224 Days
• US Class Current: 348/738
• CPC Class Codes: G06T 13/00 Animation
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/44 Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/607 for more than one sound signal, e.g. stereo, multilanguages H04N5/602 takes precedence
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

A picture conversion information supply section 130 calculates per frame an affine transformation parameter for picture conversion based on motion information about a moving picture. With reference to a reference picture, a picture conversion section 140 affine-transforms pictures making up the moving picture per frame using the calculated affine transformation parameters. Based on information indicating the center position, angle or scaling factor about the transformed pictures coming from the picture conversion section 140, a sound conversion information calculation section 190 calculates sound conversion information for converting the sound corresponding to the pictures. Based on the sound conversion information, a sound conversion processing section 200 controls the volume of each of the channels making up the sound, adds up the controlled sound of each channel, and outputs the result as output sound to speakers 220.

Claims

1. A method comprising:

receiving display area extraction information regarding a display area extracted from a whole area of a synthesized picture and picture conversion information, the synthesized picture being generated by converting a second picture which is positioned chronologically after a first picture in a time line of moving pictures based on the picture conversion information including at least one element regarding a movement of the second picture relative to the first picture, and by synthesizing the converted second picture to the synthesized picture which has been output previously based on the picture conversion information; and

controlling a right channel sound and a left channel sound based on the display area extraction information and the picture conversion information, the control of the right channel sound differing from the control of the left channel sound according to the movement of the second picture relative to the first picture.

2. The method according to claim 1, wherein the display area extraction information includes rotation of the second picture relative to the synthesized picture.

3. The method according to claim 1, wherein the display area extraction information includes scaling of the second picture relative to the synthesized picture.

4. The method according to claim 1, wherein the display area extraction information includes the position, angle or size of the second picture relative to the synthesized picture.

5. A method comprising:

converting a right channel sound and a left channel sound based on a display area extraction information and picture conversion information including at least one element regarding a movement of a second picture relative to a first picture positioned chronologically before the second picture in a time line of moving pictures, the first picture and the second picture being used to generate a synthesized picture, the synthesized picture being generated by converting the second picture based on the picture conversion information, and by synthesizing the converted second picture to the synthesized picture output previously based on the picture conversion information, the conversion of the right channel sound differing from the conversion of the left channel sound according to the movement of the second picture relative to the first picture; and

outputting the converted right channel sound and the converted left channel sound in synchrony with the moving picture.

6. The method according to claim 5, wherein the display area extraction information includes rotation of the second picture relative to the synthesized picture.

7. The method according to claim 5, wherein the display area extraction information includes scaling of the second picture relative to the synthesized picture.

8. The method according to claim 5, wherein the display area extraction information includes the position, angle or size of the second picture relative to the synthesized picture.

9. An information processing apparatus, comprising:

circuitry configured to

receive display area extraction information regarding a display area extracted from a whole area of a synthesized picture and picture conversion information, the synthesized picture being generated by converting a second picture which is positioned chronologically after a first picture in a time line of moving pictures based on the picture conversion information including at least one element regarding a movement of the second picture relative to the first picture, and by synthesizing the converted second picture to the synthesized picture which has been output previously based on the picture conversion information; and

control a right channel sound and a left channel sound based on the display area extraction information and the picture conversion information, the control of the right channel sound differing from the control of the left channel sound according to the movement of the second picture relative to the first picture.

10. The information processing apparatus according to claim 9, wherein the display area extraction information includes rotation of the second picture relative to the synthesized picture.

11. The information processing apparatus according to claim 9, wherein the display area extraction information includes scaling of the second picture relative to the synthesized picture.

12. The information processing apparatus according to claim 9, wherein the display area extraction information includes the position, angle or size of the second picture relative to the synthesized picture.

13. An information processing apparatus, comprising:

circuitry configured to

convert a right channel sound and a left channel sound based on a display area extraction information and picture conversion information including at least one element regarding a movement of a second picture relative to a first picture positioned chronologically before the second picture in a time line of moving pictures, the first picture and the second picture being used to generate a synthesized picture, the synthesized picture being generated by converting the second picture based on the picture conversion information, and by synthesizing the converted second picture to the synthesized picture output previously based on the picture conversion information, the conversion of the right channel sound differing from the conversion of the left channel sound according to the movement of the second picture relative to the first picture; and

output the converted right channel sound and the converted left channel sound in synchrony with the moving picture.

14. The information processing apparatus according to claim 13, wherein the display area extraction information includes rotation of the second picture relative to the synthesized picture.

15. The information processing apparatus according to claim 13, wherein the display area extraction information includes scaling of the second picture relative to the synthesized picture.

16. The information processing apparatus according to claim 13, wherein the display area extraction information includes the position, angle or size of the second picture relative to the synthesized picture.

17. A non-transitory computer-readable medium encoded with computer-readable instructions thereon that, when executed by a computer, cause the computer to perform a method comprising:

receiving display area extraction information regarding a display area extracted from a whole area of a synthesized picture and picture conversion information, the synthesized picture being generated by converting a second picture which is positioned chronologically after a first picture in a time line of moving pictures based on the picture conversion information including at least one element regarding a movement of the second picture relative to the first picture, and by synthesizing the converted second picture to the synthesized picture which has been output previously based on the picture conversion information; and

controlling a right channel sound and a left channel sound based on the display area extraction information and the picture conversion information, the control of the right channel sound differing from the control of the left channel sound according to the movement of the second picture relative to the first picture.

18. The non-transitory computer-readable medium according to claim 17, wherein the display area extraction information includes rotation of the second picture relative to the synthesized picture.

19. The non-transitory computer-readable medium according to claim 17, wherein the display area extraction information includes scaling of the second picture relative to the synthesized picture.

20. The non-transitory computer-readable medium according to claim 17, wherein the display area extraction information includes the position, angle or size of the second picture relative to the synthesized picture.

21. A non-transitory computer-readable medium encoded with computer-readable instructions thereon that, when executed by a computer, cause the computer to perform a method comprising:

converting a right channel sound and a left channel sound based on a display area extraction information and picture conversion information including at least one element regarding a movement of a second picture relative to a first picture positioned chronologically before the second picture in a time line of moving pictures, the first picture and the second picture being used to generate a synthesized picture, the synthesized picture being generated by converting the second picture based on the picture conversion information, and by synthesizing the converted second picture to the synthesized picture output previously based on the picture conversion information, the conversion of the right channel sound differing from the conversion of the left channel sound according to the movement of the second picture relative to the first picture; and

outputting the converted right channel sound and the converted left channel sound in synchrony with the moving picture.

22. The non-transitory computer-readable medium according to claim 21, wherein the display area extraction information includes rotation of the second picture relative to the synthesized picture.

23. The non-transitory computer-readable medium according to claim 22, wherein the display area extraction information includes scaling of the second picture relative to the synthesized picture.

24. The non-transitory computer-readable medium according to claim 21, wherein the display area extraction information includes the position, angle or size of the second picture relative to the synthesized picture.