VIDEO PROCESSING APPARATUS CAPABLE OF REPRODUCING VIDEO CONTENT INCLUDING A PLURALITY OF VIDEOS AND CONTROL METHOD THEREFOR

• US 20110149038A1
• Filed: 12/15/2010
• Published: 06/23/2011
• Est. Priority Date: 12/21/2009
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Umeyama, Manabu
• Granted Patent: US 9,338,429 B2
• Time in Patent Office: Days
• US Class Current: 348/44
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking

Abstract

In a video processing apparatus, image information relating to a graphic image corresponding to a first video is converted using difference information calculated from first view-point information relating to a first video and second view-point information relating to a second video. A graphic image corresponding to the second video is generated based on the converted image information, and the generated graphic image and an image of the second video are synthesized.

Claims

1. An apparatus capable of reproducing a video content including a plurality of videos, the apparatus comprising:

an acquisition unit configured to acquire first view-point information relating to a first video, second view-point information relating to a second video, and image information to be used to generate a graphic image corresponding to the first video;

a calculation unit configured to calculate difference information between the first and the second view-point information;

a generation unit configured to convert the image information using the difference information, and generate a graphic image corresponding to the second video based on the converted image information; and

a synthesis unit configured to synthesize the generated graphic image and an image of the second video.

2. The apparatus according to claim 1,

wherein the first and the second view-point information include at least information relating to line-of-sight directions or to view-point positions of the first video and the second video.

3. The apparatus according to claim 2,

wherein the calculation unit calculates an angle formed between the line-of-sight direction of the first video and the line-of-sight direction of the second video as difference information, and

wherein the generation unit rotates the graphic image corresponding to the first video by an angle represented by the calculated difference information to convert the image information, and generates the graphic image corresponding to the second video based on the converted image information.

4. The apparatus according to claim 2,

wherein the calculation unit calculates a difference in distance between the view-point position of the first video and the view-point position of the second video as difference information, and

wherein the generation unit enlarges or reduces the graphic image corresponding to the first video depending on the difference in distance represented by the calculated difference information, to convert the image information, and generates the graphic image corresponding to the second video based on the converted image information.

5. The apparatus according to claim 1, wherein the second video is a free view-point video content at free view-point, which is generated based on video contents captured at a plurality of view-point positions.

6. The apparatus according to claim 1, wherein the second video is one video content constituting a multi view-point video content captured at a plurality of view-point positions.

7. An apparatus capable of reproducing a video content including a plurality of videos that differ in view-points, comprising:

an acquisition unit configured to acquire the plurality of videos and a plurality of graphic images respectively corresponding to the plurality of videos;

a first generation unit configured to generate a free view-point video at free view-point based on the plurality of videos;

a second generation unit configured to generate a graphic image corresponding to the free view-point video based on the plurality of graphic images; and

a synthesis unit configured to synthesize the generated free view-point video and the generated graphic image.

8. A method for controlling an apparatus capable of reproducing a video content including a plurality of videos, the method comprising:

acquiring first view-point information relating to a first video, second view-point information relating to a second video, and image information relating to a graphic image corresponding to the first video;

calculating difference information between the acquired first view-point information and the second view-point information;

converting the image information using the difference information, and generating a graphic image corresponding to the second video based on the converted image information; and

synthesizing the generated graphic image and an image of the second video.

9. The method according to claim 8,

wherein the first and the second view-point information include at least information relating to line-of-sight directions or to view-point positions of the first video and the second video.

10. The method according to claim 9, further comprising:

calculating an angle formed between the line-of-sight direction of the first video and the line-of-sight direction of the second video as difference information; and

rotating the graphic image corresponding to the first video by an angle represented by the calculated difference information to convert the image information, and generating the graphic image corresponding to the second video based on the converted image information.

11. The apparatus according to claim 9, further comprising:

calculating a difference in distance between the view-point position of the first video and the view-point position of the second video as difference information; and

enlarging or reducing the graphic image corresponding to the first video depending on the difference in distance represented by the calculated difference information to convert the image information, and generating the graphic image corresponding to the second video based on the converted image information.

12. The method according to claim 8, wherein the second video is a free view-point video content at free view-point, which is generated based on video contents captured at a plurality of view-point positions.

13. The method according to claim 8, wherein the second video is one video content constituting a multi view-point video content captured at a plurality of view-point positions.

14. A method for controlling an apparatus capable of reproducing a video content including a plurality of videos that differ in view-points, the method comprising:

acquiring the plurality of videos and a plurality of graphic images respectively corresponding to the plurality of videos;

generating a free view-point video at free view-point based on the plurality of videos;

generating a graphic image corresponding to the free view-point video based on the plurality of graphic images; and

synthesizing the generated free view-point video and the generated graphic image corresponding to the free view-point video.