METHOD AND APPARATUS AND COMPUTER PROGRAM FOR GENERATING A 3 DIMENSIONAL IMAGE FROM A 2 DIMENSIONAL IMAGE

• US 20110129143A1
• Filed: 08/20/2010
• Published: 06/02/2011
• Est. Priority Date: 11/27/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Gillard, Clive Henry, Keating, Stephen Mark
• Granted Patent: US 8,509,521 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 2213/002 Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices

Abstract

A method of generating a three dimensional image from a two dimensional image is described. In the method, the two dimensional image has a background and a first foreground object and a second foreground object located thereon, the method comprising the steps of: applying a transformation to a copy of the background, generating stereoscopically for display the background and the transformed background, generating stereoscopically for display the first and second foreground object located on the stereoscopically displayable background and the transformed background and determining whether the first and second foreground objects occlude with one another, wherein in the event of occlusion, the occluded combination of the first and second object forms a third foreground object and, the method further comprises the step of: applying a transformation to the third foreground object, wherein the transformation applied to the third foreground object is less than or equal to the transformation applied to the background; generating a copy of the third foreground object with the transformation applied thereto and generating stereoscopically for display the third foreground object with the transform applied thereto and the copy of the third foreground object displaced relative to one another by an amount determined in accordance with the position of one of the first or second foreground objects in the image.

Claims

1. A method of generating a three dimensional image from a two dimensional image, the two dimensional image having a background and a first foreground object and a second foreground object located thereon, the method comprising the steps of:

applying a transformation to a copy of the background, generating stereoscopically for display the background and the transformed background, generating stereoscopically for display the first and second foreground object located on the stereoscopically displayable background and the transformed background and determining whether the first and second foreground objects occlude with one another, wherein in the event of occlusion, the occluded combination of the first and second object forms a third foreground object and, the method further comprises the step of;

applying a transformation to the third foreground object, wherein the transformation applied to the third foreground object is less than or equal to the transformation applied to the background;

generating a copy of the third foreground object with the transformation applied thereto and generating stereoscopically for display the third foreground object with the transform applied thereto and the copy of the third foreground object displaced relative to one another by an amount determined in accordance with the position of one of the first or second foreground objects in the image.

5. A method according to claim 1 or 4, wherein the step of determining whether occlusion has occurred comprises the step of comparing the length of the detected foreground object in one direction of the image with a threshold length, and determining that occlusion has occurred when the length of the detected foreground object is longer than the threshold length.

6. A method according to claim 1 or 4, wherein the step of determining whether occlusion has occurred comprises the step of comparing the height of the detected object in the vertical direction of the image with a threshold height, and determining that occlusion has occurred when the height of the detected foreground object is taller than the threshold height.

7. A method according to claim 5, wherein the degree of transformation applied to the third foreground object is determined in accordance with difference between the length of the detected foreground object and the threshold length.

8. A method according to any one of claim 1 or 4, comprising storing a copy of the background of the image, and in the event of applying a transformation to the third foreground object, the method further comprises the step of mapping pixels from the background version of the image to corresponding pixel positions vacated by the transformed third foreground object.

17. A computer program containing computer readable instructions which, when loaded onto a computer, configure the computer to perform a method according to any one of claims 1 to 8.

18. A computer readable storage medium configured to store a computer program according to claim 17 therein or thereon.

2. A method of generating a three dimensional image from a two dimensional image, the two dimensional image having a background and a first foreground object and a second foreground object located thereon, the method comprising the steps of:

applying a transformation to a copy of the background, generating stereoscopically for display the background and the transformed background, generating stereoscopically for display the first and second foreground object located on the stereoscopically displayable background and the transformed background and determining whether the first and second foreground objects are within a predetermined distance of one another, wherein in the event of the first and second foreground object being within a predetermined distance of one another, the method further comprises the step of;

applying a transformation to the first foreground object, wherein the transformation applied to the first foreground object is less than or equal to the transformation applied to the background;

generating a copy of the first foreground object with the transformation applied thereto and generating stereoscopically for display the first foreground object with the transform applied thereto and the copy of the first foreground object displaced relative to one another by an amount determined in accordance with the position of the first foreground object in the image.

3. A method according to claim 2 wherein the amount of transformation applied to the first and/or second foreground object increases inversely proportionally to the distance between the first and second foreground object.

4. A method according to claim 3, further comprising determining whether the first and second foreground objects occlude with one another, wherein in the event of occlusion, the occluded combination of the first and second foreground object forms a third foreground object, and the method further comprises the step of applying a transform to the third foreground object wherein the transformation applied to the third foreground object is less than or equal to the transformation applied to the background;

generating a copy of the third foreground object with the transformation applied thereto and generating stereoscopically for display the third foreground object with the transform applied thereto and the copy of the third foreground object displaced relative to one another by an amount determined in accordance with the position of the first or second foreground object in the image.

9. An apparatus for generating a three dimensional image from a two dimensional image, the two dimensional image having a background and a first foreground object and a second foreground object located thereon, the apparatus comprising:

a generator for applying a transformation to a copy of the background, a device for generating stereoscopically for display the background and the transformed background, and generating stereoscopically for display the first and second foreground object located on the stereoscopically generated background and the transformed background and, a determiner for determining whether the first foreground object and the second foreground object occlude with one another, wherein in the event of occlusion, the occluded combination of the first and second foreground object form a third foreground object and, the apparatus is operable to apply a transformation to the third foreground object wherein the transform applied to the third foreground object is less than or equal to the transform applied to the background;

the generator being further operable to generate a copy of the third foreground object with the transform applied thereto and a device being operable to generate stereoscopically for display the third foreground object with the transform applied thereto and the copy of the third foreground object displaced relative to one another by an amount determined in accordance with the position of one of the first or second foreground object in the image.

10. An apparatus for generating a three dimensional image from a two dimensional image, the two dimensional image having a background and a first foreground object and a second foreground object located thereon, the apparatus comprising a transformer operable to apply a transformation to a copy of the background, a generator operable to generate stereoscopically for display the background and the transformed background, the generator being further operable to generate stereoscopically for display the first and second foreground object located on the stereoscopically displayable background and the transformed background and a determiner operable to determine whether the first and second foreground objects are within a predetermined distance of one another, wherein in the event of the first and second foreground object being within a predetermined distance of one another, an applier is operable to apply a transformation to the first foreground object, wherein the transformation applied to the first foreground object is less than or equal to the transformation applied to the background;

a copy generator operable to generate a copy of the first foreground object with the transformation applied thereto and to generate stereoscopically for display the first foreground object with the transform applied thereto and the copy of the first foreground object being displaced relative to one another by an amount determined in accordance with the position of the first foreground object in the image.

11. An apparatus according to claim 10 wherein the amount of transformation applied to the first and/or second foreground object increases inversely proportionally to the distance between the first and second foreground object.

12. An apparatus according to claim 11, wherein the determiner is further operable to determine whether the first and second foreground objects occlude with one another, wherein in the event of occlusion, the occluded combination of the first and second foreground object forms a third foreground object, and the applier is further operable to apply a transform to the third foreground object wherein the transformation applied to the third foreground object is less than or equal to the transformation applied to the background;

the generator is further operable to generate a copy of the third foreground object with the transformation applied thereto and to generate stereoscopically for display the third foreground object with the transform applied thereto and the copy of the third foreground object displaced relative to one another by an amount determined in accordance with the position of the first or second foreground object in the image.

13. An apparatus according to claim 9 or 12, wherein the determiner is further operable to compare the length of the detected foreground object in one direction of the image with a threshold length, and to determine that occlusion has occurred when the length of the detected foreground object is longer than the threshold length.

14. An apparatus according to claim 9 or 12, wherein the determiner is operable to compare the height of the detected foreground object in the vertical direction of the image with a threshold height, and to determine that occlusion has occurred when the height of the detected foreground object is taller than the threshold height.

15. An apparatus according to claim 13, wherein the degree of transformation applied is determined in accordance with difference between the length of the detected object and the threshold length.

16. An apparatus according to any one of claim 9 or 12, comprising a store operable to store a copy of the background image, and in the event of applying a transform to the third foreground object, a mapping device is further operable to map pixels from the background version of the image to corresponding pixel positions vacated by the transformed third foreground object.