Stereoscopic image generation method and apparatus thereof

• US 5,959,663 A
• Filed: 09/23/1997
• Issued: 09/28/1999
• Est. Priority Date: 10/19/1995
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hamahata, Nariyasu, Fukudome, Hitoshi, Minami, Nobuyuki, Kumagai, Takao, Oba, Seisuke
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): VO, TUNG T
• Application Number:
  US08/860,270
• Time in Patent Office: 735 Days
• Field of Search: 348/42, 348/43, 348/44, 348/56, 348/48, 348/49, 348/345, 348/578, 345/7, 345/8, 358/3, 358/87, 358/92, 358/88, 350/334, 395/125
• US Class Current: 348/46
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 5/74 Projection arrangements for image reproduction, e.g. using eidophor

Abstract

The transform arithmetic operations by spatial image transform matrixes and perspective transform matrixes to a screen surface thereon are executed on source video signal representing two-dimensional image so as to generate left-eye and right-eye video signals. It is possible to realize the stereoscopic image generation apparatus capable of setting the visual difference between a left eye and a right eye easily.

Claims

1. A stereoscopic image generation apparatus for generating video signal in order to obtain stereoscopic effects visually, comprising:

a left-eye processor for generating left-eye stereoscopic video signal from source video signal based on a spatial image transform matrix for image transform which converts an image represented by said source video signal to virtual spatial position, and a left-eye perspective transform matrix for having a converted image represented by the converted transform video signal seen through on a screen; and

a right-eye processor for generating right-eye stereoscopic video signal from said source video signal based on said spatial image transform matrix and a right-eye perspective transform matrix for having a converted image represented by said transform video signal seen through on the screen, wherein;

said spatial image transform matrix T₀ is defined as follows;

##EQU29## said left-eye perspective transform matrix P_L is defined as follows;

##EQU30## said right-eye perspective transform matrix P_R is defined as follows;

##EQU31## provided that, r₁₁ to r₃₃ are the parameters representing spatial rotary transform;

l_x is the parameter representing parallel movement in the direction of x-axis;

l_y is the parameter representing parallel movement in the direction of y-axis;

l_z is the parameter representing parallel movement in the direction of z-axis;

S_o is the parameter representing two-dimensional extension and reduction;

P_z is the perspective value previously set; and

Δ

x is the value representing a half of distance between the right-eye and left-eye.

2. The stereoscopic image generation apparatus according to claim 1, wherein:

said left-eye perspective conversion matrix P_L is obtained as follows;

space="preserve" listing-type="equation">P.sub.L =LP.sub.0 L.sup.-1

said right-eye perspective conversion matrix P_R is obtained as follows;

space="preserve" listing-type="equation">P.sub.R =L.sup.-1 LP.sub.0

provided that, the matrix L is defined as follows;

##EQU32## the matrix L^-1 is defined as follows;

##EQU33## and the matrix P₀ is defined as follows;

##EQU34##

3. The stereoscopic image generation apparatus according to claim 2, wherein:

said left-eye processor comprises

first storing means for storing said source video signal temporarily; and

first read-out address generation means for generating read-out address, which is calculated based on said spatial image transform matrix and said left-eye perspective transform matrix, for reading out the video signal stored in said first storing means;

said right-eye processor comprises

second storing means for storing said source video signal temporarily; and

second read-out address generation means for generating read-out address, which is calculated based on said spatial transform matrix and said right-eye perspective transform matrix, for reading out the video signal stored in said second storing means.

4. The stereoscopic image generation apparatus according to claim 3, wherein:

read-out address (X_Lm, Y_Lm) supplied to said first storing means are as follows;

##EQU35## provided that, the definition will be made by the following equation;

##EQU36## read-out address (X_Rm, Y_Rm) supplied to said second storing means are;

##EQU37## provided that, the definition will be made as follows;

##EQU38## and, X_s and Y_s are screen addresses corresponding to the raster scanning.

5. The stereoscopic image generation apparatus according to claim 4, wherein

said same screen addresses are sequentially supplied to both of said first read-out address generation means and said second read-out address generation means.

6. The stereoscopic image generation apparatus according to claim 3, comprising:

write-in address generation means for generating write-in address for writing said source video signal in said first storing means and said second storing means, and wherein;

the same write-in addresses are supplied from said write-in address generation means to said first storing means and said second storing means, respectively.

7. A stereoscopic image generation apparatus for generating video signal in order to obtain stereoscopic effects visually, comprising:

a left-eye processor for generating left-eye stereoscopic video signal from source video signal based on a spatial image transform matrix for image transform which converts an image represented by said source video signal to virtual spatial position, and a left-eye perspective transform matrix for having a converted image represented by the converted transform video signal seen through on a screen; and

a right-eye processor for generating right-eye stereoscopic video signal from said source video signal based on said spatial image transform matrix and a right-eye perspective transform matrix for having a converted image represented by said transform video signal seen through on the screen, further comprising;

a first recording/reproducing device for recording the output video signal output from said left-eye processor to a recording medium and for reproducing the recorded video signal from the recording medium;

a second recording/reproducing device for recording the output video signal output from said right-eye processor to a recording medium and for reproducing the recorded video signal from the recording medium;

first key signal generation means for generating key signal from the first reproduced video signal reproduced from said first recording/reproducing device;

second key signal generation means for generating key signal from the second reproduced video signal reproduced from said second recording/reproducing device, and wherein;

said left-eye processor

receives the first reproduced video signal reproduced from said first recording/reproducing device, the key signal generated by said first key signal generation means, and background video signal used for the background of said source video signal;

executes first spatial image conversion to said first reproduced video signal and the same spatial image transform as said first spatial image transform to said key signal based on said spatial image transform matrix and said left-eye perspective transform matrix; and

composites the video signal converted by said first image transform means with said background video signal based on the key signal converted by said first image transform means; and

said right-eye processor

receives the second reproduced video signal reproduced from said second recording/reproducing device, the key signal generated by said second key signal generation means, and said background video signal;

executes the second spatial image transform to said second reproduced video signal and the same spatial image transform as said second spatial image transform to said key signal based on said spatial image transform matrix and said right-eye perspective transform matrix; and

composites the video signal converted by said second image transform means with said background video signal based on the key signal converted by said second image transform means.

8. The stereoscopic image generation apparatus according to claim 7, wherein

the key signal generated by said first key signal generation means is the same signal as the key signal generated by said second key signal generation means.

9. A video signal generation apparatus for generating video signal for visual stereoscopic effects, comprising:

image transform means for converting an image represented by source video signal to a virtual spatial position;

left-eye video signal generation means for generating left-eye video signal representing an image seen trough on a screen surface based on the virtual spatial position where there is the image of the converted video signal converted by said image transform means and left-eye spatial position; and

right-eye video signal generation means for generating right-eye video signal representing an image seen through on a screen surface based on the virtual spatial position where there is the image of said converted video signal and right-eye spatial position, wherein;

said image transform means converts said source video signal based on spatial image transform matrix;

said left-eye video signal generation means generates said left-eye video signal based on left-eye perspective transform matrix; and

said right-eye video signal generation means generates said right-eye video signal based on right-eye perspective transform matrix, wherein;

said spatial image transform matrix T₀ is defined as follows;

##EQU39## said left-eye perspective transform matrix P_L is defined as follows;

##EQU40## said right-eye perspective transform matrix P_R is defined as follows;

##EQU41## provided that, r₁₁ to r₃₃ are parameters representing spatial rotary transform;

l_x is the parameter representing parallel movement in the direction of x-axis;

l_y is the parameter representing parallel movement in the direction of y-axis;

l_z is the parameter representing parallel movement in the direction of z-axis;

S_o is the parameter representing two-dimensional extension and reduction; and

Δ

x is a value showing a half of distance of a right eye and a left-eye.

10. The video signal generation apparatus according to claim 9, wherein:

said left-eye perspective transform matrix P_L is obtained as follows;

space="preserve" listing-type="equation">P.sub.L =LP.sub.0 L.sup.-1

said right-eye perspective transform matrix P_R is obtained as follows;

space="preserve" listing-type="equation">P.sub.R =L.sup.31 1 LP.sub.0

proveded that, the matrix L is defined as follows;

##EQU42## the matrix L^-1 is defined as follows;

##EQU43## and the matrix P₀ is defined as follows;

##EQU44##

11. The video signal generation apparatus according to claim 10, wherein:

said left-eye video signal generation means comprises

first storing means for storing said source video signal temporarily; and

first read-out address generation means for generating read-out address for reading out the video signal which is calculated based on said spatial image transform matrix and said left-eye perspective transform matrix and is stored in said first storing means;

said right-eye video signal generation means comprises

second storing means for storing said source video signal temporarily; and

second read-out address generation means for generating read-out address for reading out the video signal which is calculated based on said spatial image transform matrix and said right-eye perspective transform matrix and is stored in said second storing means.

12. The video signal generation apparatus according to claim 11, wherein:

the read-out address (X_Lm, Y_Lm) supplied to said first storing means is as follows;

##EQU45## provided that, definition is done as follows;

##EQU46## the read-out address (X_Rm, Y_Rm) supplied to said second storing means is as follows;

##EQU47## provided that, definition is done as follows;

##EQU48## and provided that, X_s and Y_s are screen addresses.

13. The video signal generation apparatus according to claim 12, wherein

the same screen addresses are sequentially supplied to both said first read-out address generation means and said second read-out address generation means corresponding to raster scanning.

14. The video signal generation apparatus according to claim 11, comprising:

write-in address generation means for generating write-in address for writing said source video signal in said first storing means and said second storing means, and wherein;

same write-in address is supplied from said write-in address generation means to the write-in address supplied to said first storing means and said second storing means.

15. A video signal generation apparatus for generating video signal for visual stereoscopic effects, comprising:

image transform means for converting an image represented by source video signal to a virtual spatial position;

left-eye video signal generation means for generating left-eye video signal representing an image seen trough on a screen surface based on the virtual spatial position where there is the image of the converted video signal converted by said image transform means and left-eye spatial position; and

right-eye video signal generation means for generating right-eye video signal representing an image seen through on a screen surface based on the virtual spatial position where there is the image of said converted video signal and right-eye spatial position, wherein;

said image transform means converts said source video signal based on spatial image transform matrix;

said left-eye video signal generation means generates said left-eye video signal based on left-eye perspective transform matrix; and

said right-eye video signal generation means generates said right-eye video signal based on right-eye perspective transform matrix, comprising;

first recording/reproducing device for recording output video signal output from said left-eye video signal generation means to a recording medium and for reproducing the recorded video signal from the recording medium;

second recording/reproducing device for recording output video signal output from said right-eye video signal generation means to a recording medium and for reproducing the recorded video signal from the recording medium;

first key signal generation means for generating key signal from the first reproduced video signal reproduced from said first recording/reproducing device; and

second key signal generation means for generating key signal from the second reproduced video signal reproduced from said second recording/reproducing device, wherein;

said left-eye video signal generation means

receives the first reproduced video signal reproduced from said first recording/reproducing device, the key signal generated by said first key signal generation means, and background video signal used for the background of said source video signal;

executes first spatial image transform to said first reproduced video signal and the same spatial image transform as said first spatial image transform to said key signal based on said spatial image transform matrix and said left-eye perspective transition matrix; and

composites the video signal converted by said first image transform means with said background video signal based on the key signal converted by said first image transform means;

wherein

said right-eye processor

receives the second reproduced video signal reproduced from said second recording/reproducing device, the key signal generated by said second key signal generation means and said background video signal;

executes second spatial image transform to said second reproduced video signal and the same spatial image transform as said second spatial image transform to said key signal based on said spatial image transform matrix and said right-eye perspective transform matrix; and

composites the video signal converted by said second image transform means with said background video signal based on the key signal converted by said second image transform means.

16. The video signal generation apparatus according to claim 15, wherein

the key signal generated by said first key signal generation means is the same key signal as the key signal generated by said second key signal generation means.

17. A video signal generation method for generating video signal for obtaining the virtual stereoscopic effects, comprising:

(a) first step of converting source video signal to virtual spatial position;

(b) second step of generating left-eye video signal seen through on a screen surface based on the virtual spatial position of the converted video signal converted at said first step and spatial position of the left eye; and

(c) third step of generating right-eye video signal seen through on a screen surface based on the virtual spatial position of the converted video signal converted at said first step and spatial position of the right eye, wherein;

said source video signal is converted at said first step based on spatial image transform matrix having parameters corresponding to spatial transform operation which the operator desires;

said left-eye video signal is generated at said second step based on left-eye perspective transform matrix having parameters previously set by the operator; and

said right-eye video signal is generated at said third step based on right-eye perspective transform matrix having parameters previously set by the operator, wherein

said spatial image transform matrix T₀ is defined as follows;

##EQU49## said left-eye perspective transform matrix P_L is defined as follows;

##EQU50## said right-eye perspective transform matrix P_R is defined as follows;

##EQU51## provided that, r₁₁ to r₃₃ are the parameters representing spatial rotary transform;

l_x is the parameter representing parallel movement in the direction of x-axis;

l_y is the parameter representing parallel movement in the direction of y-axis;

l_z is the parameter representing parallel movement in the direction of z-axis;

S_o is the parameter representing two-dimensional extension and reduction;

P_z is the perspective value previously set; and

Δ

x is the value representing a half of distance between the right-eye and left-eye.

18. The video signal generation apparatus according to claim 17, wherein:

said left-eye perspective transform matrix P_L is obtained as follows;

space="preserve" listing-type="equation">P.sub.L =LP.sub.0 L.sup.-1

said right-eye perspective transform matrix P_R is obtained as follows;

space="preserve" listing-type="equation">P.sub.R =L.sup.-1 LP.sub.0

provided that, the matrix L is defined as follows;

##EQU52## the matrix L^-1 is defined as follows;

##EQU53## and the matrix P₀ is defined as follows;

##EQU54##

19. The video signal generation method according to claim 18, further comprising the steps of:

temporarily storing said source video signal to first storing means and second storing means;

generating left-eye read-out address for reading out the video signal stored in said first storing means based on said spatial image transform matrix and said left-eye perspective transform matrix; and

generating right-eye read-out address for reading out the video signal stored in said second storing means based on said spatial image transform matrix and said right-eye perspective transform matrix.

20. The video signal generation method according to claim 19, wherein:

said left-eye read-out address (X_Lm, Y_Lm) are;

##EQU55## provided that;

##EQU56## said right-eye read-out address (X_Rm, Y_Rm) are as follows;

##EQU57## provided that, it is defined as follows;

##EQU58## and, X_s and Y_s are screen addresses corresponding to the raster scanning.