Method of converting two-dimensional images into three-dimensional images

• US 5,808,664 A
• Filed: 07/11/1995
• Issued: 09/15/1998
• Est. Priority Date: 07/14/1994
• Status: Expired due to Term

• Current Assignee: Vision3D Technologies, LLC
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Yamashita, Shugo, Okino, Toshiyuki, Murata, Haruhiko, Iinuma, Toshiya, Mori, Yukio, Okada, Seiji
• Primary Examiner(s): Britton, Howard W.
• Assistant Examiner(s): Snow, Frank
• Application Number:
  US08/500,620
• Time in Patent Office: 1,162 Days
• Field of Search: 348/42, 348/43, 348/44, 348/49, 348/56, 348/50, 348/169, 348/170, 348/416, 348/420, 348/700
• US Class Current: 348/42
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 3/36 Scanning of motion picture films, e.g. for telecine
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A method of converting two-dimensional images into three-dimensional images by producing from a two-dimensional image signal a first image signal which forms a basis and a second image signal delayed by a predetermined number of fields from the first image signal and taking one of the signals as a left eye image signal and the other as a right eye image signal. It is judged whether or not a scene represented by the two-dimensional image signal is changed. When it is judged that the scene is changed, delay indicating how many fields are there from a field corresponding to the first image signal to a field corresponding to the second image signal is set to a predetermined value, for example, zero.

Claims

1. A method of converting two-dimensional images into three-dimensional images by producing from a two-dimensional image signal a first image signal which forms a basis and a second image signal delayed by a predetermined number of fields from said first image signal and taking one of the signals as a left eye image signal and the other signal as a right eye image signal, comprising:

a first step of judging whether or not a scene represented by said two-dimensional image signal is changed; and

a second step of taking delay indicating how many fields are there from a field corresponding to said first image signal to a field corresponding to said second image signal as a predetermined value when it is judged that the scene is changed.

2. The method according to claim 1, wherein

said delay is made zero when it is judged that the scene is changed in said second step.

3. The method according to claim 1, wherein

it is judged in said first step whether or not the scene is changed on the basis of a value related to a motion vector detected for each field in said two-dimensional image signal.

4. The method according to claim 3, wherein

it is judged that the scene is changed when a value related to a motion vector detected in a certain field in said two-dimensional image signal is larger by not less than a predetermined value than values related to motion vectors detected in fields preceding and succeeding the certain field.

5. The method according to claim 3, wherein

a plurality of motion vector detecting areas are set in one field in said two-dimensional image signal, and

the sum of values related to motion vectors detected for the motion vector detecting areas in each of fields is used as a value related to said motion vector detected in the field.

6. The method according to claim 4, wherein

a plurality of motion vector detecting areas are set in one field in said two-dimensional image signal, and

the sum of values related to motion vectors detected for the motion vector detecting areas in each of fields is used as a value related to said motion vector detected in the field.

7. The method according to claim 3, wherein

in a case where a value related to a motion vector detected in the t-th field in said two-dimensional image signal is taken as MX(t), and the average value of values related to motion vectors detected in a predetermined number of fields preceding a field which is two fields preceding the t-th field is taken as MXave(t-2),

it is judged that the scene is changed in the (t-1)-th field when the following expression (1) is satisfied:

space="preserve" listing-type="equation">MX(t-1)>

>

MXave(t-2)∩

MX(t-1)>

>

MX(t) (1).

8. The method according to claim 7, wherein

a plurality of motion vector detecting areas are set in one field in said two-dimensional image signal, and

a value MX(t) related to a motion vector detected in the t-th field is represented by the following equation (2):

##EQU6## Nx;

the number of motion vector detecting areas in the transverse direction of a screen

Ny;

the number of motion vector detecting areas in the longitudinal direction of the screen

M_ij (t);

a motion vector in the t-th field

ex;

a unit vector in the transverse direction.

9. The method according to claim 2, wherein

in a case where a value related to a motion vector detected in a certain field in said two-dimensional image signal is larger by not less than a predetermined value than the average value of values related to motion vectors respectively detected in several fields preceding the certain field and is larger by not less than a predetermined value than a value related to a motion vector detected in a field which is one field succeeding said certain field,

it is judged that the scene is changed.

10. The method according to claim 9, wherein

the average value MXave(t) of values related to motion vectors respectively detected in several fields preceding the t-th field is represented by the following equation (3):

##EQU7## S;

the number of fields for finding the average value MX(t+1-k) ;

the sum of values related to motion vectors in the (t+1-k)-th field.

11. The method according to claim 10, wherein

a plurality of motion vector detecting areas are set in one field in said two-dimensional image signal, and

the sum of values related to motion vectors respectively detected for the motion vector detecting areas in each of the fields is used as a value related to the motion vector detected in the field.

12. The method according to claim 1, further comprising the step of

erasing, in a case where it is judged that the scene is changed, values related to motion vectors in fields preceding a field where the scene is changed which have been stored.

13. The method according to claim 1, wherein

said first step comprises the steps of

finding a luminance level for each field in said two-dimensional image signal,

judging whether or not the amount of the change in the luminance level between fields in said two-dimensional image signal is larger than a predetermined value, and

judging that the scene is changed when it is judged that the amount of the change in the luminance level between the fields is larger than the predetermined value.

14. The method according to claim 1, wherein

said first step comprises the steps of

finding a chrominance level for each field in said two-dimensional image signal,

judging whether or not the amount of the change in the chrominance level between fields in said two-dimensional image signal is larger than a predetermined value, and

judging that the scene is changed when it is judged that the amount of the change in the chrominance level between the fields is larger than the predetermined value.

15. The method according to claim 1, wherein

said first step comprises the steps of

counting the number of horizontal synchronizing signals included in said two-dimensional image signal every time a vertical synchronizing signal is extracted from said two-dimensional image signal until the succeeding vertical synchronizing signal is extracted,

judging whether or not asynchronization occurs in said two-dimensional image signal on the basis of the number of horizontal synchronizing signals obtained, and

judging that the scene is changed when it is judged that asynchronization occurs in said two-dimensional image signal.

16. The method according to claim 1, wherein

said two-dimensional image signal is a television broadcasting signal, and

said first step is the step of judging that the scene is changed when a channel switching signal produced when a television channel is switched is detected.

17. The method according to claim 1, wherein

said first step is the step of judging that the scene is changed when a source switching signal produced when a source of said two-dimensional image signal is switched is detected.

18. The method according to claim 1, wherein

said two-dimensional image signal is a telecine converted image signal, and

said first step is the step of judging whether or not the scene is changed on the basis of a value related to a motion vector detected for each field in the telecine converted image signal.

19. The method according to claim 18, wherein

it is judged whether or not the scene is changed on the basis of a value related to a motion vector detected in a field where a frame of a cinema film (hereinafter referred to as a frame) is changed in the telecine converted image signal.

20. The method according to claim 19, wherein

in a case where a value related to a motion vector detected in a predetermined field where a frame is changed in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in a field where a frame which is one frame preceding the frame changed in the predetermined field is changed,

it is judged that the scene is changed in said predetermined field.

21. The method according to claim 20, wherein

a value related to a motion vector is the minimum value of accumulated correlation values obtained at the time of detecting a motion vector by a representative point matching method.

22. The method according to claim 19, wherein

in a case where a value related to a motion vector detected in a predetermined field where a frame is changed in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in a field where a frame which is one frame preceding the frame changed in the predetermined field is changed and a value related to a motion vector detected in a field where a frame which is two frames preceding the frame changed in the predetermined field is changed,

it is judged that the scene is changed in said predetermined field.

23. The method according to claim 22, wherein

a value related to a motion vector is the minimum value of accumulated correlation values obtained at the time of detecting a motion vector by a representative point matching method.

24. The method according to claim 19, wherein

in a case where a value related to a motion vector detected in a predetermined field where a frame is changed in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in a field where a frame which is one frame preceding the frame changed in the predetermined field is changed and a value related to a motion vector detected in a field where a frame which is two frames preceding the frame changed in the predetermined field is changed, and

the absolute value of the difference between the value related to the motion vector detected in said predetermined field where the frame is changed and the value related to the motion vector detected in the field where the frame which is one frame preceding the frame changed in the predetermined field is changed is not less than the absolute value of the difference between the value related to the motion vector detected in the field where the frame which is one frame preceding the frame changed in the predetermined field is changed and the value related to the motion vector detected in the field where the frame which is two frames preceding the frame changed in the predetermined field is changed,

it is judged that the scene is changed in said predetermined field.

25. The method according to claim 24, wherein

a value related to a motion vector is the minimum value of accumulated correlation values obtained when a motion vector is detected by a representative point matching method.

26. The method according to claim 18, wherein

said first step comprises the steps of

judging, in a case where a value related to a motion vector detected in a first predetermined field where a frame is changed in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in a field where a frame which is one frame preceding the frame changed in the first predetermined field is changed, that the scene is changed in said first predetermined field, and

judging, when in a case where a value related to a motion vector detected in a second predetermined field which is one field preceding the current field in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in said current field and is larger by not less than a predetermined value than a value related to a motion vector detected in a field which is one field preceding said second predetermined field, the value related to the motion vector detected in said second predetermined field is larger by not less than a predetermined value than a value related to a motion vector detected in each of fields succeeding the second predetermined field over fields up to a field where a frame is thereafter changed in the telecine converted image signal, that the scene is changed in said second predetermined field.

27. The method according to claim 26, wherein

a value related to a motion vector is the minimum value of accumulated correlation values obtained at the time of detecting a motion vector by a representative point matching method.

28. The method according to claim 18, further comprising the steps of

judging, in a case where a value related to a motion vector detected in a first predetermined field where a frame is changed in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in a field where a frame which is one frame preceding the frame changed in the first predetermined field is changed and a value related to a motion vector detected in a field where a frame which is two frames preceding the frame changed in the first predetermined field is changed, that the scene is changed in said first predetermined field, and

judging, when in a case where a value related to a motion vector detected in a second predetermined field which is one field preceding the current field in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in said current field and is larger by not less than a predetermined value than a value related to a motion vector detected in a field which is one field preceding said second predetermined field, the value related to the motion vector detected in said second predetermined field is larger by not less than a predetermined value than a value related to a motion vector detected in each of fields succeeding the second predetermined field over fields up to a field where a frame is thereafter changed in the telecine converted image signal, that the scene is changed in said second predetermined field.

29. The method according to claim 28, wherein

a value related to a motion vector is the minimum value of accumulated correlation values obtained at the time of detecting a motion vector by a representative point matching method.

30. The method according to claim 18, wherein

said first step comprises

judging, in a case where a value related to a motion vector detected in a first predetermined field where a frame is changed in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in a field where a frame which is one frame preceding the frame changed in the first predetermined field is changed and a value related to a motion vector detected in a field where a frame which is two frames preceding the frame changed in the first predetermined field is changed, and the absolute value of the difference between the value related to the motion vector detected in said first predetermined field where the frame is changed and the value related to the motion vector detected in the field where the frame which is one frame preceding the frame changed in the first predetermined field is changed is not less than the absolute value of the difference between the value related to the motion vector detected in the field where the frame which is one frame preceding the frame changed in the first predetermined field is changed and the value related to the motion vector detected in the field where the frame which is two frames preceding the frame changed in the first predetermined field is changed, it is judged that the scene is changed in said first predetermined field, and

judging, when in a case where a value related to a motion vector detected in a second predetermined field which is one field preceding the current field in the telecine converted image signal is larger by not less than a predetermined value than a value related to a motion vector detected in said current field and is larger by not less than a predetermined value than a value related to a motion vector detected in a field which is one field preceding said second predetermined field, the value related to the motion vector detected in said second predetermined field is larger by not less than a predetermined value than a value related to a motion vector detected in each of fields succeeding the second predetermined field over fields up to a field where a frame is thereafter changed in the telecine converted image signal, that the scene is changed in said second predetermined field.

31. The method according to claim 28, wherein

a value related to a motion vector is the minimum value of accumulated correlation values obtained at the time of detecting a motion vector by a representative point matching method.