Method and device for obtaining a stereoscopic signal

• US 20060036383A1
• Filed: 07/13/2005
• Published: 02/16/2006
• Est. Priority Date: 07/13/2004
• Status: Abandoned Application

• Current Assignee: Canon Finetech Incorporated (Canon Inc.)
• Original Assignee: Canon Finetech Incorporated (Canon Inc.)
• Inventors: Gisquet, Christophe, Clare, Maryline, Henry, Fellx
• Application Number:
  US11/179,490
• Publication Number:
  US 20060036383A1
• Time in Patent Office: Days
• US Class Current: 702/85
• CPC Class Codes: G03B 35/10 having single camera with stereoscopic-base-defining system
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

The invention relates to a method of obtaining a stereoscopic signal from a sequence of monoscopic images. The method firstly comprises a step (S10) of obtaining a sequence of monoscopic images having been captured by an image acquisition apparatus in an acquisition mode enabling several images to be shot in the course of a regular movement substantially tangential to the plane of the lens of the acquisiton apparatus. That step is followed by a step (S31) of forming pairs of images from the sequence of images, each pair being formed on the basis of a predetermined temporal distance then a step of calibrating (S32) the images of the pairs formed, so as to improve the visual correspondence between the two images; Finally, a stereoscopic signal is constructed (S33) from the pairs so calibrated. FIG. 3.

Claims

1. A method of obtaining a stereoscopic signal from a sequence of monoscopic images comprising the following steps:

obtaining (S10) a sequence of monoscopic images having been captured by an image acquisition apparatus in an acquisition mode enabling several images to be shot in the course of a regular movement substantially tangential to the plane of the lens of the acquisition apparatus.

forming (S31) pairs of images from the sequence of images, each pair being formed on the basis of a predetermined temporal distance;

calibrating (S32) the images of the pairs formed, so as to improve the visual correspondence between the two images;

constructing (S33) a stereoscopic signal from the pairs so calibrated.

2. A method of obtaining a stereoscopic signal according to claim 1, wherein the predetermined temporal distance depends on the speed of acquisition of the images of the sequence of images.

3. A method of obtaining a stereoscopic signal according to claim 2, wherein the speed of acquisition of the images is deduced by the calculation of at least one movement vector between the images.

4. A method of obtaining a stereoscopic signal according to claim 1, wherein the step of forming a pair of images comprises the following sub-steps:

selecting (S41) an image of the sequence constituting the first image of the pair;

determining (S43) a group of images situated temporally at a distance that is close to the predetermined temporal distance with respect to the first image;

constructing (S44, S45) the second image of the pair from images of the group determined.

5. A method of obtaining a stereoscopic signal according to claim 4, wherein constructing the second image of the pair is performed by selecting the image situated at a temporal distance that is the closest to the predetermined distance.

6. A method of obtaining a stereoscopic signal according to claim 4, wherein constructing the second image of the pair is performed by interpolating at least a part of the images of the group determined (S45).

7. A method of obtaining a stereoscopic signal according to claim 1, wherein the calibrating step is performed by geometric readjustment (S61).

8. A method of obtaining a stereoscopic signal according to claim 1, wherein the calibrating step is performed by a readjustment of the signal (S63).

9. A method of obtaining a stereoscopic signal according to claim 8, wherein the readjustment of the signal is a luminance readjustment.

10. A method of obtaining a stereoscopic signal according to claim 7, wherein the geometric readjustment is a vertical readjustment.

11. A method of obtaining a stereoscopic signal according to claim 7, wherein the geometric readjustment is a rotational readjustment.

12. A method according to claim 11, wherein the rotational readjustment comprises the following steps:

defining an image part on an image to calibrate of the pair formed;

searching with respect to at least one block of predetermined size of the image part for a rotation with respect to a spatially corresponding block in the other image of the pair;

in case the search is positive,

verifying the correspondence of the rotation found with respect to at least one other part of the image to calibrate;

in case of positive verification,

correcting the image .to calibrate by performing the opposite rotation to the rotation found.

13. A method according to claim 12, wherein prior to the searching step it comprises a step of determining at least one significant block in the defined image part.

14. A method according to claim 13, wherein the block is significant if the value of the variance calculated with respect to the block is greater than a predetermined threshold.

15. A method according to claim 12, wherein in case of negative search or negative verification, the block size is decremented and the searching step is performed for that new block size.

16. A method according to claim 12, wherein the step of searching for a rotation comprises the following steps:

defining several rotation centers and several rotation angles;

for all the rotation centers and for all the rotation angles;

calculating similarity between the current block of the image to calibrate having undergone a rotation about one of the rotation centers and through one of the rotation angles, and the spatially corresponding block of the other image of the pair;

comparing the similarities so calculated, the greatest similarity being that corresponding to the rotation center and the rotation angle of the rotation to be found.

17. A method according to claim 16, wherein the step of verifying the correspondence of the rotation found comprises the steps of:

calculating similarity between the current block of the image part to calibrate having undergone a rotation about the rotation center and through the rotation angle of the rotation found and the spatially corresponding block of the other image of the pair;

comparing the similarity so calculated with a threshold, the verification being positive when said similarity is greater than said threshold.

18. A method of obtaining a stereoscopic signal according to claim 1, wherein constructing a stereoscopic signal is performed by grouping together (S33) the pairs of images formed so as to obtain a sequence of stereoscopic images.

19. A method of obtaining a stereoscopic signal according to claim 1, wherein constructing a stereoscopic signal is performed by selecting (S33) a pair of images from the pairs of images formed, so as to obtain a stereoscopic image.

20. A method of obtaining a stereoscopic signal according to claim 19, wherein selecting a pair is performed according to a criterion specific to the signal such as the variance of the histogram or the mathematical correlation between the images of the pair.

21. A method according to claim 19, wherein selecting a pair of images is performed via a user interface making it possible to vary the angles of view of the images and/or the depth of the images.

22. A method according to claim 21, wherein selecting a pair of images by the user interface is followed by the step of calibrating the images of the selected pair then displaying the stereoscopic image constructed from the calibrated images, the steps of selecting, calibrating and displaying being performed iteratively until validation is performed by the user.

23. A method according to claim 21, wherein the user interface makes it possible to change one of the two images of the pair or each of the two images of the pair by an image that is earlier or later with respect to the sequence of images captured.

46. An information storage means which can be read by a computer or a microprocessor storing instructions of a computer program, wherein it makes it possible to implement a method of obtaining a stereoscopic signal according to claim 1.

47. A partially or totally removable information storage means which can be read by a computer or a microprocessor storing instructions of a computer program, wherein it makes it possible to implement a method of obtaining a stereoscopic signal according to claim 1.

48. A computer program product which can be loaded into a programmable apparatus, comprising sequences of instructions for implementing a method of obtaining a stereoscopic signal according to claim 1, when the program is loaded and executed by the programmable apparatus.

24. A device for obtaining a stereoscopic signal from a sequence of monoscopic images comprising:

means (1) for obtaining a sequence of monoscopic images having been captured by an image acquisition apparatus in an acquisition mode enabling several images to be shot in the course of a regular movement substantially tangential to the plane of the lens of the acquisition apparatus.

means (21) for forming pairs of images from the sequence of images, each pair being formed in the basis of a predetermined temporal distance;

means (S32) for calibrating the images of the pairs formed, so as to improve the visual correspondence between the two images;

means (23) for constructing a stereoscopic signal from the pairs so calibrated.

25. A device according to claim 24, wherein the predetermined temporal distance depends on the speed of acquisition of the images of the sequence of images.

26. A device according to claim 25, further comprising means for calculating at least one movement vector between the images so as to deduce the speed of acquisition of the images.

27. A device according to claim 24, wherein the means for forming a pair of images comprise:

means for selecting an image of the sequence constituting the first image of the pair;

means for determining a group of images situated temporally at a distance that is close to the predetermined temporal distance with respect to the first image;

means for constructing the second image of the pair from images of the group determined.

28. A device according to claim 27, wherein the means for constructing the second image of the pair comprise means for selecting the image situated at a temporal distance that is the closest to the predetermined distance.

29. A device according to claim 27, wherein the means for constructing the second image of the pair comprise means for interpolating at least a part of the images of the group determined.

30. A device according to claim 24, wherein the calibrating means comprise means for readjustment of the signal.

31. A device according to claim 30, wherein the means for signal readjustment are means for luminance readjustment.

32. A device according to claim 24, wherein the calibrating means comprise means for geometric readjustment.

33. A device according to claim 32, wherein the means for geometric readjustment are means for vertical readjustment.

34. A device according to claim 32, wherein the means for geometric readjustment are means for rotational readjustment.

35. A device according to claim 32, wherein the means for rotational readjustment comprise:

means for defining an image part on an image to calibrate of the pair formed;

means for searching with respect to at least one block of predetermined size of the image part for a rotation with respect to a spatially-corresponding block in the other image of the pair;

means for verifying the correspondence of the rotation found with respect to at least one other part of the image to calibrate;

implementation in case of positive search by the searching means;

means for correcting the image to calibrate by performing the inverse rotation to the rotation found, implemented in case of positive verification by the verification means.

36. A device according to claim 35, further comprising means for determining at least one significant block in the defined image part.

37. A device according to claim 35, wherein the means for searching for a rotation comprise:

means for defining several rotation centers and several rotation angles;

means for calculating similarity between the current block of the image to calibrate having undergone a rotation about one of the rotation centers and through one of the rotation angles, and the spatially corresponding block of the other image of the pair, implemented for all the rotation centers and for all the rotation angles;

means for comparing the similarities so calculated, the greatest similarity being that corresponding to the rotation center and the rotation angle of the rotation to be found.

38. A device according to claim 37, wherein the means for verifying the correspondence of the rotation found comprise:

means for calculating similarity between the current block of the image part to calibrate having undergone a rotation about the rotation center and through the rotation angle of the rotation found and the spatially corresponding block of the other image of the pair;

means for comparing the similarity so calculated with a threshold, the verification being positive when said similarity is greater than said threshold.

39. A device according to claim 24, wherein the means for constructing a stereoscopic signal comprise means for grouping together the pairs of images formed so as to obtain a sequence of stereoscopic images.

40. A device according to claim 24, wherein the means for constructing a stereoscopic signal comprise means for selecting a pair of images from the pairs of images formed, so as to obtain a stereoscopic image.

41. A device according to claim 40, wherein the means for selecting a pair comprise means for calculating a criterion specific to the signal such as the variance of the histogram or the mathematical correlation between the images of the pair.

42. A device according to claim 40, wherein the means for selecting a pair of images comprise a user interface making it possible to vary the angles of view of the images and/or the depth of the images.

43. A device according to claim 42, wherein it comprises means for displaying the constructed stereoscopic image and means for validation by the user.

44. A device according to claim 42, wherein the user interface makes it possible to change one of the two images of the pair or each of the two images of the pair by an image that is earlier or later with respect to the sequence of images captured.

45. An image acquisition device, comprising a device according to claim 24.