Method and apparatus for image enhancement

• US 6,075,555 A
• Filed: 10/02/1997
• Issued: 06/13/2000
• Est. Priority Date: 04/04/1995
• Status: Expired due to Term

• Current Assignee: Graham S.B. Street
• Original Assignee: Graham S.B. Street
• Inventors: Street, Graham S. B.
• Primary Examiner(s): Tung, Bryan
• Application Number:
  US08/942,365
• Time in Patent Office: 985 Days
• Field of Search: 348/42, 348/43, 348/46, 348/49, 348/51, 348/57, 348/61
• US Class Current: 348/43
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/225 using parallax barriers
• CPC Class Codes: H04N 13/229 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/31 using parallax barriers
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/334 using spectral multiplexing
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/346 using prisms or semi-transparent mirrors
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

Apparatus and method for image enhancement in which an autostereoscopic image, comprising a different perspective view of an object field for each eye of an observer is, typically provided. The optical cross-talk, which is often a feature of autostereoscopic images, is suppressed when images are displayed, by preconditioning of the image intended for each eye. Supplementary image data is produced by mixing the different perspective image data and this is applied to the primary data. By inversion of a cross-talk matrix, two or more different perspective channels can be preconditioned to cancel the effects of such cross-talk, providing clear images for each eye without the unwanted interference from other views. The viewer'"'"'s location with respect to the display device can be used to modify the characteristics of both the display and the level of cross-talk correction. A camera or endoscope for providing stereoscopic images can also suffer from optical cross-talk. The imagery captured by such a system may be improved by applying the same principles as are applied to the display process.

Claims

1. Apparatus for use in displaying stereoscopic images by substantially simultaneous display of a plurality of perspective views comprising:

display means having a non-linear voltage-light transfer characteristic;

means for preconditioning first and second primary image data, said first and second primary image data corresponding respectively to a first and a second two-dimensional perspective view to be displayed, after preconditioning by said preconditioning means, for viewing by respectively the first and the second eye of an observer, said preconditioning means being arranged to combine first supplementary image data with said primary image data and second supplementary image data with said second primary image data, wherein the first supplementary image data is derived from at least the second primary image data and the second supplementary image data is derived from at least the first primary image data,

wherein said display means includes means with a non-linear characteristic for driving said display means, so that, in use, the light transmitted to the observer from a point on said display means is linearly related to the signal input to said driving means corresponding to said point.

2. Apparatus as claimed in claim 1, further comprising means for modifying the non-linear characteristic of said driving means as a function of the observer'"'"'s location with respect to said display means.

3. Apparatus as claimed in claim 2 in which the preconditioning of image data is modified, in use, as a function of the observer'"'"'s head position.

4. Apparatus as claimed in claim 3 in which the derived supplementary data is a function of both the primary data from which it is derived and the location within the image corresponding to said primary data.

5. Apparatus as claimed in claim 4 in which the preconditioning means comprises means for imposing an overall neutral bias on the signals for both eyes.

6. Apparatus as claimed in claim 5, further comprising means for capturing image data corresponding to perspective views of an object field from at least two real or virtual pupil locations, said pupils having spaced centers of gravity, so that, in use, light corresponding to different perspectives is passed at respective pupil locations to provide the first and the second primary image data.

7. Apparatus as claimed in claim 1 in which the preconditioning of image data is modified, in use, as a function of the observer'"'"'s head position data.

8. Apparatus as claimed in claim 1 in which the derived supplementary data is a function of both the primary data from which it is derived and the location within the image corresponding to said primary data.

9. Apparatus as claimed in claim 1 in which the preconditioning means comprises means for imposing an overall neutral bias on the signals for both eyes.

10. Apparatus as claimed in claim 1, further comprising means for capturing image data corresponding to perspective views of an object field from at least two real or virtual pupil locations, said pupils having spaced centers of gravity, so that, in use, light corresponding to different perspectives is passed at respective pupil locations to provide the first and the second primary image data.

11. Apparatus as claimed in claim 10, further comprising a transparent birefringent element intermediate the object field and a real pupil, whereby the two virtual pupils having spaced centers of gravity are provided, so that, in use, a first virtual pupil passes light polarized in a first direction and corresponding to a first perspective view of the object field, and the second virtual pupil passes light polarized in a second direction and corresponding to a second perspective view of the object field.

12. Apparatus as claimed in claim 11, further comprising coherent optical fibre means to simultaneously relay the two polarized different perspective views of the object field to the image capture means, and means for separating said polarized components prior to image capture, to provide, in use, the first and second primary image data.

13. Apparatus as claimed in claim 12, in which the two polarized views are circularly polarized, respectively clockwise and counter-clockwise.

14. Apparatus as claimed in claim 11, wherein the first polarization direction is orthogonal to the second polarization direction.

15. Apparatus as claimed in claim 1, wherein said display means includes a liquid crystal display.

16. Apparatus for capturing at least first and second perspective views of an object field, comprising:

image capture means;

optical means for transmission of the views to said image capture means, in which said optical means causes light from the at least first and second views to suffer some optical cross-talk to form at least a first and a second composite image on said capture means; and

processing means,

wherein said optical means includes at least two real or virtual pupil locations, said pupils having spaced centers of gravity;

said image capture means provides a linear response to light impinging thereon; and

said processing means, in use, combines in a linear manner the video data comprising the first composite image with the video data comprising the second composite image to provide first and second primary image data corresponding substantially to the first and the second perspective views of the object field.

17. Apparatus as claimed in claim 16, further comprising a transparent birefringent element intermediate the object field and a real pupil, whereby the two virtual pupils having spaced centers of gravity are provided, so that, in use, a first virtual pupil passes light polarized in a first direction and corresponding to a first perspective view of the object field, and the second virtual pupil passes light polarized in a second direction and corresponding to a second perspective view of the object field.

18. Apparatus as claimed in claim 17, wherein the first polarization direction is orthogonal to the second polarization direction.

19. Apparatus as claimed in claim 18, further comprising coherent optical fibre means to simultaneously relay the two orthogonally polarized different perspective views of the object field to the image capture means, and means for separating said polarized components prior to image capture, to provide, in use, the first and second primary image data.

20. Apparatus as claimed in claim 19, in which the two orthogonally polarized views are circularly polarized, respectively clockwise and counter-clockwise.

21. Apparatus as claimed in claim 16, further comprising coherent optical fibre means to simultaneously relay two orthogonally polarized different perspective views of the object field to the image capture means, and means for separating said polarized components prior to image capture, to provide, in use, the first and second primary image data.

22. A method for use in displaying stereoscopic images by substantially simultaneous display of a plurality of perspective views comprising:

preconditioning first and second primary image data, said first and second primary data corresponding respectively to a first and a second two-dimensional perspective view, by deriving first supplementary image data from at least the second primary image data and second supplementary image data from at least the first primary image data;

combining said first supplementary image data with said first primary image data, and said second supplementary image data with said second primary image data; and

displaying on at least one liquid crystal display said first preconditioned image data to a first eye of an observer and said second preconditioned image data to the second eye of said observer, said displaying including driving the liquid crystal display with a non-linear characteristic so that, in use, the light transmitted to the observer is linearly related to the signal input to said driving step after said combining step.

23. A method as claimed in claim 22 in which preconditioning primary data comprises constructing the inverse of a matrix representing the optical cross talk between each perspective channel and the other perspective channels, said matrix being square and having a number of elements equal to the square of the number of perspective channels;

and applying said inverse matrix to said primary data.

24. A method as claimed in claim 23 in which preconditioning primary data includes sampling a complete image comprised of image data, determining the minimum value of the preconditioned data and applying a uniform bias to said preconditioned data to ensure that this does not represent a demand for negative values to be displayed by the display means.

25. A method as claimed in claim 22 in which preconditioning primary data includes sampling a complete image comprised of image data, determining the minimum value of the preconditioned data and applying a uniform bias to said preconditioned data to ensure that this does not represent a demand for negative values to be displayed by the display means.

26. A method for capturing at least first and second perspective views of an object field which includes the steps of:

providing at least two real or virtual pupils with spaced centers of gravity;

providing at least one CCD having a linear response to light impinging thereon;

transmitting said perspective views to the CCD via the pupils, thereby causing light from the at least first and second views to suffer some cross-talk resulting in the formation of at least first and second composite images on the CCD; and

combining in a linear manner at least the captured image data corresponding to the first composite image with captured image data corresponding to the second composite image thereby providing first and second primary image data corresponding substantially to the first and the second perspective views of the object field.

27. A method for preconditioning image data to be displayed, the image data including first and second primary image signals corresponding to respective views to be displayed simultaneously, said method comprising:

deriving first supplementary image data from at least the second primary image signal, and deriving second supplementary image data from at least the first primary image signal, the first and second primary image signals used in the deriving steps corresponding to respective views to be displayed simultaneously;

combining the first primary image signal with said first supplementary image data, thereby producing a first preconditioned signal;

combining the second primary image signal with said second supplementary image data, thereby producing a second preconditioned signal; and

forwarding said first and second preconditioned signals for display.

28. A method as claimed in claim 27, wherein each of the first and second primary image signals is a color signal.

29. A method as claimed in claim 28, wherein the first and second primary image signals have no color in common.

30. A method as claimed in claim 28, wherein the first primary image signal includes at least one of Red, Green, and Blue;

and the second primary image signal includes at least one of the Red, Green, and Blue excluded from the first primary image signal.

31. A method as claimed in claim 30, wherein, during a first time interval, the first primary image signal includes Red and Blue and the second primary image signal includes Green;

and further wherein, during a second time interval, the first primary image signal includes Green and the second primary image signal includes Red and Blue.

32. A method as claimed in claim 31, wherein the second time interval is immediately subsequent to the first time interval such that the first and second time intervals are alternating fields.

33. A method as claimed in claim 32, further comprising the step of displaying said first preconditioned image signal to one eye of an observer, and displaying said second preconditioned image signal to the observer'"'"'s other eye.

34. A method as claimed in claim 27, wherein the first and second primary image signals are real-time image signals.

35. A method as claimed in claim 27, further comprising the step of displaying said first preconditioned image signal to one eye of an observer, and displaying said second preconditioned image signal to the observer'"'"'s other eye.

36. Apparatus for preconditioning image data to be displayed, the image data including first and second primary image signals corresponding to respective views to be displayed simultaneously, said apparatus comprising:

a deriving circuit for deriving first supplementary image data from at least the second primary image signal, and for deriving second supplementary image data from at least the first primary image signal, the first and second primary image signals corresponding to respective views to be displayed simultaneously;

a combining circuit for combining the first primary image signal with said first supplementary image data, thereby producing a first preconditioned signal; and

for combining the second primary image signal with said second supplementary image data, thereby producing a second preconditioned signal; and

an output for outputting said first and second preconditioned signals.

37. Apparatus as claimed in claim 36, wherein each of the first and second primary image signals is a color signal.

38. Apparatus as claimed in claim 37, wherein the first and second primary image signals have no color in common.

39. Apparatus as claimed in claim 37, wherein the first primary image signal includes at least one of Red, Green, and Blue;

and the second primary image signal includes at least one of the Red, Green, and Blue excluded from the first primary image signal.

40. Apparatus as claimed in claim 39, wherein, during a first time interval, the first primary image signal received by said deriving circuit includes Red and Blue and the second primary image signal includes Green;

and further wherein, during a second time interval, the first primary image signal received by said deriving circuit includes Green and the second primary image signal includes Red and Blue.

41. Apparatus as claimed in claim 40, wherein the second time interval is immediately subsequent to the first time interval.

42. Apparatus as claimed in claim 41, further comprising a display for displaying said first preconditioned image signal to one eye of an observer, and for displaying said second preconditioned image signal to the observer'"'"'s other eye.

43. Apparatus as claimed in claim 36, wherein the first and second primary image signals are real-time image signals.

44. Apparatus as claimed in claim 36, further comprising a display for displaying said first preconditioned image signal to one eye of an observer, and for displaying said second preconditioned image signal to the observer'"'"'s other eye.