System and method for capturing and viewing stereoscopic panoramic images

• US 7,477,284 B2
• Filed: 04/19/2001
• Issued: 01/13/2009
• Est. Priority Date: 09/16/1999
• Status: Expired due to Fees

• Current Assignee: Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. (Hebrew University of Jerusalem)
• Original Assignee: Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. (Hebrew University of Jerusalem)
• Inventors: Ben-Ezra, Moshe, Pritch, Yael, Peleg, Shmuel
• Primary Examiner(s): Vo; Tung
• Application Number:
  US09/861,859
• Publication Number:
  US 20040223051A1
• Time in Patent Office: 2,826 Days
• Field of Search: 348/36, 348/42, 348/51, 348/158, 348/218.1, 348/98, 348/44, 348/46, 348/48, 348/49, 348/52, 348/53, 382/154, 382/232, 382/294, 382/299, 382/284, 382/268, 352/57, 352 69- 70, 396/324
• US Class Current: 348/53
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems

Abstract

Method and apparatus for generating images of a scene from image data of the scene and displaying the images to provide a sense of depth. In some embodiments of the method and apparatus the generated images are mosaics.

Claims

1. Imaging apparatus comprising:

at least one imager that moves relative to a scene so as to acquire a plurality of optical images of at least portions of the scene, each of at least two of said optical images being viewed from a different respective viewing position;

a processor that receives image data representative of said at least two of the optical images and processes the data to divide each image into a plurality of segments and to generate a plurality of mosaics of the scene, such that;

each mosaic contains segments taken from different ones of said optical images;

segments relating to at least one part of the scene are derived from at least two optical images and appear in at least two mosaics;

the different segments of the two optical images in a given mosaic represent different parts of the scene; and

a display that receives a plurality of the mosaics and displays them so as to provide a sense of depth of the scene.

2. Imaging apparatus according to claim 1 wherein the imaging apparatus is a portable hand-held device including a housing for accommodating the at least one imager, the processor and the display.

3. Imaging apparatus according to claim 1 and comprising communication apparatus that transmits image data from the imaging apparatus.

4. Imaging apparatus according to claim 3 wherein the communication apparatus is housed in a housing that also accommodates the at least one imager, the processor and the display.

5. Imaging apparatus according to claim 3 wherein the image data transmitted by the communication apparatus comprises at least a portion of a mosaic.

6. Imaging apparatus according to claim 3 wherein the image data transmitted by the communication apparatus comprises image data provided by the at least one imager.

7. Imaging apparatus according to claim 3 wherein the communication apparatus comprises apparatus for transmitting the data over a wireless communication channel.

8. Imaging apparatus according to claim 7 wherein the communication channel comprises a cellular telephone channel.

9. Imaging apparatus according to claim 1 wherein the display displays at least two of the mosaics one after another sufficiently rapidly to provide the sense of depth.

10. Imaging apparatus according to claim 1 wherein the display displays the at least two mosaics so that each eye sees a different one of the mosaics.

11. Imaging apparatus according to claim 1 wherein mosaics of the plurality of mosaics comprise panoramic mosaics.

12. Imaging apparatus according to claim 1 wherein the processor adjusts disparity between mosaics of the plurality of mosaics to provide selected visual effects.

13. Imaging apparatus according to claim 12 wherein the processor adjusts disparity between mosaics of the plurality of mosaics to provide at least one selected disparity for objects in the scene.

14. Imaging apparatus according to claim 12 wherein the processor adjusts disparity between mosaics so that a disparity for objects at a finite distance in the scene is at least a disparity required for human stereo fusion.

15. Imaging apparatus according to claim 12 wherein the processor adjusts disparity between mosaics so that disparity for all objects in the scene is within a predetermined range.

16. Imaging apparatus according to claim 12 wherein the processor adjusts disparity for different objects in the scene differently.

17. Imaging apparatus according to claim 1 wherein the at least one imager acquires image data from the scene used to generate a mosaic at a same time.

18. Imaging apparatus according to claim 1 wherein the at least one imager acquires image data of the scene used to generate at least two mosaics of the plurality of mosaics at a same time.

19. Imaging apparatus according to claim 1, comprising a housing that accommodates the at least one imager, the processor and the display.

20. Imaging apparatus according to claim 1, wherein the processor is adapted to generate at least three series of image strips, each of said series comprising strips of a series of images of a scene as would be recorded by a camera from a respective series of positions relative to the scene, the image strips of the respective series representing strips of the respective images, said strips being displaced from one another by at least one selected displacement;

and being further adapted to mosaic the respective series of image strips together thereby to construct said plurality of mosaics of the scene.

21. Imaging apparatus according to claim 10, wherein the display is a lenticular print.

25. Imaging apparatus according to claim 1, wherein, in respect of any two of said optical images that both contribute at least two different segments to each of any two mosaics, the segments contributed by both optical images to a first one of the mosaics are all to the left of the segments contributed by both optical images to a second one of the mosaics.

22. Imaging apparatus comprising:

at least one imager that moves relative to a scene so as to acquire a plurality of optical images of at least portions of the scene, each of at least two of said optical images being viewed from a different respective viewing position;

a processor that receives image data representative of said at least two of the optical images and processes the data to divide each image into a plurality of segments and to generate a plurality of mosaics of the scene, such that each mosaic contains segments taken from different ones of said optical images and such that at least two of the optical images contributes a different segment to each of at least two of the mosaics and such that the different segments of the two optical images in a given mosaic represent different parts of the scene; and

a display that receives a plurality of the mosaics and displays them so as to provide a sense of depth of the scene.

23. Imaging apparatus according to claim 22 and comprising communication apparatus that transmits image data from the imaging apparatus.

24. Imaging apparatus according to claim 22 wherein the display displays at least two of the mosaics one after another sufficiently rapidly to provide the sense of depth.

26. Imaging apparatus comprising:

at least one imager that moves relative to a scene so as to acquire a plurality of optical images of at least portions of the scene, each of at least two of said optical images being viewed from a different respective viewing position;

a processor that receives image data responsive to said at least two of the optical images and processes the data to divide each image into a plurality of segments and to generate a plurality of mosaics of the scene, such that;

each mosaic contains segments taken from different ones of said optical images;

at least two of the optical images contributes a different segment to each of at least two of the mosaics; and

the different segments of the two optical images in a given mosaic represent different parts of the scene,

wherein respective first segments of each optical image contributed to a first mosaic image are always to the left of respective second segments of the respective optical image contributed to a second mosaic image; and

a display that receives a plurality of the mosaics and displays them so as to provide a sense of depth of the scene.

27. Imaging apparatus comprising:

at least one imager that moves relative to a scene so as to acquire a plurality of optical images of at least portions of the scene, each of at least two of said optical images being viewed from a different respective viewing position;

a processor that receives image data representative of said at least two of the optical images and processes the data to divide each image into a plurality of segments and to generate a plurality of mosaics of the scene, such that;

each mosaic contains segments taken from different ones of said optical images;

at least two different segments in each mosaic relate to a different part of the scene that appears in each of the two respective optical images;

segments relating to at least one part of the scene are derived from at least two optical images and appear in at least two mosaics; and

in respect of any two of said optical images that both contribute at least two different segments to each of any two mosaics, the segments contributed by both optical images to a first one of the mosaics are all to the left of the segments contributed by both optical images to a second one of the mosaics; and

a display that receives a plurality of the mosaics and displays them so as to provide a sense of depth of the scene.

28. Imaging apparatus according to claim 27 wherein the imaging apparatus is a portable hand-held device including a housing for accommodating the at least one imager, the processor and the display.

29. Imaging apparatus according to claim 27 and comprising communication apparatus that transmits image data from the imaging apparatus.

30. Imaging apparatus according to claim 29 wherein the communication apparatus is housed in a housing that also accommodates the at least one imager, the processor and the display.

31. Imaging apparatus according to claim 29 wherein the image data transmitted by the communication apparatus comprises at least a portion of a mosaic.

32. Imaging apparatus according to claim 29 wherein the image data transmitted by the communication apparatus comprises image data provided by the at least one imager.

33. Imaging apparatus according to claim 29 wherein the communication apparatus comprises apparatus for transmitting the data over a wireless communication channel.

34. Imaging apparatus according to claim 33 wherein the communication channel comprises a cellular telephone channel.

35. Imaging apparatus according to claim 27 wherein the display displays at least two of the mosaics one after another sufficiently rapidly to provide the sense of depth.

36. Imaging apparatus according to claim 27 wherein the display displays at least two of the mosaics so that each eye sees a different one of the mosaics.

37. Imaging apparatus according to claim 27, wherein the processor is adapted to generate at least three series of image strips, each of said series comprising strips of a series of images of a scene as would be recorded by a camera from a respective series of positions relative to the scene, the image strips of the respective series representing strips of the respective images, said strips being displaced from one another by at least one selected displacement;

and being further adapted to mosaic the respective series of image strips together thereby to construct said plurality of mosaics of the scene.

38. A method for processing image data representative of at least portions of a scene acquired by at least one imager that moves relative to a scene so as to acquire a plurality of optical images of the scene, each of at least two of said optical images being viewed from a different respective viewing position, the method comprising:

dividing each image into a plurality of segments and generating a plurality of mosaics of the scene, such that;

each mosaic contains segments taken from different ones of said optical images;

segments relating to at least one part of the scene are derived from at least two optical images and appear in at least two mosaics;

the different segments of the two optical images in a given mosaic represent different parts of the scene; and

in respect of any two of said optical images that both contribute at least two different segments to each of any two mosaics, the segments contributed by both optical images to a first one of the mosaics are all to the left of the segments contributed by both optical images to a second one of the mosaics; and

displaying a plurality of the mosaics so as to provide a sense of depth of the scene.

39. The method according to claim 38 wherein said displaying comprises displaying at least two of the mosaics one after another sufficiently rapidly to provide the sense of depth.

40. The method according to claim 38 wherein said displaying comprises displays the at least two mosaics so that each eye sees a different one of the mosaics.

41. The method according to claim 38 wherein said generating comprises adjusting disparity between mosaics of the plurality of mosaics to provide selected visual effects.

42. The method according to claim 41 wherein said adjusting disparity between mosaics of the plurality of mosaics comprises providing at least one selected disparity for objects in the scene.

43. The method according to claim 41 wherein said adjusting disparity between mosaics of the plurality of mosaics comprises providing a at least disparity for objects at a finite distance in the scene that is required for human stereo fusion.

44. The method according to claim 41 wherein said adjusting disparity between mosaics of the plurality of mosaics is performed so that disparity for all objects in the scene is within a predetermined range.

45. The method according to claim 41 wherein said adjusting disparity between mosaics of the plurality of mosaics comprises adjusting disparity for different objects in the scene differently.

46. The method according to claim 38, wherein:

said dividing comprises generating at least three series of image strips, each of said series comprising strips of a series of images of a scene as would be recorded by a camera from a respective series of positions relative to the scene, the image strips of the respective series representing strips of the respective images, said strips being displaced from one another by at least one selected displacement; and

said generating comprises mosaicking the respective series of image strips together thereby to construct said plurality of mosaics of the scene.

47. Method for processing image data representative of at least portions of a scene acquired by at least one imager that moves relative to a scene so as to acquire a plurality of optical images of the scene, each of at least two of said optical images being viewed from a different respective viewing position, the method comprising:

dividing each image into a plurality of segments and generating a plurality of mosaics of the scene, such that;

each mosaic contains segments taken from different ones of said optical images;

at least two different segments in each mosaic relate to a different part of the scene that appears in each of the two respective optical images;

segments relating to at least one part of the scene are derived from at least two optical images and appear in at least two mosaics; and

in respect of any two of said optical images that both contribute at least two different segments to each of any two mosaics, the segments contributed by both optical images to a first one of the mosaics are all to the left of the segments contributed by both optical images to a second one of the mosaics; and

displaying a plurality of the mosaics so as to provide a sense of depth of the scene.

48. The method according to claim 47 wherein said displaying comprises displaying at least two of the mosaics one after another sufficiently rapidly to provide the sense of depth.

49. The method according to claim 47 wherein said displaying comprises displays the at least two mosaics so that each eye sees a different one of the mosaics.

50. The method according to claim 47 wherein said generating comprises adjusting disparity between mosaics of the plurality of mosaics to provide selected visual effects.

51. The method according to claim 50 wherein said adjusting disparity between mosaics of the plurality of mosaics comprises providing at least one selected disparity for objects in the scene.

52. The method according to claim 50 wherein said adjusting disparity between mosaics of the plurality of mosaics comprises providing a at least disparity for objects at a finite distance in the scene that is required for human stereo fusion.

53. The method according to claim 50 wherein said adjusting disparity between mosaics of the plurality of mosaics is performed so that disparity for all objects in the scene is within a predetermined range.

54. The method according to claim 50 wherein said adjusting disparity between mosaics of the plurality of mosaics comprises adjusting disparity for different objects in the scene differently.

55. The method according to claim 47, wherein:

said dividing comprises generating at least three series of image strips, each of said series comprising strips of a series of images of a scene as would be recorded by a camera from a respective series of positions relative to the scene, the image strips of the respective series representing strips of the respective images, said strips being displaced from one another by at least one selected displacement; and

said generating comprises mosaicking the respective series of image strips together thereby to construct said plurality of mosaics of the scene.