BROAD VIEWING ANGLE DISPLAYS AND USER INTERFACES

• US 20110128555A1
• Filed: 07/09/2009
• Published: 06/02/2011
• Est. Priority Date: 07/10/2008
• Status: Active Grant

• Current Assignee: Real View Imaging Ltd.
• Original Assignee: Real View Imaging Ltd.
• Inventors: Kaufman, Aviad, Rotschild, Carmel
• Granted Patent: US 8,500,284 B2
• Time in Patent Office: Days
• US Class Current: 356/625
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 30/50 the image being built up from image elements distributed over a 3D volume, e.g. voxels
• CPC Class Codes: G02B 30/56 by projecting aerial or floating images
• CPC Class Codes: G03H 1/0005 Adaptation of holography to specific applications holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042
• CPC Class Codes: G03H 1/0248 Volume holograms
• CPC Class Codes: G03H 1/08 Synthesising holograms, i.e. holograms synthesized from objects or objects from hologramsusing electric digital computers G06F; G06T
• CPC Class Codes: G03H 1/0866 Digital holographic imaging, i.e. synthesizing holobjects from holograms
• CPC Class Codes: G03H 1/2205 using downstream optical component
• CPC Class Codes: G03H 1/2286 Particular reconstruction light G03H1/24 takes precedence; Beam properties
• CPC Class Codes: G03H 1/2294 Addressing the hologram to an active spatial light modulator
• CPC Class Codes: G03H 1/265 Angle multiplexing; Multichannel holograms G03H1/268 takes precedence
• CPC Class Codes: G03H 2001/0038 analogue or digital holobjects holographic interferometry G01B9/021; investigating particles G01N15/0227
• CPC Class Codes: G03H 2001/0055 in advertising or decorative art
• CPC Class Codes: G03H 2001/0061 in haptic applications when the observer interacts with the holobject
• CPC Class Codes: G03H 2001/0426 Extended parallax, e.g. panoramic or 360deg. holograms
• CPC Class Codes: G03H 2001/2207 Spatial filter, e.g. for suppressing higher diffraction orders
• CPC Class Codes: G03H 2001/221 Element having optical power, e.g. field lens
• CPC Class Codes: G03H 2001/2223 Particular relationship between light source, hologram and observer
• CPC Class Codes: G03H 2001/2231 Reflection reconstruction
• CPC Class Codes: G03H 2001/2236 Details of the viewing window
• CPC Class Codes: G03H 2001/2239 : Enlarging the viewing window
• CPC Class Codes: G03H 2001/2242 : Multiple viewing windows
• CPC Class Codes: G03H 2001/2271 : RGB holobject
• CPC Class Codes: G03H 2001/2284 : Superimposing the holobject with other visual information
• CPC Class Codes: G03H 2001/2292 : Using scanning means
• CPC Class Codes: G03H 2001/2297 : using frame sequential, e.g. for reducing speckle noise
• CPC Class Codes: G03H 2001/262 : not in optical contact G03H1/30 takes precedence
• CPC Class Codes: G03H 2001/2655 : Time multiplexing, i.e. consecutive records wherein the period between records is pertinent per se
• CPC Class Codes: G03H 2001/266 : Wavelength multiplexing
• CPC Class Codes: G03H 2001/303 : Interleaved sub-holograms, e.g. three RGB sub-holograms having interleaved pixels for reconstructing coloured holobject
• CPC Class Codes: G03H 2210/20 : 2D object
• CPC Class Codes: G03H 2210/30 : 3D object
• CPC Class Codes: G03H 2210/32 : 3D+2D, i.e. composition of 3D and 2D sub-objects, e.g. scene in front of planar background
• CPC Class Codes: G03H 2210/36 : Occluded features resolved due to parallax selectivity
• CPC Class Codes: G03H 2222/20 : Coherence of the light source
• CPC Class Codes: G03H 2222/22 : Spatial coherence
• CPC Class Codes: G03H 2222/33 : Pulsed light beam
• CPC Class Codes: G03H 2222/52 : Divergent beam
• CPC Class Codes: G03H 2223/24 : Reflector; Mirror
• CPC Class Codes: G03H 2225/60 : Multiple SLMs
• CPC Class Codes: G03H 2225/61 : for multicolour processing
• CPC Class Codes: G03H 2226/02 : Computing or processing means, e.g. digital signal processor [DSP]
• CPC Class Codes: G03H 2226/04 : Transmission or communication means, e.g. internet protocol
• CPC Class Codes: G03H 2226/05 : Means for tracking the observer
• CPC Class Codes: G03H 2227/02 : Handheld portable device, e.g. holographic camera, mobile holographic display
• CPC Class Codes: G03H 2227/03 : Means for moving one component G03H1/0476, G03H2001/2695 take precedence
• CPC Class Codes: G03H 2240/61 : SLM related parameters, e.g. pixel size
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 : Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: H04N 13/366 : using viewer tracking
• CPC Class Codes: H04N 13/368 : for two or more viewers
• CPC Class Codes: H04N 13/383 : for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 13/388 : Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume

Abstract

Disclosed are methods and systems for displaying images, and for implementing volumetric user interfaces. One exemplary embodiment provides a system comprising: a light source; an image producing unit, which produces an image upon interaction with light approaching the image producing unit from the light source; an eyepiece; and a mirror, directing light from the image to a surface of the eyepiece, wherein the surface has a shape of a solid of revolution formed by revolving a planar curve at least 180° around an axis of revolution. Another exemplary embodiment provides a method for implementing a floating-in-the-air user interface, including displaying a first image in a display space of a first floating-in-the-air display, inserting a real object into the display space of the first floating-in-the-air display, locating a location of the real object within the display space of the first floating-in-the-air display, locating the real object in the display space, and providing the location as input to the floating-in-the-air user interface.

Claims

1. A method of floating-in-the-air image display, comprising:

providing a floating-in-the-air display device at a location; and

projecting from said device one or more floating-in-the-air computer generated images, viewable over a range of angles including at least 200 arc degrees surrounding said location.

2. A method according to claim 1, wherein projecting comprises selectively projecting over a small angle where a viewer is expected.

3. A method according to claim 1, wherein projecting comprises selectively projecting using a plurality of image generation modules.

4. A method according to claim 1, wherein projecting comprises projecting different images with a same coordinate system, in different directions.

5. A method according to claim 1, wherein projecting comprises projecting a 2D image.

6. A method according to claim 1, wherein projecting comprises projecting image of an object such that the object'"'"'s presentation angle changes with a viewing angle thereof to match an effect of moving around said object.

7. A method according to claim 1, wherein projecting comprises projecting a 3D image.

8. A method according to claim 1, wherein projecting comprises projecting a hologram.

9. A method according to claim 1, wherein projecting comprises adjusting a projection distance of said image.

10. A method according to claim 1, wherein projecting comprises adjusting a focal distance of said image.

11. A method according to claim 1, wherein projecting comprises projecting different images for different eyes of a same viewer.

12. A method according to claim 1, wherein projecting comprises projecting from a single point in said device.

13. A method according to claim 1, wherein projecting comprises projecting images with a shared coordinate system.

14. A method according to claim 1, wherein projecting comprises imaging said images at a location not occupied by a display substrate.

15. A hologram display device projecting a floating in the air computer generated hologram simultaneously viewable over a range of viewing angles of at least 180 degrees.

16. A display device according to claim 15, wherein said hologram shares a same coordinate set from viewing points separated by at least 20 degrees.

17. A display device according to claim 15, comprising a hologram generating unit and at least one lens for projecting the hologram.

18. A display device according to claim 15, comprising at least one distance control unit.

19. A display device according to claim 15, comprising at least one hologram aiming mechanism.

20. A method of displaying content to multiple viewers, the method comprising:

forming multiple volumetric images, each of at least a portion of the content and each viewable from its own viewability space; and

overlapping a portion of one or more of said viewability spaces with a pupil of each of the viewers.

21. A method according to claim 20, wherein said viewability spaces can cover over 90 degrees.

22. A method according to claim 20, wherein the content is a single scene;

and each of said volumetric images is of a face of said single scene as viewable from a different viewing point.

23. A method according to claim 20, wherein the viewers are at different azimuthal angles around the space occupied by one of said volumetric images.

24. A method according to claim 23, wherein said different azimuthal angles span an entire circle.

25. A method according to claim 23, wherein said different azimuthal angles span at least half a circle.

26. A method according to any of claims 20 to 25, wherein two of said viewers are distanced from each other at least 1 meter.

27. A method according to any of claims 20 to 26, wherein the viewers see the images at a same time.

28. A method according to any of claims 20 to 27, wherein a viewability space overlaps with an eye of a viewer only for a sequence of short periods, and said short periods are timely spaced such that said viewer sees a continuous view.

29. A system for displaying content to multiple viewers, the system comprising

a unit for generating volumetric images each of at least a portion of the content and each viewable from its own viewability space;

and

an optical system, controlling a portion of one or more of said viewability spaces with a pupil of each of the viewers.

30. A system according to claim 29, wherein the optical system comprises an azimuth determining element that determines the azimuth of at least one of said viewability spaces in respect of the volumetric image viewable from said viewability space.

31. A system comprising:

an image producing unit, producing a paraxial image; and

an optical system defining a stage and imaging said paraxial image to said stage, such that the image on the stage is viewable from a viewability space,

wherein said optical system comprises an eyepiece and a mirror, the mirror being configured to direct light to the eyepiece in a plurality of different azimuthal angles, and wherein

each of said azimuthal angles determines a different location for said viewability space; and

for each of said azimuthal angles the location of the stage is the same.

32. A system according to claim 31, wherein for at least two different azimuthal angles, two different elevations are provided.

33. A method of imaging a paraxial image to be seen by a viewer having a pupil in a first location and looking at a second location, the method comprising:

generating the paraxial image;

imaging the paraxial image to the location at which the viewer looks, such that an image of the paraxial image is viewable from a viewability space having a broadest portion and narrower portions;

selecting, responsive to the location of the viewer'"'"'s pupil, a third location; and

imaging the broadest portion of the viewability space to the selected third location.

34. A method of displaying a scene to a viewer looking towards a given location, the method comprising:

estimating the position of the viewer'"'"'s eye;

estimating which scene-portions the viewer would have seen if the scene was at the given location in a given orientation; and

imaging to the given location a computer generated hologram of only a portion of the scene, said portion comprising said estimated scene-portions,

wherein imaging is such that the viewer can see the hologram.

35. A method according to claim 34, comprising tracking the position of the viewer'"'"'s eye, and imaging the computer generated hologram such that when the viewer moves he keeps seeing the hologram at said given location.

36. A system for imaging a computer-generated hologram (CGH), the system comprising:

a hologram generating unit comprising a spatial light modulator (SLM);

an optical system configured to image a hologram generated by said unit to a first location and an image of the SLM to a second location; and

a controller configured to control the imaging of said image of the SLM to the second location, such that the CGH is imaged to a location between the optical system and the image of the SLM.

37. A system according to claim 36, wherein said controller is configured to control said optical system to generate said image of the SLM at said desired location at a desired orientation.

38. A system according to claim 36 or claim 37, wherein the controller is configured to change the desired location and/or orientation online.

39. A system according to claim 38, comprising an input for online receipt of indications of said desired location and/or orientation.

40. A system according to claim 39, wherein said input comprises a receiver for receiving, from an eye-tracking unit, a signal indicative of the location of a viewer'"'"'s eye, and the controller controls said optical system to project said image of the SLM to said desired location such that the viewer'"'"'s eye is within a viewability space comprising said desired location.

41. A system according to any of claims 36 to 40, wherein said optical system comprises:

an objective;

an eyepiece, and an optical-length adjusting unit, controllable to adjust the optical length between the objective and the eyepiece responsive to a distance between said desired location and one or more of the eyepiece and the objective.

42. A system according to claim 41, wherein said optical-length adjusting unit is configured to adjust said optical length online.

43. A system according to any of claims 36 to 42, comprising a mirror that reflects light reaching the mirror from the objective to a portion of the eyepiece, wherein the mirror is controllable to reflect said light to different portions of the eyepiece.

44. A system according to any of claims 36 to 43 wherein said eyepiece comprises a hollow body.

45. A system according to claim 44, wherein said hollow body is shaped as a solid of revolution obtainable by revolving a parabola around an axis that is not on the same plane as the parabola.

46. A system according to claim 44, wherein said hollow cylindrical body is a portion of a sphere.

47. A system according to any of claims 44 to 46, wherein the hologram generating unit is inside said eyepiece.

48. A system according to any of claims 44 to claim 47, comprising a revolving mirror, revolving around a central axis of said eyepiece.

49. A system according to claim 48, wherein light coming to said revolving mirror from the objective is reflected towards the eyepiece.

50. A system according to claim 49, wherein said light reaches the revolving mirror from the objective through one or more optical elements.

51. A system according to claim 50, wherein said one or more optical elements comprises an optical-length adjusting element, controllable to adjust the optical length between the objective and the eyepiece.

67. A system according to claim 36, wherein the controller controls the optical system.

68. A system according to claim 36 or claim 67, wherein the controller controls the hologram generating unit.

52. A system for imaging, the system comprising:

an eyepiece having an inner reflecting wall covering at least 90 degrees arc angle; and

an image generating unit, residing inside said cavity,

wherein said eye piece is configured to project an image from said image generation unit to one or more viewers.

53. A system according to claim 52, wherein said image generation unit includes at least one element which moves so as to project said image over a range of angles.

54. A system according to claim 52, wherein said system is configured so that said viewer is surrounded by said wall.

55. A system according to claim 52, wherein said eye piece is hollow, defining a cavity and wherein image generating unit comprises a hologram generating unit, residing inside said cavity;

and comprising

an objective on an optical path going from a hologram generated by said hologram generating unit to said inner wall.

56. A system according to claim 55, comprising a plurality of hologram generating units, each associated with an objective.

57. A system according to claim 56, configured to create, using said eyepiece, images of holograms produced by different ones of the computer generating units at a single location.

58. A system according to claim 57, wherein said single location is inside said eyepiece.

59. A system according to any of claims 55 to 58, wherein said hollow eyepiece has an inner reflective surface.

60. A system according to claim 59, wherein the shape of said inner surface is obtainable by revolving a curve residing on a first plane around an axis of revolution residing on a second plane, other than said first plane.

61. A system according to claim 60, wherein said axis of revolution is perpendicular to said first plane.

62. A system according to any of claims 55 to 58, wherein said inner surface is shaped as a portion of a spherical shell.

63. A system according to any of claims 55 to 61, wherein each of the plurality of hologram generating units comprises a spatial light modulator (SLM), and each of the plurality of unit cells comprises a light-converging objective placed with a focal point between the SLM and the hologram produced by said SLM.

64. A system according to any of claims 55 to 63, wherein each of the plurality of unit cells has an objective, and an optical-length determining element configured to determine the optical length between said objective and the eyepiece.

65. A system according to claim 64, wherein each of said optical length determining elements is controllable independently of the other optical length determining elements.

66. A system according to any of claims 55 to 65 comprising a revolving mirror at the center of said eyepiece, configured to receive light from the plurality of unit cells and reflect said light onto the eyepiece.

69. A method for implementing a floating-in-the-air user interface, comprising:

displaying a first image in a display space of a first floating-in-the-air display;

inserting a real object into the display space of the first floating-in-the-air display;

locating a location of the real object within the display space of the first floating-in-the-air display;

locating the real object in the display space; and

providing the location as input to the floating-in-the-air user interface.

70. The method of claim 69 and further comprising displaying a second image in the display space of the first floating-in-the-air display, based, at least in part, on the location.

71. The method of claim 69 in which the floating-in-the-air display is a volumetric display.

72. The method of claim 70 in which the second image is displayed at near real-time after the inserting the real object into the display space.

73. The method of claim 72 in which the time is less than 1/24 of a second.

74. The method of claim 70 wherein the first image is a blank image, and the second image includes a display of the location.

75. The method of claim 69, wherein the real object is a finger.

76. The method of claim 69, and further comprising:

displaying an actuator in the first image;

moving the location of the real object substantially close to the actuator; and

interpreting the location input as the real object actuating the actuator.

77. The method of claim 69, and further comprising:

moving the location of the real object;

tracking the location of the real object over time;

interpreting the location input as the real object manipulating at least a part of the first image.

78. The method of claim 77, and further comprising sending control commands to a robotic device based, at least in part, on the interpreting.

79. The method of claim 69, wherein the real object further comprises a plurality of real objects, and using the locations of each of the real objects as location inputs for the volumetric user interface.

80. The method of claim 69, wherein the second image is different from the first image.

81. The method of claim 69, wherein the second image is substantially equal to the first image plus an added indicator of the location input.

82. The method of claim 69, wherein the location comprises a location of what is substantially a point on the real object.

83. The method of claim 82, and further comprising capturing a sub-image based, at least in part, on the location.

84. The method of claim 83, wherein the sub-image comprises a voxel.

85. The method of claim 82, wherein the location further comprises a plurality of locations based, at least in part, on a plurality of locations of the point on the real object.

86. The method of claim 85, wherein a path connecting the plurality of locations is displayed by the first floating-in-the-air display.

87. The method of claim 85, wherein the plurality of locations comprises two locations, and further comprising defining a line in three dimensions based, at least in part, on the two locations.

88. The method of claim 85, wherein the plurality of locations comprises three locations not in a straight line, and further comprising defining a plane in three dimensions based, at least in part, on the three locations.

89. The method of claim 85, wherein the plurality of locations comprises four locations not in a one plane, and further comprising defining a volume in three dimensions based, at least in part, on the four locations.

90. The method of claim 85, and further comprising implementing, based, at least in part, on the plurality of locations, one function of the group of functions:

zooming in on the first image;

zooming out of the first image;

cropping the first image;

rotating the first image;

slicing the first image;

measuring a length within the first image;

measuring an area within the first image; and

measuring a volume within the first image.

91. The method of claim 85, further comprising implementing a sub-image capture based, at least in part, on the plurality of locations.

92. The method of claim 82, and further comprising marking the point so as to substantially contrast with a rest of the real object.

93. The method of claim 92, and further comprising the marking to be marking by a substantially compact light source.

94. The method of claim 69, wherein the location comprises a line defined by a long axis of the real object.

95. The method of claim 69, wherein the location comprises a frame corresponding to a shape of the real object.

96. The method of claim 69 and further comprising the first floating-in-the-air display displaying a second image at substantially the same time as the first floating-in-the-air display displays the first image, and wherein the first image is displayed to a first user, and the second image is displayed to a second user.

97. The method of claim 96, wherein the first image and the second image appear to be in a same location in space.

98. The method of claim 69 and further comprising:

a second floating-in-the-air display displaying a second image at substantially the same time as the first floating-in-the-air display displays the first image, and

wherein the first image is displayed to a first user, and the second image is displayed to a second user.

99. The method of claim 98 wherein the first floating-in-the-air display is substantially remote from the second floating-in-the-air display, and further comprising a communication channel between the first floating-in-the-air display and the second floating-in-the-air display.

100. The method of claim 96 wherein the first display and the second display are used to implement a remote medicine interaction between the first user and the second user.

101. The method of claim 96 wherein the first display and the second display are used to implement a whiteboard-like collaborative sharing between the first display and the second display.

102. The method of claim 96 wherein the first display and the second display are used to implement remote teaching by a user at the first floating-in-the-air display.

103. The method of claim 96 wherein the first display and the second display are used to implement a game in which the first user and the second user participate.

104. The method of claim 96 wherein the first display differs from the second display.

105. The method of claim 96 wherein the first display displays more content than the second display.

106. A method for enabling a viewing of dynamically generated a floating-in-the-air displayed object and a real object in a same display space, comprising:

displaying the volumetrically displayed object on a first floating-in-the-air display; and

inserting the real object into the display space of the first floating-in-the-air display.

107. The method of claim 106 wherein the floating-in-the-air display is a volumetric display.

108. The method of claim 106 wherein dynamically generated comprises computer generated.

109. The method of claim 106 and further comprising comparing the real object to at least a portion of the floating-in-the-air displayed object.

110. The method of claim 109 wherein the real object comprises a standard against which objects are measured, and the comparing enables a judgment of conformance to standard.

111. The method of claim 109 wherein the real object is a medical device for inserting into a body, and the at least a portion of the floating-in-the-air displayed object is at least a portion of the body, generated from a three-dimensional dataset.

112. The method of claim 109 wherein the comparing further comprises measuring a difference in size between the real object and the at least a portion of the floating-in-the-air displayed object.

113. The method of claim 112 wherein the measuring the difference in size comprises at least one of the group consisting of:

a difference in length;

a difference in planar area;

a difference in surface area; and

a difference in volume.

114. A method for enabling a viewing of a floating-in-the-air displayed body part from a three dimensional data set of the body and a volumetrically displayed virtual object from a three dimensional data set of one or more virtual objects, comprising:

displaying the floating-in-the-air displayed body part on a first floating-in-the-air display; and

overlaying the virtual object into the display space of the first floating-in-the-air display.

115. The method of claim 114 wherein the virtual object and the floating-in-the-air displayed body part are moved relative to each other in the display space of the first floating-in-the-air display.

116. The method of claim 114 and further comprising comparing the virtual object to at least a portion of the body part.

117. A user interface comprising:

a first floating-in-the-air display; and

a first input unit adapted to accept input from a first location within a first display space, the first display space being a volume within which objects are displayed by the first floating-in-the-air display.

118. The user interface of claim 117, wherein the floating-in-the-air display is a volumetric display.

119. The user interface of claim 117, wherein the floating-in-the-air display is a two-dimensional floating-in-the-air display.

120. The user interface of claim 117, wherein the first floating-in-the-air display is adapted to display the first location.

121. The user interface of claim 120, further comprising a second floating-in-the-air display, wherein the second floating-in-the-air display displays an identical display as the first floating-in-the-air display, including displaying the first location.

122. The user interface of claim 121, and further comprising a second input unit adapted to accept input from a second location within a second display space, the second display space being a volume within which objects displayed by the second floating-in-the-air display appear, and wherein the first floating-in-the-air display is adapted to display an identical display as the second floating-in-the-air display, including displaying the second location.

123. The user interface of claim 122, wherein the input from the first location and the input from the second location are both displayed.

124. The user interface of claim 122, wherein the first floating-in-the-air display is located in a different room than the second floating-in-the-air display.

125. The user interface of claim 122, wherein the first floating-in-the-air display is at least 100 meters away from the second floating-in-the-air display.

126. The user interface of claim 117, wherein the first floating-in-the-air volumetric display is adapted to provide sensory feedback based, at least in part, on the location and on what is being displayed in the location.

127. The user interface of claim 117, wherein the first floating-in-the-air volumetric display is adapted to display a hologram.

128. A method for implementing a floating-in-the-air user interface, comprising:

displaying a first image in a display space of a first floating-in-the-air display;

inserting a real object into the display space;

detecting location of the real object within the display space;

using the location as an input for the user interface; and

highlighting the location in the display space.

129. A user interface comprising:

means for displaying a floating-in-the-air display;

means for accepting an input from a location within the display space, the display space being a volume within which objects displayed by the floating-in-the-air display appear.