System and method for 3-D projection and enhancements for interactivity

• US 9,946,076 B2
• Filed: 10/04/2011
• Issued: 04/17/2018
• Est. Priority Date: 10/04/2010
• Status: Active Grant

• Current Assignee: Dan Kikinis, Gerard Dirk Smits
• Original Assignee: Dan Kikinis, Gerard Dirk Smits
• Inventors: Smits, Gerard Dirk, Kikinis, Dan
• Primary Examiner(s): Patel, Kumar
• Assistant Examiner(s): WOO, KUO-KONG
• Application Number:
  US13/877,652
• Publication Number:
  US 20130300637A1
• Time in Patent Office: 2,387 Days
• Field of Search: 345 7, 345 8, 345419, 345156, 345619, 359630, 359464
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 30/26 of the autostereoscopic type
• CPC Class Codes: G02B 5/128 transparent spheres being embedded in matrix
• CPC Class Codes: G03B 35/18 by simultaneous viewing
• CPC Class Codes: H04N 13/302 for viewing without the aid of special glasses, i.e. using autostereoscopic displays
• CPC Class Codes: H04N 13/356 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388

Abstract

A system projects a user-viewable, computer-generated or -fed image, wherein a head-mounted projector is used to project an image onto a retro-reflective surface, so only the viewer can see the image. The projector is connected to a computer that contains software to create virtual 2-D and or 3-D images for viewing by the user. Further, one projector each is mounted on either side of the user'"'"'s head, and, by choosing for example a retro angle of less than about 10 degrees, each eye can only see the image of one of the projectors at a give distance up to 3 meters, in this example, from the retro-reflective screen. The retro angle used may be reduced with larger viewing distance desired. These projectors use lasers to avoid the need for focusing, and in some cases there projectors use instead of lasers highly collimated LED light sources to avoid the need for focusing.

Claims

1. A method for image projection, comprising:

providing a screen that includes a retro-reflective surface, wherein the retro-reflective surface is operative to reflect a light beam such that a cone angle of the diffused reflection of the light beam is at least ten times less than 180 degrees;

providing a strobe light beam that scans for the retro-reflective surface, wherein detection of the diffused reflection of the strobe light beam determines when a gaze of a user is directed to the retro-reflective surface, and wherein the strobe light beam is non-visible to the user;

employing at least one head-mounted projector that is mounted on a head of a user to project an image onto the retro-reflective surface when the user'"'"'s gaze is directed to the retro-reflective surface in such that only the user is able to see the image, and wherein the image is at least one of a computer-generated image or a computer-fed image, and wherein two or more fiducial markings that are located on the retro-reflective surface are used to stabilize and scale the projection of the image respective to the user'"'"'s gaze; and

when the user'"'"'s gaze is detected to be undirected to the retro-reflective surface, ceasing the projection of the image by the head-mounted projector and pausing one or more movies when it is included in the image and the user'"'"'s gaze is undirected to the retro-reflective surface.

2. The method as in claim 1, wherein the at least one head-mounted projector includes a first projector and a second projector, the first projector is mounted on one side of the user'"'"'s head, the second projector is mounted on an opposite side of the user'"'"'s head, and wherein employing the at least one head-mounted projector to project the image onto the retro-reflective surface includes employing the first projector projects a beam onto the retro-reflective surface such that the beam reflects with a first retro-reflective angle to project a first projector image, and employing the second projector to project another beam onto the retro-reflective surface such that said another beam reflects with a second retro-reflective angle to project a second projector image, and such that the first retro-reflective angle and the second retro-reflective angle are each sufficiently small that-each eye of the user can only see a corresponding one of the first projector image and the second projector image at a particular distance from the user to the retro-reflective surface.

3. The method as in claim 1, wherein employing the at least one head-mounted projector to project the image onto the retro-reflective surface includes projecting lasers onto the retro-reflective surface.

4. The method as in claim 1, wherein employing the at least one head-mounted projector to project the image onto the retro-reflective surface includes projecting highly collimated LED light sources.

5. The method as in claim 1, further comprising employing at least one of a camera or sensor that is mounted near the at least one head-mounted projector to adjust the image.

6. The method as in claims 1, further comprising employing at least one camera that is mounted near the at least one head-mounted projector to observe an interaction of the user with the image.

7. The Method of claim 1, further comprising employing a plurality of edge fiducials on the retro-reflective surface to determine a location of the screen relative to the gaze of the user.

8. A system for image projection, comprising:

a computing device that is arranged to provide image information;

a strobe light beam that scans for a retro-reflective surface, wherein detection of the diffused reflection of the strobe light beam determines when a gaze of a user is directed to the retro-reflective surface on a screen, and wherein the strobe light beam is non-visible to the user; and

at least one head-mounted projector that is configured to, based at least in part on the image information, project an image onto a retro-reflective surface such that only the viewer can see the image when the user'"'"'s gaze is directed to the retro-reflective surface, wherein the retro-reflective surface is operative to reflect a light beam such that a cone angle of the diffused reflection of the light beam is at least ten times less than 180 degrees employing the at least one head-mounted projector that is mounted on a head of a user to project an image onto the retro-reflective surface when the user'"'"'s gaze is directed to the retro-reflective surface in such that only the user is able to see the image, and wherein two or more fiducial markings that are located on the retro-reflective surface are used to stabilize and scale the projection of the image respective to the user'"'"'s gaze; and

when the user'"'"'s gaze is detected to be undirected to the retro-reflective surface, ceasing the projection of the image by the head-mounted projector and pausing one or more movies when it is included in the image and the user'"'"'s gaze is undirected to the retro-reflective surface.

9. The system as in claim 8, wherein the at least one head-mounted projector includes a first projector and a second projector, wherein the first projector is mounted on one side of the user'"'"'s head, the second projector is mounted on an opposite side of the user'"'"'s head, and wherein the at least one head-mounted projector is configured to project the image onto the retro-reflective surface such that the first projector projects a beam onto the retro-reflective surface such that the beam reflects with a first retro-reflective angle to project a first projector image, and the second projector projects another beam onto the retro-reflective surface such that said another beam reflects with a second retro-reflective angle to project a second projector image, and such that the first retro-reflective angle and the second retro-reflective angle are each less than about 10 degrees, whereby each eye of the user can only see a corresponding one of the first projector image or the second projector image at a distance less than or equal to about 1 meter from the user to the retro-reflective screen.

10. The system as in claim 9, wherein the at least one head-mounted projector is arranged such that at least one head-mounted projector can be adjusted to reduce the first and second retro-reflective angles.

11. The system of claim 8, wherein the at least one head-mounted-projector is arranged to project the image by projecting lasers.

12. The system of claim 8, wherein the at least one head-mounted projector is arranged to project the image by projecting highly collimated LED light sources.

13. The system of claim 8, wherein the at least one head-mounted-projector is connected to the computing device, the computing device includes a central processing unit (CPU), the computing device contains a software instance running on the CPU, and wherein the software instance is configured to create virtual 2-D and or 3-D images for viewing by the user.

14. The system as in claim 8, further comprising a sensor, wherein the computing device is configured to control the at least one head-mounted projector such that, duration operation, the at least one head-mounted projector projects the image by projecting a beam, and wherein the computing device is further configured to control the at least one head-mounted projected such that, during operation, an additional beam is added to the beam in an invisible wavelength, wherein the sensor is configured to sense whether a retro-reflection of fiduciary markings in color not visible to the human eye, but contrasting in said invisible wavelengths, are present on the retro-reflective surface, and further configured to read the fiduciary markings if the fiduciary markings are present on the retro-reflective surface.

15. The system as in claim 14, wherein the sensor is further configured to detect interjecting objects between the at least one head-mounted projector and the retro-reflective surface, the computing device is configured to, in response to the detection of interjecting objects, control the at least one head-mounted projector to create a disruption of said another beam, and wherein the computing device is further configured to, in response to the detection of the interjecting objects, provide commands for actions.

16. The system as in claim 14, wherein the software instance is further configured to calculate a position and orientation of the user'"'"'s head relative to the retro-reflective surface.

17. The system of claim 15, wherein the commands for actions include at least one of navigation in a virtual environment, launching of programs, or manipulation of data.

18. The system of claim 8, further comprising a plurality of edge fiducials on the retro-reflective surface to determine a location of the screen relative to the gaze of the user.