System and method for 3-dimensional display of image data

• US 8,130,260 B2
• Filed: 08/26/2008
• Issued: 03/06/2012
• Est. Priority Date: 11/09/2005
• Status: Active Grant

• Current Assignee: Johns Hopkins University
• Original Assignee: Johns Hopkins University
• Inventors: Krill, Jerry A., Beser, Nicholas D., Blodgett, David W., Roth, Michael W.
• Primary Examiner(s): Rao, Andy
• Application Number:
  US11/463,415
• Publication Number:
  US 20090256904A1
• Time in Patent Office: 1,288 Days
• Field of Search: 348 42- 60
• US Class Current: 348/53
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 30/34 Stereoscopes providing a stereoscopic pair of separated images corresponding to parallactically displaced views of the same object, e.g. 3D slide viewers
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

Disclosed is a system for a 3-dimensional display that includes at least one eyepiece having first and second eyepiece sockets, having a first display contained in said first eyepiece socket associated with a first eye, two eye cameras mounted such that a first eye camera tracks eye reference points of said first eye and a second eye camera tracks reference points of a second eye, a processor for providing overall control of said at least one eyepiece; a second display; and a main processor; signals are transmitted between said main processor and said at least one eyepiece, wherein said two eye cameras track said reference points, said eyepiece transceiver transmits said reference points to said main processor via said main transceiver, said main processor generates image data based on said reference points and transmits said image data to said displays to produce a 3-dimensional image.

Claims

1. A system for a 3-dimensional display, comprising:

at least one eyepiece, comprising;

a frame including first and second eyepiece sockets;

a first display disposed in said first eyepiece socket associated with a first eye for displaying first image data;

a clear lens disposed in said second eyepiece socket associated with a second eye;

two eye cameras mounted to the frame and arranged such that

a first eye camera receives a reflection of said first eye from said first display and tracks eye reference points of said first eye reflection and

a second eye camera receives a reflection of said second eye from said clear lens and tracks eye reference points of said second eye reflection;

a processor for providing overall control of said at least one eyepiece; and

an eyepiece transceiver for transmitting and receiving signals;

a second display, at a location separated from said at least one eyepiece, for displaying second image data at the separated location;

a main processor; and

a main transceiver connected to said main processor for transmitting signals between said main processor and said at least one eyepiece,

wherein said two eye cameras track said reference points of said reflections of said first and second eyes, said eyepiece transceiver transmits said reflected reference points to said main processor via said main transceiver, said main processor calculates a focal point based on said reflected eye reference points and generates said first and second image data based on said focal point, said first and second image data transmitted to and output from said first display in said one eyepiece and said second display at said separated location, and

wherein said first and second image data output from said first and second displays produces a 3-dimensional image.

2. The system for a 3-dimensional display of claim 1, wherein said second display is viewed by said second eye through said second eyepiece socket.

3. The system for a 3-dimensional display of claim 1, wherein said first and second image data is generated by said main processor to produce on said at least one display a fine focus area of fine resolution and a course focus area of course resolution.

4. The system for a 3-dimensional display of claim 3, wherein said fine focus area is located about said calculated focal point, and said course focus area is located about said fine focus area.

5. The system for a 3-dimensional display of claim 1, wherein said reflected eye reference points are utilized in pupil tracking calculations.

6. The system for a 3-dimensional display of claim 1, wherein said reflected eye reference points are utilized in eye elongation tracking calculations.

7. A method for 3-dimensional displaying of image data,

comprising the steps of:

tracking eye reference points of a first eye;

tracking eye reference points of a second eye;

calculating a focal point based on the eye reference points;

generating 3-dimensional image data from base data based on said focal point; and

displaying said 3-dimensional image data on at least one display;

wherein the step of generating 3-dimensional image data based on said focal point comprises generating first and second image data for display on a first and a second display, such that when said first and second displays are viewed simultaneously said first and second image data produces a 3-dimensional image, and

wherein said first display is contained in a first eyepiece socket of an eyepiece and said second display is at a location separated from said eyepiece and viewed through a second eyepiece socket of said eyepiece.

8. The method for 3-dimensional displaying of image data of claim 7, wherein said 3-dimensional image data is generated to produce a fine focus area of fine resolution and at least one course focus area of course resolution.

9. The method for 3-dimensional displaying of image data of claim 8, wherein said fine focus area is located about said focal point, and said at least one course focus area is located about said fine focus area.

10. The method for 3-dimensional displaying of image data of claim 7, further comprising the steps of:

(a) dividing said base data into at least two pieces such that each piece can be rendered as a primitive;

(b) generating a representation of said at least two pieces;

(c) representing said at least two pieces as a data node on a tree;

(d) dividing each of said at least two pieces into at least two sub-pieces; and

(e) repeating steps (a) through (d) until smallest primitives are reached.

11. The method for 3-dimensional displaying of image data of claim 10, wherein the 3-dimensional image data is generated by traversing the tree, and wherein when the traversal reaches the size of a pixel the rendering stops.

12. The method for 3-dimensional displaying of image data of claim 11, wherein base data outside of a viewing frustum is discarded at high speed as the tree is traversed.

20. The method for 3-dimensional displaying of image data of claim 7, wherein the steps of tracking eye reference points of a first eye and tracking eye reference points of a second eye include tracking reflected eye reference points of said first eye and tracking reflected eye reference points of said second eye.

13. A system for a 3-dimensional display, comprising:

at least one eye camera for tracking eye reference points of a first eye and a second eye;

a processor for calculating a focal point based on said eye reference points and for generating 3-dimensional image data based on said focal point; and

at least one display for displaying said 3-dimensional image data,

wherein said processor generates first and second image data for display on a first and a second display, such that when first and second displays are viewed simultaneously said first and second image data produces a 3-dimensional image,

wherein said first display is contained in a first eyepiece socket of an eyepiece and said second display is viewed through a second eyepiece socket of said eyepiece, and

wherein the second display is at a location separated from said eyepiece.

14. The system for a 3-dimensional display of claim 13 wherein said image data is generated by said processor to produce on said at least one display a fine focus area of fine resolution and a course focus area of course resolution.

15. The system for a 3-dimensional display of claim 13 wherein said at least one course focus area decrease in resolution as a distance from said focal point increases.

16. The system for a 3-dimensional display of claim 14, wherein said fine focus area is located about said focal point, and said course focus area is located about said fine focus area.

17. The system for a 3-dimensional display of claim 13, wherein said processor (a) divides the base data into at least two pieces such that each piece can be rendered as a primitive, (b) generates a representation of said at least two pieces, (c) represents said at least two pieces as a data node on a tree, (d) divides each of said at least two pieces into at least two sub-pieces, and repeats steps (a) through (d) until smallest primitives are reached.

18. The system for 3-dimensional display of claim 17, wherein the 3-dimensional image data is generated by traversing the tree, and wherein when the traversal reaches the size of a pixel the rendering stops.

19. The system for 3-dimensional display of claim 18, wherein base data outside of a viewing frustum is discarded at high speed as the tree is traversed.

21. The system for a 3-dimensional display of claim 13, wherein said first and second eyepieces generate reflected images of said first and second eyes and corresponding eye reference points, respectively, and said processor calculates said focal point based on said reflected eye reference points.