Head mounted camera with eye monitor and stereo embodiments thereof

• US 6,307,589 B1
• Filed: 05/04/1998
• Issued: 10/23/2001
• Est. Priority Date: 01/07/1993
• Status: Expired due to Fees

• Current Assignee: Simulated Percepts, LLC
• Original Assignee: Francis J. Maquire Jr
• Inventors: Maquire, Francis J. Jr.
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Vu, Ngoc-Yen, Vu, Ngoc-Yen
• Application Number:
  US09/072,202
• Time in Patent Office: 1,268 Days
• Field of Search: 345/7, 345/961, 345/965, 345/967, 345/8, 345/9, 348/13, 348/15, 348/42-48, 348/49, 348/51, 348/53, 348/135, 348/140, 348/169, 348/222, 348/239, 348/373-376, 348/115, 348/333.01, 348/333.02, 358/530, 358/540, 396/51
• US Class Current: 348/333.03
• CPC Class Codes: G02B 30/34 Stereoscopes providing a stereoscopic pair of separated images corresponding to parallactically displaced views of the same object, e.g. 3D slide viewers
• CPC Class Codes: G09G 2310/0205 Simultaneous scanning of several lines in flat panels
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 3/20 for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/289 Switching between monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/322 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 13/398 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 : specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 : Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/55 : Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 7/002 : Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098
• CPC Class Codes: Y10S 715/961 : Operator interface with visual structure or function dictated by intended use
• CPC Class Codes: Y10S 715/965 : for process control and configuration
• CPC Class Codes: Y10S 715/967 : Visual or iconic programming

Abstract

A method and apparatus for providing, in response to image signals originating in an object space, mixed image signals for providing nonuniform resolution images for stimulating simulated active percepts for passive perception by a viewer in an image space. The images have a highly detailed component which has its image content changed according to changes in the direction of a simulated observer'"'"'s eye position in an object space. The images may be provided stereoscopically. The images may be provided at various apparent distances and the relationship between accommodation and convergence may be preserved. Audio waves for directionally simulating that which would be heard by the simulated observer may be provided.

Claims

1. A method, comprising the steps of,

monitoring a human eye in a human head, for providing an eye direction signal having a magnitude indicative of a direction of a visual axis of said human eye in said human head as said eye views objects in a space with said objects therein;

receiving light reflected from said objects with a lens of a video camera mounted on said human head, for providing an optical image; and

casting said optical image onto a surface sensitive thereto in said camera for converting said optical image to an electrical image signal for display according to said eye direction signal.

2. The method of claim 1, wherein said step of converting comprises the step of:

encoding said electrical image signal with image information for successive images, each image having nonuniform resolution comprising an area of greater resolution in a position in the image and within a surrounding area of lesser resolution together being simulative of retinal resolution, wherein the position of the area of greater resolution in the image changes between the successive images, said changes in the position of the area of greater resolution being indicative of changes in said direction of said visual axis.

3. The method of claim 1, wherein said lens is adjustable in response to said eye direction signal for said casting said optical image onto said surface with changing magnification corresponding to changes in said direction of said visual axis.

4. The method of claim 1, wherein said display is for a passive viewer.

5. The method of claim 1, wherein said human head is that of an active viewer and said human eye in said human head of said active viewer views objects in an object space with said objects therein, and wherein said display is of said objects in an image space for viewing by an eye of a passive viewer in a direction emulative of said direction of said visual axis of said human eye in said human head of said active viewer.

6. The method of claim 1, wherein said human head has various translational and rotational degrees of freedom, wherein said monitoring is for providing an eye direction signal as said head executes said translational and rotational movements in positioning and orienting said head in various corresponding positions and directions, and wherein said lens of said video camera is for forming said optical image in said various corresponding positions and directions.

7. The method of claim 1, wherein said step of monitoring comprises the step of monitoring a pair of human eyes in said human head, for providing a corresponding pair of eye direction signals indicative of directions of visual axes of said pair of human eyes in said human head as said pair of eyes view said objects, wherein said step of receiving light from said objects is carried out with a pair of lenses of a corresponding pair of video cameras mounted on said human head for providing a corresponding pair of optical images, and wherein said step of casting comprises the step of casting said pair of optical images onto a corresponding pair of surfaces sensitive thereto in said pair of cameras for converting said optical images to a corresponding pair of electrical image signals for display according to said pair of eye direction signals.

8. The method of claim 7, wherein said lenses are adjustable in response to said eye direction signals for said casting said optical images onto said surfaces with changing magnification corresponding to changes in said directions of said visual axes.

9. The method of claim 7, wherein said method further comprises the steps of:

determining from said pair of eye direction signals a distance from said eyes for providing a distance signal indicative thereof; and

encoding with said pair of electrical image signals said distance signal for controlling apparent distances of display images provided according to said image signals.

10. The method of claim 7, wherein said converting comprises encoding said electrical image signals with image information for successive images, each image having nonuniform resolution comprising an area of greater resolution in a position in the image and within a surrounding area of lesser resolution together being simulative of retinal resolution, wherein the position of the area of greater resolution in the image changes between the successive images, said changes in the position of the area of greater resolution being indicative of changes in said directions of said visual axes.

11. The method of claim 8, wherein said method further comprises the steps of:

determining from said pair of eye direction signals a distance from said eyes for providing a distance signal indicative thereof; and

encoding with said pair of electrical image signals said distance signal for controlling apparent distances of display images provided according to said image signals.

12. Apparatus, comprising:

means for monitoring a human eye in a human head, for providing an eye direction signal having a magnitude indicative of a direction of a visual axis of said human eye in said human head as said eye views objects in a space with said objects therein; and

a video camera having a lens for receiving light reflected from said objects with said video camera adjacent said human head, said lens for casting an optical image onto a surface sensitive thereto in said camera for converting the optical image to an electrical image signal for display according to said eye direction signal.

13. The apparatus of claim 12, further comprising means for encoding said electrical image signal with image information for successive images, each image having nonuniform resolution comprising an area of greater resolution in a position in the image and within a surrounding area of lesser resolution together being simulative of retinal resolution, wherein the position of the area of greater resolution in the image changes between the successive images, said changes in the position of the area of greater resolution being indicative of changes in said direction of said visual axis.

14. The apparatus of claim 12, wherein said display is for a passive viewer.

15. The apparatus of claim 12, wherein said human head is that of an active viewer, and said space is an object space with said objects therein, and wherein said display is of said objects in an image space for viewing by an eye of a passive viewer in a direction emulative of said direction of said visual axis of said human eye in said human head of said active viewer.

16. The apparatus of claim 12, wherein said human head has various translational and rotational degrees of freedom, wherein said monitoring is for providing an eye direction signal as said head executes translational and rotational movements in positioning and orienting said head in various corresponding positions and directions, and wherein said lens of said video camera is for forming said optical image in said various corresponding positions and directions.

17. The apparatus of claim 12, wherein said camera includes means for adjusting said lens in response to said eye direction signal for said casting said optical image onto said surface with changing magnification corresponding to changes in said direction of said visual axis.

18. The apparatus of claim 17, wherein said method further comprises the steps of:

means for determining from said pair of eye direction signals a distance from said eyes for providing a distance signal indicative thereof; and

means for encoding with said pair of electrical image signals said distance signal for controlling apparent distances of display images provided according to said image signals.

19. The apparatus of claim 12, wherein the means for monitoring a human eye in a human head comprises means for monitoring two eyes in said human head, for providing a pair of eye direction signals indicative of directions of visual axes of said eyes as said eyes view said objects in said space and wherein said video camera comprises a pair of video cameras having a corresponding pair of lenses for receiving light reflected from said objects with said video cameras adjacent said head, said lenses for casting optical images onto corresponding surfaces sensitive thereto in said camera for converting the optical images to corresponding electrical image signals for display according to said eye direction signals.

20. The apparatus of claim 19, wherein said cameras further comprise means for adjusting said lenses in response to said eye direction signals for said casting said optical images onto said surfaces with changing magnification corresponding to changes in said directions of said visual axes.

21. The apparatus of claim 19, wherein said apparatus further comprises a controller for determining from said pair of eye direction signals a distance from said eyes for providing a distance signal indicative thereof and for encoding with said pair of electrical image signals said distance signal for controlling apparent distances of display images provided according to said image signals.

22. The apparatus of claim 19, wherein said apparatus includes a controller for encoding said electrical image signals with image information for successive images, each image having nonuniform resolution comprising an area of greater resolution in a position in the image and within a surrounding area of lesser resolution together being simulative of retinal resolution, wherein the position of the area of greater resolution in the image changes between the successive images, said changes in the position of the area of greater resolution being indicative of changes in said directions of said visual axes.