Head-mounted display

• US 5,742,264 A
• Filed: 01/23/1996
• Issued: 04/21/1998
• Est. Priority Date: 01/24/1995
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Nishigori, Yoshihisa, Inagaki, Satoru
• Primary Examiner(s): Powell, Mark R.
• Assistant Examiner(s): Kovalick, Vincent E.
• Application Number:
  US08/589,959
• Time in Patent Office: 819 Days
• Field of Search: 345/8, 345/7, 345/9
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G09G 2300/023 Display panel composed of stacked panels
• CPC Class Codes: G09G 2320/0238 Improving the black level
• CPC Class Codes: G09G 2320/066 for control of contrast
• CPC Class Codes: G09G 2380/10 Automotive applications
• CPC Class Codes: G09G 2380/12 Avionics applications
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 3/3611 Control of matrices with row and column drivers
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers
• CPC Class Codes: H04N 5/66 Transforming electric information into light information scanning details H04N3/00
• CPC Class Codes: H04N 5/7491 of head mounted projectors

Abstract

A head-mounted display for displaying an image that matches a viewer'"'"'s head movement has a head tracker for detecting the viewer'"'"'s head movement, an eye tracker for detecting the viewer'"'"'s eye movement, and an adaptive filter for adaptively filtering the output of the head tracker in accordance with the output of the head tracker and the output of the eye tracker, and when tracker, and when the viewer is watching a particular object in the displayed image, the adaptive filter is set as a low-pass filter, and this and the output gain of the head tracker is lowered; in other cases, the adaptive filter is set as an all-pass filter, and this prevents minute shaking of the head from being reflected on the displayed image, while retaining image display response to head motion, and furthermore, this serves to keep the particular object from being shifted outside the field of view.

Claims

1. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a head information signal;

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal;

adaptive filtering means for filtering head jitter from said head information signal based on said head information signal and said line-of-sight information signal; and

image generating means for generating an image by using said filtered head information signal and said line-of-sight information signal.

31. A head-mounted display according to claim 1 wherein said adaptive filtering means includes means for filtering said head information signal based on a rate of change of the viewer'"'"'s head position and a rate of change the viewer'"'"'s line-of-sight direction.

2. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a first head information signal;

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal;

filter control means for calculating an amount of head movement and an amount of line-of-sight movement from a first head information signal output from said head detecting means and the line-of-sight information signal output from said line-of-sight detecting means, and for generating from said amount of head movement and said amount of line-of-sight movement a filter control signal for controlling a pass band;

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting the result of the control as a second head information; and

image generating means for generating an image by using the second head information signal output from said variable filter means and the line-of-sight information signal output from said line-of-sight detecting means.

3. A head-mounted display according to claim 2, wherein

said variable filter means includes

delay means consisting of at least n (n≧

1) delay circuits each for delaying the first head information signal, output from said head detecting means, by a unit time,

multiplying means consisting of n+1 multipliers, for which an n-tuple filter coefficient is determined by the filter control signal output from said filter control means, and which multiplies an input to said delay means and an output of each of said delay circuits by said filter coefficient,

adding means for adding together values output from said n+1 multipliers and thereby outputting a translated head information signal, and head signal combining means for combining the translated head information signal output from said adding means with the first head information signal output from said head detecting means, and thereby generating the second head information signal.

4. A head-mounted display according to claim 2 or 3, wherein

said filter control means

includes first calculating means for calculating an amount of head rotation from the first head information signal output from said head detecting means, and

supplies said variable filter means with the filter control signal for determining the filter coefficient such that when said line-of-sight direction coincides with the direction of head rotation, a low-pass characteristic is provided, with a cut-off frequency being made lower when the amount of head rotation calculated by said first calculating means is small than when it is large, and when said line-of-sight direction is different from the direction of head rotation, a low-pass characteristic is provided.

5. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a first head information signal;

filter control means for calculating at least a first derivative of the head position or head orientation with respect to unit time from the first head information signal output from said head detecting means, and for generating a filter control signal to provide a low-pass characteristic when said first derivative is smaller than a prescribed value and an all-pass characteristic when said first derivative is not smaller than the prescribed value;

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal; and

image generating means for generating an image by using the second head information signal output from said variable filter means.

6. A head-mounted display according to claim 5, wherein

said variable filter means includes

delay means consisting of at least n (n≧

1) delay circuits each for delaying the first head information signal, output from said head detecting means, by a unit time,

multiplying means consisting of n+l multipliers, for which an n+1 tuple filter coefficient is determined by the filter control signal output from said filter control means, and which multiplies an input to said delay means and an output of each of said delay circuits by said filter coefficient,

adding means for adding together values output from said n+1 multipliers and thereby outputting a translated head information signal, and head signal combining means for combining the translated head information signal output from said adding means with the first head information signal output from said head detecting means, and thereby generating the second head information signal.

7. A head-mounted display according to claim 5 or 6, wherein

said filter control means further calculates a second derivative of the head position or head orientation with respect to unit time, and

said filter control means includes

a first subtractor for subtracting an output signal of a first delay circuit, an element of said delay means, from an input signal thereof;

a second subtractor for subtracting an output signal of a second delay circuit, an element of said delay means, from an input signal thereof;

a third subtractor for subtracting an output of said second subtractor from an output of said first subtractor, and

filter coefficient control means for accepting the output of said first subtractor as the first derivative of the head position or head orientation with respect to unit time and an output of said third subtractor as the second derivative of the head position or head orientation with respect to unit time, and for generating a filter coefficient control signal,

and supplies said variable filter means with the filter control signal for determining the filter coefficient such that when said first derivative and said second derivative of the head position or head orientation are both equal to or greater than the prescribed value, a low-pass characteristic is provided, and when said first derivative and said second derivative of the head position or head orientation are both smaller than the prescribed value, an all-pass characteristic is provided.

8. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a first head information signal;

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal;

threshold output means for accepting at an input thereof the line-of-sight information signal output from said line-of-sight detecting means, and for outputting a small threshold when an amount of line-of-sight movement is equal to or greater than a prescribed value and a large threshold when said amount is smaller than the prescribed value;

second head information generating means for judging whether to proceed to generate an image or stop the image, by comparing the threshold output from said threshold output means with said first head information signal, and for outputting the result of the judgement as a second head information signal; and

image generating means for generating an image by using said second head information signal and the line-of-sight information signal output from said line-of-sight detecting means.

9. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a first head information signal;

threshold output means for accepting at an input thereof the output of said head detecting means, and for outputting a small threshold when at least a head moving speed or head rotation speed is equal to or greater than a prescribed value and a large threshold when said speed is smaller than the prescribed value;

second head information generating means for judging whether to proceed to generate an image or stop the image, by comparing the threshold output from said threshold output means with said first head information signal, and for outputting the result of the judgement as a second head information signal; and

image generating means for generating an image by using said second head information signal.

10. A head-mounted display according to claim 9, wherein

said threshold output means calculates from said head information signal a head moving speed or head rotation speed at unit time and a head moving acceleration or head rotation acceleration at unit time, and outputs said threshold.

11. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a first head information signal;

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal;

moving amount control means for accepting said first head information signal and said line-of-sight information signal, and for outputting a moving amount control signal;

moving amount translating means for translating said first head information signal in accordance with said moving amount control signal, and for outputting a second head information signal; and

image generating means for generating an image on the basis of said second head information signal.

13. A head-mounted display according to claim 11 or 12, wherein

said moving amount control means has the viewer'"'"'s stationary-state eyeball motion prestored therein, measures the eyeball motion from said line-of-sight information signal, calculates a difference between the thus measured eyeball motion and the stationary-state eyeball motion, and outputs a moving amount control signal that matches the difference.

14. A head-mounted display according to claim 11 or 12, wherein

said moving amount control means compares a line-of-sight rotation angle contained in said line-of-sight information signal with a head rotation angle contained in said first head information signal, and when rotation directions are opposite to each other, outputs a moving amount control signal for said moving amount translating means to suppress the head rotation angle contained in said first head information signal.

12. A head-mounted display for displaying an image on a display while controlling a viewing angle of the image and a moving speed of an entire image on the basis of a viewer'"'"'s head position, head orientation, and line-of-sight direction, comprising:

head detecting means for detecting the viewer'"'"'s head position and head orientation, and for outputting the result of the detection as a first head information signal;

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal;

moving amount control means for accepting said first head information signal and said line-of-sight information signal, and for outputting a moving amount control signal;

moving amount translating means for translating said first head information signal in accordance with said moving amount control signal, and for outputting a second head information signal;

error compensating means for accepting said first head information signal, said second head information signal, and said moving amount control signal, errors between said second head information signal and said first head information signal being accumulated as error signals, and for generating an error compensating signal based on accumulated values of said error signals; and

image generating means for generating an image by reference to said line-of-sight information signal such that, when the line of sight is not fixed and when the head is moving, the image is generated by adding said error compensating signal to said second head information signal, and when the line of sight is fixed or when the head is not moving, the image is generated without performing the addition but by using said second head information signal.

15. A see-through type head-mounted display for viewing outside light and a displayed image in overlaid fashion with a position of the displayed image being changed to match head movement, comprising:

video signal input means for inputting a video signal;

magnification control means for outputting a display magnification signal for controlling an image display magnification;

a head detection sensor for detecting a viewer'"'"'s head position and head orientation, and for outputting the same as a first head information signal;

image control means for controlling the image display magnification and image display position on the basis of said first head information signal and said display magnification signal;

display means for displaying an image output from said image control means;

a projection lens for projecting the image of said display means in a designated size; and

a mirror having prescribed transmissivity such that the image projected by said projection lens is reflected into the viewer'"'"'s field of view, while, at the same time, allowing the outside light to enter the viewer'"'"'s eyes.

16. A head-mounted display according to claim 15, wherein said display means includes

a liquid-crystal panel and

a backlight whose illuminating area varies in accordance with an output signal of said image control means.

17. A head-mounted display according to claim 15, further comprising

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal,

filter control means for calculating the viewer'"'"'s viewpoint position on said mirror from said first head information signal and said line-of-sight information signal, and for generating a filter control signal for performing a prescribed band pass control when the viewer is viewing the image projected on said mirror by said projection lens, and a filter control signal that provides an all-pass characteristic when the viewer is viewing a portion other than the image projected on said mirror by said projection lens, and

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal,

wherein said image control means controls the image display magnification and image display position on the basis of said second head information signal and said display magnification signal.

18. A head-mounted display according to claim 17, wherein said display means includes

a liquid-crystal panel and

a backlight whose illuminating area varies in accordance with an output signal of said image control means.

19. A see-through type head-mounted display in which a displayed image position is changed to match head movement, comprising:

video signal input means for inputting a video signal;

magnification control means for outputting a display magnification signal for controlling an image display magnification;

a head detection sensor for detecting a viewer'"'"'s head position and head orientation, and for outputting the same as a first head information signal;

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal;

image control means for calculating the image display magnification and image display position on the basis of said first head information signal and said display magnification signal, and for outputting the same as an image control signal;

display means for displaying the input image in accordance with said image control signal;

a projection lens for projecting the image of said display means in a designated size; and

a reflective/transmissive panel constructed from a transparent panel and a liquid-crystal panel attached back-to-back, said liquid-crystal panel being driven in accordance with said image control signal.

20. A head-mounted display according to claim 19, wherein said display means includes

a liquid-crystal panel and

a backlight whose illuminating area varies in accordance with an output signal of said image control means.

21. A head-mounted display according to claim 19, further comprising

filter control means for calculating the viewer'"'"'s viewpoint position on said reflective/transmissive panel from said first head information signal and said line-of-sight information signal, and for generating a filter control signal for performing a prescribed band pass control when the viewer is viewing the image projected on said reflective/transmissive panel by said projection lens, and a filter control signal that provides an all-pass characteristic when the viewer is viewing a portion other than the image projected on said reflective/transmissive panel by said projection lens, and

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal,

wherein said image control means controls the image display magnification and image display position on the basis of said second head information signal, instead of said first head information signal, and said display magnification signal.

22. A head-mounted display according to claim 21, wherein said display means includes

a liquid-crystal panel and

a backlight whose illuminating area varies in accordance with an output signal of said image control means.

23. A see-through type head-mounted display for viewing outside light and a displayed image in overlaid fashion with a position of the displayed image being changed to match head movement, comprising:

video signal input means for inputting a video signal;

magnification control means for outputting a display magnification signal for controlling an image display magnification;

display means for displaying the input video signal;

a head detection sensor for detecting a viewer'"'"'s head position and head orientation, and for outputting the same as a first head information signal;

a projection lens for projecting the image displayed on said display means by enlarging the image in accordance with said display magnification signal;

driving means for driving said display means and said projection lens, integrally constructed, in accordance with said first head information signal, and thereby controlling the direction of image projection; and

a mirror having prescribed transmissivity such that the image projected by said projection lens is reflected into the viewer'"'"'s field of view, while, at the same time, allowing the outside light to enter the viewer'"'"'s eyes.

24. A head-mounted display according to claim 23, further comprising

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal, filter control means for calculating the viewer'"'"'s viewpoint position on said mirror from said first head information signal and said line-of-sight information signal, and for generating a filter control signal for performing a prescribed band pass control when the viewer is viewing the image projected on said mirror by said projection lens, and a filter control signal that provides an all-pass characteristic when the viewer is viewing a portion other than the image projected on said mirror by said projection lens, and

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal,

wherein said driving means controls the image display magnification and image display position on the basis of said second head information signal.

25. A see-through type head-mounted display in which a displayed image position is changed to match head movement, comprising:

video signal input means for inputting a video signal;

magnification control means for outputting a display magnification signal for controlling an image display magnification;

display means for displaying the input video signal;

a head detection sensor for detecting a viewer'"'"'s head position and head orientation, and for outputting the same as a first head information signal;

a projection lens for projecting the image displayed on said display means by enlarging the image in accordance with said display magnification signal;

driving means for driving said display means and said projection lens as an integral unit in accordance with said first head information signal, and thereby controlling the direction of image projection; and

a reflective/transmissive panel constructed from a transparent panel and a liquid-crystal panel attached back-to-back, said liquid-crystal panel being driven in accordance with said image control signal.

26. A head-mounted display according to claim 25, further comprising

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal,

filter control means for calculating the viewer'"'"'s viewpoint position on said reflective/transmissive panel from said first head information signal and said line-of-sight information signal, and for generating a filter control signal for performing a prescribed band pass control when the viewer is viewing the image projected on said reflective/transmissive panel by said projection lens, and a filter control signal that provides an all-pass characteristic when the viewer is viewing a portion other than the image projected on said reflective/transmissive panel by said projection lens, and

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal,

wherein said driving means controls the image display magnification and image display position on the basis of said second head information signal.

27. A see-through type head-mounted display for viewing outside light and a displayed image in overlaid fashion with a position of the displayed image being changed to match head movement, comprising:

video signal input means for inputting a video signal;

a liquid-crystal panel for displaying an input image;

an eyepiece, interposed between said liquid-crystal panel and a viewer, for bringing the image of said liquid-crystal panel into focus within a clear vision range of the viewer;

a magnification adjusting lens positioned behind said liquid-crystal panel, as viewed from the viewer, and whose magnification is the reciprocal of that of the eyepiece;

a head detection sensor for detecting the viewer'"'"'s head position and head orientation, and for outputting the same as a first head information signal; and

driving means for moving said liquid-crystal panel, said eyepiece, and said magnification adjusting lens as an integral unit in vertical and horizontal directions in accordance with said first head information signal, and thereby controlling the same to a desired position.

28. A head-mounted display according to claim 27, further comprising

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal,

filter control means for calculating the viewer'"'"'s viewpoint position from said first head information signal and said line-of-sight information signal, and for generating a filter control signal for performing a prescribed band pass control when the viewer is viewing the image displayed on said liquid-crystal panel, and a filter control signal that provides an all-pass characteristic when the viewer is viewing a portion other than the image displayed on said liquid-crystal panel, and

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal, wherein said driving means controls the image display magnification and image display position on the basis of said second head information signal.

29. A see-through type head-mounted display in which a displayed image position is changed to match head movement, comprising:

video signal input means for inputting a video signal;

display means for displaying an input image signal;

an eyepiece, interposed between said display means and a viewer, for bringing the image of said display means into focus within a clear vision range of the viewer;

a head detection sensor for detecting the viewer'"'"'s head position and head orientation, and for outputting the same as a first head information signal; and

driving means for moving said display means and said eyepiece as an integral unit in vertical and horizontal directions in accordance with said first head information signal, and thereby controlling the same to a desired position.

30. A head-mounted display according to claim 29, further comprising

line-of-sight detecting means for detecting the viewer'"'"'s line-of-sight direction, and for outputting the result of the detection as a line-of-sight information signal, filter control means for calculating the viewer'"'"'s viewpoint position from said first head information signal and said line-of-sight information signal, and for generating a filter control signal for performing a prescribed band pass control when the viewer is viewing the image displayed on said display means, and a filter control signal that provides an all-pass characteristic when the viewer is viewing a portion other than the image displayed on said display means, and

variable filter means, whose filter characteristic is controlled by said filter control signal, for outputting a second head information signal,

wherein said driving means controls the image display magnification and image display position on the basis of said second head information signal.