Visual stabilization system for head-mounted displays

• US 9,489,044 B2
• Filed: 11/09/2015
• Issued: 11/08/2016
• Est. Priority Date: 11/07/2014
• Status: Expired due to Fees

• Current Assignee: Eye Labs, LLC
• Original Assignee: Eye Labs, LLC
• Inventors: Fateh, Sina
• Primary Examiner(s): Dang, Hung
• Assistant Examiner(s): Park, Sunghyoun
• Application Number:
  US14/936,520
• Publication Number:
  US 20160131908A1
• Time in Patent Office: 365 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0132 comprising binocular systems
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/147 : using display panels
• CPC Class Codes: G06T 13/20 : 3D [Three Dimensional] animation
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 : involving 3D image data
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/20172 : Image enhancement details
• CPC Class Codes: G06T 2210/36 : Level of detail
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G09G 2340/0407 : Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0485 : Centering horizontally or vertically
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 3/003 : to produce spatial visual effects
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays
• CPC Class Codes: H04N 2213/002 : Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices

Abstract

Introduced herein are various techniques for displaying virtual and augmented reality content via a head-mounted display (HMD). The techniques can be used to improve the effectiveness of the HMD, as well as the general experience and comfort of users of the HMD. A binocular HMD system may present visual stabilizers to each eye that allow users to more easily fuse the digital content seen by each eye. In some embodiments the visual stabilizers are positioned within the digital content so that they converge to a shared location when viewed by a user, while in other embodiments the visual stabilizers are mapped to different locations within the user'"'"'s field of view (e.g., peripheral areas) and are visually distinct from one another. These techniques allow the user to more easily fuse the digital content, thereby decreasing the eye fatigue and strain typically experienced when viewing virtual or augmented reality content.

Claims

1. A system for generating and implementing visual stabilizers that serve as visual cues for a user of a binocular head-mounted device and that allow digital content to be more easily visually fused by the user, thereby decreasing eye fatigue experienced by the user over time, the system comprising:

a processor communicatively coupled to an image display system of a head-mounted device and a memory, the processor operable to execute instructions stored in the memory; and

the memory, which includes specific instructions for generating visual stabilizers that are presented within digital content viewable by a user on a first optical display surface and a second optical display surface of the head-mounted device, wherein the specific instructions cause the processor to;

generate a first visual stabilizer that is a first digital representation of at least part of a geometric shape;

integrate the first visual stabilizer into a first digital image to form a first composite image, the first composite image to be shown to one of the user'"'"'s eyes;

cause the first composite image to be presented on the first optical display surface;

generate a second visual stabilizer that is a second digital representation of at least part of the geometric shape;

integrate the second visual stabilizer into a second digital image to form a second composite image, the second composite image to be shown to the user'"'"'s other eye; and

cause the second composite image to be presented on the second optical display surface,

where the first and second visual stabilizers are positioned within the first and second composite images, respectively, so that the first and second visual stabilizers substantially overlay one another when simultaneously viewed by the user, thereby decreasing an effort required to visually fuse the first and second composite images together.

2. The system of claim 1, wherein the specific instructions further cause the processor to:

determine whether the first and second composite images are projected on the first and second optical displays, respectively, such that the first and second visual stabilizers substantially overlay one another when the first and second composite images are viewed simultaneously by the user; and

upon determining that the first and second visual stabilizer do not substantially overlay one another,

make a modification to the projection of the first composite image, the second composite image, or both.

3. The system of claim 2, wherein the modification includes a shift left, right, up, down, or some combination thereof.

4. The system of claim 1, wherein the image display system includes a first image display system configured to project the first composite image toward the first optical display surface and a second image display system configured to project the second composite image toward the second optical display surface, and wherein the processor and memory are housed within the head-mounted device.

5. The system of claim 1, wherein the processor and the memory are communicatively coupled to the head-mounted device via a wireless connection across a network.

6. The system of claim 1, wherein the first and second visual stabilizers are integrated into a user interface presented within the first and second digital images, and wherein the first and second visual stabilizers are indistinguishable from other elements of the user interface by the user.

7. The system of claim 1, wherein the first and second visual stabilizers converge to a shared location when the first and second composite images are simultaneously viewed and visually fused together by the user.

8. The system of claim 1, wherein the first and second visual stabilizers are located in different locations and are visually distinct from one another when the first and second composite images are simultaneously viewed by the user.

9. The system of claim 8, wherein the first and second visual stabilizers form a complete representation of the geometric shape when the first and second composite images are simultaneously viewed by the user, and wherein the complete representation of the geometric shape is a frame that surrounds the first and second composite images.

10. The system of claim 1, wherein the first visual stabilizer is one of a first plurality of visual stabilizers generated by the processor and integrated into the first digital image and the second visual stabilizer is one of a second plurality of visual stabilizers generated by the processor and integrated into the second digital image.

11. The system of claim 10, wherein the first and second pluralities of visual stabilizers increase in count as an overlap portion of digital content that is shown within each of the first and second digital images decreases.

12. The system of claim 10, wherein at least one of the first plurality of visual stabilizers and at least one of the second plurality of visual stabilizers converge to a shared location, and wherein at least one of the first plurality of visual stabilizers and at least one of the second plurality of visual stabilizers are located in different locations when the first and second composite images are simultaneously viewed by the user.

13. A method for generating and implementing visual stabilizers that serve as visual cues for a user of a head-mounted device and that allow digital content seen by each of the user'"'"'s eyes to be more easily visually fused by the user, the method comprising:

identifying a visual stabilizer from a plurality of visual stabilizers based on a characteristic of a user of a head-mounted device or of digital content to be viewed on the head-mounted device;

generating a first composite image that includes a first instance of the visual stabilizer and a first digital image;

generating a second composite image that includes a second instance of the visual stabilizer and a second digital image; and

simultaneously presenting the first composite image and the second composite image to the user on the head-mounted device,

where the first and second instances of the visual stabilizer are positioned within the first and second composite images so that the first and second instances of the visual stabilizer converge to a shared location when the first and second composite images are viewed by the user, thereby decreasing a visual effort required to visually fuse the first and second composite images.

14. The method of claim 13, wherein the characteristic is selected from:

size of an overlap portion of the digital content presented to the user;

whether the digital content includes augmented reality content or virtual reality content;

a type of digital content;

operating parameters of an image display system or of first and second optical display surfaces of the head-mounted device;

an age of the user;

a visual impairment experienced by the user;

a time of day;

an amount of time the user has used the head-mounted device;

or

an ambient light level of a local environment.

15. The method of claim 13, further comprising:

upon receiving an input from the user or upon exceeding a predetermined period of use, modifying the first and second instances of the visual stabilizer by increasing brightness, size, contrast, or resolution of the first and second instances.

16. The method of claim 13, further comprising:

upon receiving an input from the user or upon exceeding a predetermined period of use,

adding another instance of the visual stabilizer to each of the first and second composite images, or

adding an instance of another visual stabilizer to each of the first and second composite images.

17. The method of claim 13, wherein the first and second instances of the visual stabilizer are created using a software development kit and are integrated into the first and second digital images during creation of the digital content.

18. The method of claim 13, further comprising:

modifying a characteristic of the first and second instances of the visual stabilizer after the head-mounted device has been used by the user for a certain period of time,

wherein the characteristic is selected from size, shape, position, color, brightness, and contrast of the first and second instances of the visual stabilizer.