DISPLAY UPDATE TIME REDUCTION FOR A NEAR-EYE DISPLAY

• US 20140184475A1
• Filed: 12/27/2012
• Published: 07/03/2014
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Tantos, Andras, Fleck, Rod G., Perry, Jedd, Bohn, David D.
• Granted Patent: US 10,514,541 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: H03M 13/03 Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
• CPC Class Codes: H03M 13/356 Unequal error protection [UEP]
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/67 involving unequal error protection [UEP], i.e. providing protection according to the importance of the data
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002

Abstract

Technology is described for reducing display update time for a near-eye display (NED) device. A point of focus in the NED field of view is identified, often based on natural user input data. A communication module of a computer system communicatively coupled to the NED device transmits lossless priority data, an example of which is user focal region image data, using one or more communication techniques for satisfying lossless transmission criteria. Allowed loss image data is identified based at least in part on its distance vector from a point of focus in the display field of view. An example of allowed loss image data is image data to be displayed outside the user focal region. The allowed loss image data is transmitted and extracted from received image data allowing for lossy transmission.

Claims

1. A method for reducing display update time for a near-eye display (NED) device comprising:

identifying a distance vector in a display field of view of the near-eye display device from a point of focus to image data for display in the display field of view;

based on the distance vector from the point of focus and low perception criteria, a computer system communicatively coupled to the NED device determining whether at least some of the image data to be displayed qualifies as lossless priority data which does not satisfy low perception criteria;

based on the distance vector from the point of focus, the computer system communicatively coupled to the NED device determining whether at least some of the image data to be displayed qualifies as allowed loss data which satisfies the low perception criteria;

responsive to at least some of the image data qualifying as allowed loss data, transmitting from the computer system to the NED device the allowed loss data using one or more communication techniques allowing for lossy transmission; and

responsive to at least some of the image data qualifying as lossless priority data, transmitting to the NED device the lossless priority image data by the computer system using one or more communication techniques satisfying lossless transmission criteria.

2. The method of claim 1 wherein transmitting to the NED device the lossless priority image data by the computer system using one or more communication techniques satisfying lossless transmission criteria further comprises

encoding the lossless priority image data with one or more error correction techniques for satisfying lossless transmission criteria;

and

wherein transmitting from the computer system to the NED device the allowed loss data using one or more communication techniques allowing for lossy transmission further comprises

encoding at least some of the allowed loss image data by setting one or more data integrity header bits of packets and omitting redundant data for the at least some of the allowed loss image data with a level of error correction allowing for lossy transmission.

3. The method of claim 2 wherein the one or more communication techniques for satisfying lossless transmission criteria includes forward error correction (FEC).

4. The method of claim 1 wherein the lossless transmission criteria comprises a first bit error rate satisfying a probability criteria of error-free transmission of the lossless priority image data and allowing for lossy transmission comprises allowing for a second bit error rate which does not satisfy the probability criteria.

5. The method of claim 1 wherein transmitting from the computer system to the NED device the allowed loss data using one or more communication techniques allowing for lossy transmission further comprises

transmitting the allowed loss image data over a communication channel not satisfying a noise criteria of the lossless transmission criteria.

6. The method of claim 1 wherein transmitting to the NED device

the lossless priority image data by the computer system using one or more communication techniques satisfying lossless transmission criteria further comprises

modulating the lossless priority image data with a modulation technique satisfying a bit error rate criteria of the lossless transmission criteria;

and

wherein transmitting from the computer system to the NED device the allowed loss data using one or more communication techniques allowing for lossy transmission further comprises

modulating the at least the portion of the allowed loss image data with a modulation technique not satisfying the bit error rate criteria of the lossless transmission criteria.

7. The method of claim 6 wherein modulating the lossless priority image data with the modulation technique satisfying the bit error rate criteria of the lossless transmission criteria further comprises

modulating the lossless priority image data with a first constellation encoding scheme of bits per symbol satisfying the bit error rate;

and

wherein modulating the at least the portion of the allowed loss image data with the modulation technique not satisfying the bit error rate criteria of the lossless transmission criteria further comprises

modulating at least the portion of the allowed loss image data with a second constellation encoding scheme having more bits per symbol than the first constellation encoding scheme and which does not satisfy the bit error rate.

8. The method of claim 1 further comprising

the lossless priority image data comprises user focal region image data for display in a user focal region, and

the allowed loss image data comprises at least some secondary image data for display outside the user focal region;

and

the secondary image data has a lower image resolution than the user focal region image data.

9. The method of claim 8 further comprising

the secondary image data has a different display update rate than the user focal region image data.

10. A method for reducing display update time for a near-eye display (NED) device comprising:

receiving image data from a communicatively coupled computer system;

identifying user focal region image data for display in a user focal region in a display field of view of the NED device within the image data being received;

retrieving the user focal region image data using one or more communication techniques for satisfying lossless transmission criteria;

identifying at least some secondary image data for display outside the user focal region within the image data being received;

retrieving the at least some secondary image data using one or more communication techniques allowing for lossy transmission; and

displaying the user focal region image data and the at least some secondary image data in the display field of view of the NED device.

11. The method of claim 10 wherein the at least some secondary image data has a lower image resolution than an image resolution of the user focal region image data.

12. The method of claim 10 further comprising:

identifying natural user input in image data captured by one or more image capture devices of the NED device; and

determining the user focal region in the display field of view based on the natural user input.

13. The method of claim 12 wherein identifying natural user input in the image data captured by one or more image capture devices of the NED device further comprises capturing eye movements with an eye tracking camera.

14. The method of claim 12 wherein identifying natural user input in the image data captured by one or more image capture devices of the NED device further comprises recognizing a pointing gesture from the image data.

15. The method of claim 10 wherein retrieving the at least some secondary image data using one or more communication techniques allowing for lossy transmission further comprises omitting performing one or more error correction techniques on the at least some secondary image data being within the image data being received.

16. A near-eye display (NED) device system comprising:

a near-eye display (NED) device including

a near-eye support structure,

a near-eye display (NED) supported by the near-eye support structure and having a display field of view, and

an image generation unit supported by the near-eye support structure and outputting image data which is optically coupled into the near-eye display (NED) for projection display in the display field of view;

one or more processors being communicatively coupled to the image generation unit for controlling display of image data;

the one or more processors determining a user focal region in the display field of view based on data representing natural user input stored in an accessible memory;

the one or more processors identifying user focal region image data for display within the user focal region and identifying secondary image data for display outside the user focal region based on one or more applications being executed by the NED device system; and

a first communication module being communicatively coupled to the one or more processors and being communicatively coupled over a communication medium to a computer system, the communication module using one or more communication techniques for satisfying lossless transmission criteria when retrieving user focal region image data from the computer system and using one or more communication techniques allowing for lossy transmission when retrieving secondary image data from the computer system.

17. The near-eye display device system of claim 16 wherein the first communication module and the one or more processors are supported by the near-eye support structure.

18. The near-eye display device system of claim 16 further comprising a companion processing module separate from and communicatively coupled to the NED device wherein the companion processing module includes the first communication module.

19. The near-eye display device system of claim 16 further comprising a companion processing module separate from and communicatively coupled to the NED device wherein the companion processing module is the computer system communicatively coupled to the first communication module;

and

the companion processing module including a second communication module using one or more communication techniques for satisfying lossless transmission criteria when transmitting user focal region image data to the first communication module and using one or more communication techniques allowing for lossy transmission when transmitting secondary image data to the first communication module.

20. The near-eye display device system of claim 19 wherein the companion processing module further comprises one or more processors which downsample an image resolution of the secondary image data prior to the second communication module transmitting the secondary image data to the first communication module.