Color vision deficit correction

• US 9,398,844 B2
• Filed: 06/18/2012
• Issued: 07/26/2016
• Est. Priority Date: 06/18/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ambrus, Tony, Smith-Kipnis, Adam, Latta, Stephen, McCulloch, Daniel, Mount, Brian, Geisner, Kevin, McIntyre, Ian
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Wilson, Nicholas R
• Application Number:
  US13/526,315
• Publication Number:
  US 20130335435A1
• Time in Patent Office: 1,499 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 3/066 for testing colour vision
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2380/08 Biomedical applications
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence

Abstract

Embodiments related to improving a color-resolving ability of a user of a see-thru display device are disclosed. For example, one disclosed embodiment includes, on a see-thru display device, constructing and displaying virtual imagery to superpose onto real imagery sighted by the user through the see-thru display device. The virtual imagery is configured to accentuate a locus of the real imagery of a color poorly distinguishable by the user. Such virtual imagery is then displayed by superposing it onto the real imagery, in registry with the real imagery, in a field of view of the user.

Claims

1. In a see-thru display device, a method to address a color vision deficiency, the method comprising:

constructing virtual imagery to superpose onto a real world background viewable through the see-thru display device, where constructing the visual imagery includes identifying one or more loci of one or more real world objects of a color poorly distinguishable based upon the color deficiency and identifying a focal point of a user of the see-thru display device, the virtual imagery configured to accentuate the one or more loci such that loci nearer to the focal point are accentuated to a greater degree than loci farther from the focal point; and

displaying the virtual imagery such that the virtual imagery is superposed onto the one or more real world objects, in spatial registry with the one or more real world objects, in a field of view of the see-thru display device.

2. The method of claim 1 wherein the virtual imagery is configured to write one or more of text and a symbol adjacent the one or more loci.

3. The method of claim 1 further comprising acquiring an image of the real world background with a front-facing camera of the see-thru display device, wherein the virtual imagery is constructed based on the acquired image, and the one or more loci are identified from the acquired image.

4. The method of claim 1, further comprising presenting a visual color test, and receiving one or more user inputs in response to the visual color test.

5. The method of claim 1 further comprising receiving input that specifies the color poorly distinguishable, wherein constructing the virtual imagery includes accentuating one or more loci of the specified color of the one or more real world objects.

6. The method of claim 1 wherein constructing the virtual imagery includes identifying a plurality of loci of matching colors in the one or more real world objects and overwriting the loci with one or more of text, a symbol, and a segmentation pattern to indicate the matching colors.

7. The method of claim 1 wherein the virtual imagery includes first virtual imagery and second virtual imagery, the method further comprising:

identifying a first real world object and a second real world object in the real world background, where the first real world object is a recognized object and the second real world object is an unrecognized object; and

displaying the first virtual imagery such that the first virtual imagery is superposed onto the first real world object and displaying the second virtual imagery such that the second virtual imagery is superposed onto the second real world object, where the first real world object is accentuated differently than the second real world object.

8. The method of claim 7 wherein the second virtual imagery is chosen to be distinguishable when superposed on the first real world object together with the first virtual imagery.

9. The method of claim 1 wherein the first and second real world objects are recognized based on one or both of object color and object shape.

10. A see-thru display device configured to address a color-resolving ability of a user having a color vision deficiency, the see-thru display device comprising:

a front-facing camera configured to acquire an image of a real world background viewable through the see-thru display device;

a logic subsystem operatively coupled to a storage subsystem, the storage subsystem storing instructions that cause the logic subsystem to construct virtual imagery to superpose onto the real world background, the virtual imagery configured to accentuate one or more loci of one or more real world objects of a color poorly distinguishable based upon the color vision deficiency wherein constructing the virtual imagery includes identifying a plurality of loci of matching colors in the real world object and overwriting the loci with one or more of text, a symbol, and a segmentation pattern to indicate the matching colors;

an eye tracker configured to track a focal point of the user, wherein the virtual imagery is configured to accentuate a first locus nearer to the focal point to a greater degree, and accentuate a second locus farther from the focal point to a lesser degree; and

a microdisplay configured to display the virtual imagery such that the virtual imagery is superposed onto the one or more real world objects, in registry with the one or more real world objects, in a field of view of the see-thru display device.

11. The device of claim 10 further comprising a controllable dimming filter configured to controllably reduce a brightness of the real world background as sighted by the user.

12. The device of claim 10, where the instructions are executable to measure a length of time the user spends observing selected aspects of a scene in order to identify the color vision deficiency of the user.

13. The device of claim 10, where the instructions are executable to recognize one or more real world objects in the real world background, and where the virtual imagery is configured to accentuate loci of recognized real world objects differently from loci of unrecognized real world objects.

14. The device of claim 10, where the instructions are executable to recognize a first real world object and a second real world object in the real world background, and where the virtual imagery is configured to accentuate loci of the first real world object to a greater extent than loci of the second real world object based on a nature of the first real world object.

15. The device of claim 14, where the first real world object is a sign.

16. The device of claim 10, where the instructions are executable to receive an input from the user indicating a severity of the color vision deficiency.

17. The device of claim 10, where the instructions are executable to construct first virtual imagery for display to a first eye of the user and second virtual imagery for display to a second eye of the user, where the first virtual imagery accentuates loci of real world objects differently from the second virtual imagery.

18. In a see-thru display device, a method to improve a color-resolving ability of a user of the see-thru display device based upon a color vision deficiency of the user, the method comprising:

acquiring, with a front-facing camera of the see-thru display device, an image of a real world background viewable by the user through the see-thru display device;

recognizing a first real world object and a second real world object in the acquired image;

constructing virtual imagery to superpose onto the first and second real world objects, the virtual imagery configured to accentuate loci of the first and second real world objects having a color poorly distinguishable based upon the color vision deficiency, and constructed to accentuate loci of the first real world object to a greater extent than loci of the second real world object based on a nature of the first real world object; and

displaying the virtual imagery such that the virtual imagery is superposed onto the first and second real world objects, in spatial registry with the first and second real world objects, in a field of view of the see-thru display device.

19. The method of claim 18 wherein the first real world object is a sign.

20. The method of claim 18 wherein the virtual imagery is configured to shift the color of the first and second real world objects.