IMAGE TRANSFORMING VISION ENHANCEMENT DEVICE

• US 20160104453A1
• Filed: 07/20/2015
• Published: 04/14/2016
• Est. Priority Date: 10/14/2014
• Status: Active Grant

• Current Assignee: Digital Vision Enhancement Inc.
• Original Assignee: Digital Vision Enhancement Inc.
• Inventors: Neuendorffer, Thomas P., Borenstein, Nathaniel S.
• Granted Patent: US 9,443,488 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G09G 2340/06 Colour space transformation
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85

Abstract

Image transforming vision enhancement device that enhances vision by transforming images provided by one or more cameras into modified images projected on one or more displays. The system may be embedded in glasses, contact lenses, binoculars, or other vision devices, in computer screens, or in components of moving vehicles. Image transformations may include modifying colors to assist colorblind users or to highlight color ranges, mapping invisible frequencies into visible colors, adding labels or graphics, and generating time-varying images with flashing or changing features. Images from multiple cameras may be combined, providing users with panoramic vision from a single device. Low light vision may be enhanced, and excessive glare may be attenuated. The system may magnify images with a variable magnification. User interfaces may be provided to configure and customize the image transformations.

Claims

1. An image transforming vision enhancement device comprising:

one or more displays that present display images to one or both eyes of a user, wherein each of said display images comprises a plurality of pixels with values in a display color space;

one or more cameras configured to capture camera images, wherein each of said camera images comprises a plurality of pixels with values in a camera color space;

a processor coupled to said one or more displays and to said one or more cameras, wherein said processor

receives said camera images from said one or more cameras;

transforms said camera images into said display images via an image transformation function;

sends said display images to said one or more displays to be viewed by said user;

wherein said image transformation function maps values from said camera color space into values of said display color space.

2. The image transforming vision enhancement device of claim 1, wherein

said one or more displays are configured to be placed or worn proximal to said one or both eyes of said user;

said one or more cameras are proximal to said one or more displays; and

the field of view of said one or more cameras has a non-empty intersection with the normal field of view of said user.

3. The image transforming vision enhancement device of claim 1, wherein said one or more displays, or said one or more cameras, or both of said one or more displays and said one or more cameras, are located within eyeglasses or contact lenses or intraocular lenses.

4. The image transforming vision enhancement device of claim 1, wherein said one or more displays, or said one or more cameras, or both of said one or more displays and said one or more cameras, are located within a television or a game system or a desktop computer or a laptop computer or a tablet computer or a mobile phone or a personal digital assistant or a diving helmet or a welding helmet or a medical device or a scientific device.

5. The image transforming vision enhancement device of claim 1, wherein

said one or more displays are coupled to a network;

said camera images, or said display images, or both of said camera images and said display images are transmitted over said network to said one or more displays.

7. The image transforming vision enhancement device of claim 1, wherein said one or more cameras, said image transformation function, or both, provide a zoom capability to increase or reduce magnification of said display images relative to said user'"'"'s normal vision.

8. The image transforming vision enhancement device of claim 7, wherein said one or more displays, or said one or more cameras, or both of said one or more displays and said one or more cameras, are located within binoculars or a monocular or a telescope or a microscope.

9. The image transforming vision enhancement device of claim 1, further comprising a plurality of said cameras, wherein said image transformation function combines said camera images from two or more of said cameras onto a single display image.

10. The image transforming vision enhancement device of claim 9 wherein said image transforming function combines said camera images from two or more of said cameras into a picture-in-picture display image.

11. The image transforming vision enhancement device of claim 9, wherein

the field of view of at least one of said cameras has a non-empty intersection with the normal field of view of said user;

and

the field of view of at least one other of said cameras has an empty intersection with the normal field of view of said user.

12. The image transforming vision enhancement device of claim 1, further comprising a plurality of said displays, wherein

said image transformation function comprises a plurality of display-specific image transformation functions;

each of said display-specific image transformation functions generates said display images for its corresponding display.

13. The image transforming vision enhancement device of claim 12 wherein two or more of said display-specific image transformation functions map at least one value from said camera color space into different values in said display color space.

14. The image transforming vision enhancement device of claim 1, wherein said image transformation function maps two or more values in said camera color space that are difficult or impossible for a vision-impaired user to distinguish into values in said display color space that said vision-impaired user can distinguish.

15. The image transforming vision enhancement device of claim 1, wherein said image transformation function comprises:

an image mask that selects pixels in said one or more camera images having said color values in one or more color ranges of interest; and

a selected pixel modification function that modifies said pixels in said one or more camera images that are selected by said image mask.

16. The image transforming vision enhancement device of claim 1, wherein said image transformation function alters the brightness of one or more of said pixels in said camera images to improve visibility of dark scenes, or to reduce the brightness of excessively bright regions, or both.

17. The image transforming vision enhancement device of claim 16 wherein said one or more displays, or said one or more cameras, or both of said one or more displays and said one or more cameras, are coupled to a moving vehicle.

18. The image transforming vision enhancement device of claim 1, wherein said camera color space includes one or more light frequencies outside of the human visible range of light.

19. The image transforming vision enhancement device of claim 18, wherein

said display color space represents frequencies or frequency mixtures within the human visible range of light;

said image transformation function partitions said display color space into a visible frequency representation subspace and an invisible frequency representation subspace;

said image transformation function maps values from said camera color space with one or more frequencies outside of the human visible range of light into said invisible frequency representation subspace;

said image transformation function maps values from said camera color space that represent frequencies or frequency mixtures within the human visible range of light into said visible frequency representation subspace.

20. The image transforming vision enhancement device of claim 19, wherein said image transformation function maps distinct values in said camera color space that represent frequencies or frequency mixtures within the human visible range of light into distinct values in said visible frequency representation subspace.

21. The image transforming vision enhancement device of claim 20, wherein

said camera color space comprises a camera hue value for each of said pixels of said camera images;

said display color space comprises a display hue value for each of said pixels of said display images, wherein said display hue values are within a display hue range;

said image transformation function

partitions said display hue range into a compressed visible hue range and an invisible hue representation range;

maps distinct camera hue values in said camera color space into distinct display hue values in said display hue range;

maps camera hue values in said camera color space that represent frequencies or frequency mixtures within the human visible range of light into display hue values in said compressed visible hue range; and

,

maps camera hue values in said camera color space that contain frequencies outside the human visible range of light into display hue values in said invisible hue representation range.

22. The image transforming vision enhancement device of claim 1, wherein

said display color space comprises a time series of color values for each of said pixels of each of said display images;

said processor

receives said camera images from said one or more cameras;

transforms each of said camera images into a time series of said display images via said image transformation function;

sends said time series of said display images to said one or more displays to be displayed in sequence and viewed by said user; and

wherein said image transformation function maps values from said camera color space into said time series of color values of said display color space.

23. The image transforming vision enhancement device of claim 22, wherein said image transformation function maps two or more values in said camera color space that are difficult or impossible for a vision-impaired user to distinguish into different time series values in said display color space that said vision-impaired user can distinguish.

24. The image transforming vision enhancement device of claim 23, wherein said different times series values comprise repeating time series values with different periods.

25. The image transforming vision enhancement device of claim 1, wherein said image transformation function further adds labels or graphics or patterns onto said display images that identify one or more colors.

26. The image transforming vision enhancement device of claim 1, further comprising a user interface that accepts input from an operator, wherein said input is used to select or modify said image transformation function.

27. The image transforming vision enhancement device of claim 7, further comprising a user interface that accepts input from an operator, wherein said input is used to select or modify the magnification, or the field of view, or both the magnification and the field of view, of said one or more cameras or of said image transformation function.

28. The image transforming vision enhancement device of claim 1, further comprising a microphone coupled with said processor wherein said processor generates display information related to sound obtained from said microphone and sends said display information related to sound to said one or more displays to be viewed by said user.

6. The image transforming vision enhancement device of claim 6, wherein said network is the Internet.