VISUAL SYSTEM HAVING MULTIPLE CAMERAS

• US 20140125760A1
• Filed: 11/05/2012
• Published: 05/08/2014
• Est. Priority Date: 11/05/2012
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Au, Kwong Wing, Venkatesha, Sharath
• Granted Patent: US 9,338,370 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A visual system including a near-to-eye display supported by a frame adapted to be worn by a user such that each display is positioned proximate an eye of a wearer, a set of cameras supported by the frame, the cameras having different overlapping fields of view, and a processor coupled to receive images from the cameras and adapted to integrate and transform images from the set of cameras to provide a mosaic image for viewing on the display having a wider field of view than an individual camera provides.

Claims

1. A visual system comprising:

a near-to-eye display supported by a frame adapted to be worn by a user such that each display is positioned proximate an eye of a wearer;

a set of cameras supported by the frame, the cameras having overlapping fields of view; and

a processor coupled to receive images from the cameras and adapted to integrate and transform images from the set of cameras to provide a fused, corrected image having a wider field of view than an individual camera provides.

2. The visual system of claim 1 and further comprising an additional near-to-eye display and set of cameras for a second eye of the user.

3. The visual system of claim 2 wherein the sets of cameras have common cameras shared in the two sets.

4. The visual system of claim 2 wherein a subset of cameras for each display has far field focus and a further subset of cameras for each display has a near field focus.

5. The visual system of claim 4 wherein the processor is adapted with a corrective transformation that performs combinations of mosaic, fusion, focus determination and correction, view center transform, and camera-display FOV transformation utilizing overlapped areas of the adjacent cameras, auxiliary sensor information and brightness and calibration parameters to provide the image that has optimal brightness, correct focus, wide field of view, and realistic object size for the user.

6. The visual system of claim 2 and further comprising a sensor subsystem wherein the sensor subsystem provides orientation of the visual system and the pupil position relative to the frame of the visual system, and the processor selects the video feed from the cameras based on the orientation of the visual system and the pupil position relative to the frame of the visual system.

7. The visual system of claim 2 wherein a subset of cameras in the set of cameras have different intrinsic and extrinsic camera properties including the field of view, focal length, resolution and orientation in space.

8. The visual system of claim 2 wherein the camera in the sets of cameras is responsive to visible, near infrared, short wave infrared, mid-wave infrared, long-wave infrared or ultraviolet.

9. The visual system of claim 2 wherein each camera includes a micro drive to provide a small range of auto focal adjustments.

10. The visual system of claim 1 and further comprising an additional near-to-eye display wherein the output of the processor is fed to both displays for both eyes of the user.

11. A method comprising:

receiving images from a set of cameras mounted on a frame to be worn by a user; and

integrating and transforming the received images from the set of cameras to form a fused, corrected image for a near-to-eye display supported by the frame, the fused corrected image having a wider field of view than that provided by an individual camera.

12. The method of claim 11, wherein the set of cameras has a near field camera and a far field camera, and wherein the images from each are fused and focus adjusted to display a sharp image on the near-to-eye display.

13. The method of claim 12 wherein fusing and correcting the images includes applying a perspective transformation based on a relative location of the user'"'"'s eyes and the location of the cameras.

14. The method of claim 12 wherein the images overlap for a portion of adjacent images due to overlapping fields of view of the cameras, the method further comprising determining sharper focus between overlapping portion of the images and integrating the images to form the mosaic image such that the mosaic image is in correct focus for the user.

15. The method of claim 11 wherein integrating the images includes utilizing overlapped areas of the adjacent cameras and calibration parameters to provide the mosaic image for the display.

16. The method of claim 11 wherein a subset of cameras in the set of cameras have different intrinsic and extrinsic camera properties including the field of view, focal length, resolution and orientation in space.

17. The method of claim 11 wherein the camera in the sets of cameras is responsive to visible, near infrared, short wave infrared, mid-wave infrared, long-wave infrared or ultraviolet.

18. The method of claim 11 wherein each camera includes a micro drive to provide a small range of auto focal adjustments.

19. The method of claim 11 and further comprising an additional near-to-eye display wherein the output of the processor is fed to both displays for both eyes of the user.

20. The method of claim 11 and further comprising of a sensor subsystem wherein the sensor subsystem provides orientation of the visual system and the pupil position relative to the frame of the visual system, and the processor selects the video feed from the far field or near field camera based on the orientation of the visual system and the pupil position relative to the frame of the visual system.