WIDE-ANGLE STEREOSCOPIC VISION WITH CAMERAS HAVING DIFFERENT PARAMETERS

• US 20180249148A1
• Filed: 02/23/2018
• Published: 08/30/2018
• Est. Priority Date: 02/24/2017
• Status: Active Grant

• Current Assignee: ImmerVision Inc. (f/k/a 6115187 Canada Inc.)
• Original Assignee: ImmerVision Inc. (f/k/a 6115187 Canada Inc.)
• Inventors: ROULET, Patrice, PARENT, Jocelyn, KONEN, Pierre, THIBAULT, Simon, NINI, Pascale, DU, Xiaojun, BATAILLE, Valentin, LEE, Jhinseok
• Granted Patent: US 10,666,923 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

A stereoscopic vision system uses at least two cameras having different parameters to image a scene and create stereoscopic views. The different parameters of the two cameras can be intrinsic or extrinsic, including, for example, the distortion profile of the lens in the cameras, the field of view of the lens, the orientation of the cameras, the positions of the cameras, the color spectrum of the cameras, the frame rate of the cameras, the exposure time of the cameras, the gain of the cameras, the aperture size of the lenses, or the like. An image processing apparatus is then used to process the images from the at least two different cameras to provide optimal stereoscopic vision to a display.

Claims

1. - A stereoscopic vision system for capturing a scene, the system comprising:

a. a first camera having a plurality of first imaging parameters and a first capture position relative to the scene, the first camera being configured to capture a first output image of the scene;

b. a second camera having a plurality of second imaging parameters and a second capture position relative to the scene, the second camera being configured to capture a second output image of the scene, the first and second capture positions being different from each other, one or more of the first imaging parameters being different from a corresponding one or more of the second imaging parameters, the first and second output images being different from each other according to the differing first and second capture positions and the one or more differing first and second imaging parameters;

c. a processing unit connected to the first and second cameras, the processing unit being configured to;

i. receive the first and second output images from the respective first and second cameras, and

ii. process the first and second output images according to a geometrical difference due to parallax from the first and second capture positions and according to any remaining differences due to the one or more differing first and second imaging parameters, in order to produce first and second processed images.

2. - The system of claim 1, further comprising at least one display for displaying the first and second processed images.

3. - The system of claim 2 wherein the at least one display is on one of a head-mounted virtual reality headset, an augmented reality headset, or a mobile device capable of insertion into a headset.

4. - The system of claim 1 wherein the one or more differing first and second imaging parameters includes a difference in a distortion profile of lenses inside each of the first and second cameras.

5. - The system of claim 1 wherein the one or more differing first and second imaging parameters includes a difference in a distortion profile of each of the first and second cameras after smart-binning by a sensor or processing inside each of the first and second cameras.

6. - The system of claim 1 wherein the one or more differing first and second imaging parameters includes either a difference in orientation of each of the first and second cameras, a difference in a field of view of each of the first and second cameras, a difference in a color spectrum of each of the first and second cameras, a difference in a frame rate of each of the first and second cameras or a difference in at least one of an exposure time, a gain, or an aperture size of each of the first and second cameras.

7. - The system of claim 1 wherein the first and second capture positions are modifiable to change the desired view of the scene.

8. - The system of claim 1 wherein the processing unit is further configured to:

iii. pre-store difference information on the capture position and imaging parameter differences of the first and second cameras,

iv. receive manual input of the difference information from a user, or

v. receive the difference information from the first and second cameras written in a marker and/or metadata.

9. - A stereoscopic vision system for capturing a scene, the system comprising:

a. a first camera including one or more lenses creating a first distortion profile, the first camera having a first capture position relative to the scene and being configured to capture a first output image of the scene; and

b. a second camera including one or more lenses creating a second distortion profile different from the first distortion profile, the second camera having a second capture position relative to the scene and being configured to capture a second output image of the scene, the first and second capture positions being different from each other, the first and second output images being different from each other according to the differing first and second capture positions and the differing first and second distortion profiles;

wherein the first and second distortion profiles are configured to minimize or avoid completely the processing of the distortion in the first and second images before they are displayed to a user.

10. - The system of claim 9, further comprising at least one display configured to display the first and second output images.

11. - The system of claim 10 wherein the at least one display is on at least one of a head-mounted virtual reality headset, an augmented reality headset, or a mobile device capable of insertion into a headset.

12. - A stereoscopic vision system for capturing a scene, the system comprising:

a. a first camera creating a first distortion profile either via smart-binning by a sensor or via processing inside the camera, the first camera having a first capture position relative to the scene and being configured to capture a first output image of the scene; and

b. a second camera creating a second distortion profile either via smart-binning by a sensor or via processing inside the camera, the second distortion profile being different from the first distortion profile, the second camera having a second capture position relative to the scene and being configured to capture a second output image of the scene, the first and second capture positions being different from each other, the first and second output images being different from each other according to the differing first and second capture positions and the differing first and second distortion profiles;

wherein the first and second distortion profiles are configured to minimize or avoid completely the processing of the distortion in the first and second images before they are displayed to a user.

13. - The system of claim 12, further comprising at least one display configured to display the first and second output images.

14. - The system of claim 13 wherein the at least one display is on at least one of a head-mounted virtual reality headset, an augmented reality headset, or a mobile device capable of insertion into a headset.

15. - A stereoscopic vision system for analyzing information about a scene, the system comprising:

a. a first camera having a plurality of first imaging parameters and a first capture position relative to the scene, the first camera being configured to capture a first output image of the scene;

b. a second camera having a plurality of second imaging parameters and a second capture position relative to the scene, the second camera being configured to capture a second output image of the scene, the first and second capture positions being different from each other, one or more of the first imaging parameters being different from a corresponding one or more of the second imaging parameters, the first and second output images being different from each other according to the differing first and second capture positions and the one or more differing first and second imaging parameters;

c. a processing unit connected to the first and second cameras, the processing unit being configured to;

i. receive the first and second output images from the respective first and second cameras, and

ii. process the first and second output images according to a geometrical difference due to parallax from the first and second capture positions and according to any remaining differences due to the one or more differing first and second imaging parameters, in order to analyze the scene.

16. - The system of claim 15, further comprising an algorithm unit connected to the processing unit and configured to receive an analysis of the scene from the processing unit.

17. - The system of claim 16, wherein the algorithm unit is using the analysis of the scene from the processing unit to calculate distance information about the scene from an origin point.

18. - The system of claim 16, wherein the algorithm unit is using the analysis of the scene from the processing unit to enhance image quality of the images.

19. - The system of claim 16, wherein the algorithm unit is using the analysis of the scene from the processing unit to generate a single image with superposed left eye and right eye stereoscopic content to be separated via active or passive glasses.

20. - The system of claim 16, wherein the processing unit or the algorithm unit are trained to analyze the scene or the resulting analysis via an artificial intelligence process.