IMAGE PROCESSING METHOD, APPARATUS, AND DEVICE

• US 20200137302A1
• Filed: 12/20/2019
• Published: 04/30/2020
• Est. Priority Date: 06/23/2017
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: LI, An, WANG, Qingping, WANG, Tizheng
• Granted Patent: US 11,095,812 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 13/004 having four lenses
• CPC Class Codes: G02B 13/0045 having five or more lenses
• CPC Class Codes: G02B 7/285 including two or more different focus detection devices, e.g. both an active and a passive focus detecting device
• CPC Class Codes: G02B 9/62 having six components only
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen

Abstract

An embodiment of the present invention discloses an image processing method, an apparatus, and a device. The method is applied to a terminal having two specially manufactured cameras. A first sub-image that is of a to-be-photographed object and that is photographed by a first camera is obtained, where a corresponding field-of-view range is [0, θ₁]; a second sub-image that is of the to-be-photographed object and that is photographed by a second camera is obtained, where a corresponding field-of-view range is [θ₂, θ₃], and quality of the first sub-image and the second sub-image satisfies a definition requirement of an extra-large aperture; and the first sub-image and the second sub-image are spliced and fused to obtain a target image that has a larger field-of-view range and satisfies the extra-large aperture.

Claims

1. An image processing method, comprising:

obtaining a first image of an object photographed by a first camera of a photographing device having the first camera and a second camera, wherein optical axes of the first camera and the second camera are parallel to each other, and a spacing between the first camera and the second camera is less than a preset distance, wherein f-numbers of the first camera and the second camera are less than 1.6, and neither of quantities of lenses of the first camera or the second camera is greater than 6;

obtaining a second image of the object photographed by the second camera;

obtaining a first sub-image of the first image, wherein definition of the first sub-image satisfies a preset definition standard;

obtaining a second sub-image of the second image, wherein definition of the second sub-image satisfies the preset definition standard, there is an image intersection set between the first sub-image and the second sub-image, and an image union set between the first sub-image and the second sub-image can represent the object; and

performing fusion processing on the first sub-image and the second sub-image to obtain a target image.

2. The method according to claim 1, wherein the obtaining a first sub-image of the first image comprises:

obtaining a first physical design parameter of the first camera, wherein the first physical design parameter indicates that, in any image photographed by the first camera, definition of an image in a first region is higher than definition of the image in a second region and satisfies the preset definition standard, and the second region is a complementary set of the first region in the image photographed by the first camera;

obtaining a first region of the first image based on the first physical design parameter;

obtaining an image receiving region P of an image sensor in the first camera; and

determining, as the first sub-image, an image of a region S1 that is an intersection set between the first region of the first image and P.

3. The method according to claim 1, wherein the obtaining a second sub-image of the second image comprises:

obtaining a second physical design parameter of the second camera, wherein the second physical design parameter indicates that, in any image photographed by the second camera, definition of an image in a third region is higher than definition of the image in a fourth region and satisfies the preset definition standard, and the fourth region is a complementary set of the third region in the image photographed by the second camera;

obtaining a third region of the second image based on the second physical design parameter;

obtaining an image receiving region Q of an image sensor in the second camera; and

determining, as the second sub-image, an image of a region S2 that is an intersection set between the third region of the second image and Q.

4. The method according to claim 1, wherein the first physical design parameter indicates that:

a modulation transfer function (MTF) value corresponding to an image photographed by the first camera in a field-of-view range of [0, θ

₁] at a preset spatial frequency is greater than a preset first threshold; and

an MTF value corresponding to an image photographed by the first camera in another field-of-view range at the preset spatial frequency is not greater than the preset first threshold, wherein θ

₁ is less than ½

of a field of view of the first camera.

5. The method according to claim 4, wherein the second physical design parameter indicates that:

an MTF value corresponding to an image photographed by the second camera in a field-of-view range of [θ

₂, θ

₃] at the preset spatial frequency is greater than a preset second threshold; and

an MTF value corresponding to an image photographed by the second camera in another field-of-view range at the preset spatial frequency is not greater than the preset second threshold, wherein θ

₃ is less than ½

of a field of view of the second camera, and 0<

θ

₂<

θ

₁<

θ

₃.

6. The method according to claim 1, wherein the performing fusion processing on the first sub-image and the second sub-image to obtain a target image comprises:

determining an image of a region S3 that is an intersection set between an image of a region S1 and an image of a region S2;

determining an image of a region S32 that is a complementary set of S3 in S2; and

performing fusion processing on the image of S1 and the image of S32 to obtain the target image.

7. The method according to claim 1, wherein the performing fusion processing on the first sub-image and the second sub-image, to obtain a target image comprises:

determining an image of a region S3 that is an intersection set between an image of region S1 and an image of region S2;

determining an image of a region S31 that is a complementary set of S3 in S1; and

performing fusion processing on the image of S31 and the image of S2 to obtain the target image.

8. The method according to claim 1, wherein the performing fusion processing on the first sub-image and the second sub-image, to obtain a target image comprises:

determining an image of a region S3 that is an intersection set between an image of region S1 and an image of region S2;

determining an image of a region S31 that is a complementary set of S3 in S1;

determining an image of a region S32 that is a complementary set of S3 in S2;

performing, based on S1 and S2, enhancement processing on S3 to obtain an image of S4; and

performing fusion processing on the image of S31, the image of S32, and the image of S4, to obtain the target image.

9. The method according to claim 1, wherein the first camera and the second camera have a same f-number, or a same focal length, or a same field of view.

10. The method according to claim 1, further comprising:

controlling an adjustment apparatus of the photographing device to adjust the spacing between the first camera and the second camera.

11. An image processing apparatus, comprising:

a first obtaining module configured to obtain a first image an object photographed by a first camera of a photographing device having the first camera and a second camera, wherein optical axes of the first camera and the second camera are parallel to each other, and a spacing between the first camera and the second camera is less than a preset distance, wherein f-numbers of the first camera and the second camera are less than 1.6, and neither of quantities of lenses of the first camera or the second camera is greater than 6;

a second obtaining module configured to obtain a second image of the object photographed by the second camera;

a third obtaining module configured to obtain a first sub-image of the first image, wherein definition of the first sub-image satisfies a preset definition standard;

a fourth obtaining module configured to obtain a second sub-image of the second image, wherein definition of the second sub-image satisfies the preset definition standard, wherein there is an image intersection set between the first sub-image and the second sub-image, and an image union set between the first sub-image and the second sub-image can represent the object; and

an image splicing module configured to perform fusion processing on the first sub-image and the second sub-image to obtain a target image.

12. The apparatus according to claim 11, wherein the third obtaining module is configured to:

obtain a first physical design parameter of the first camera, wherein the first physical design parameter indicates that, in any image photographed by the first camera, definition of an image in a first region is higher than definition of the image in a second region and satisfies the preset definition standard, and the second region is a complementary set of the first region in the image photographed by the first camera;

obtain a first region of the first image based on the first physical design parameter;

obtain an image receiving region P of an image sensor in the first camera; and

determine, as the first sub-image, an image of a region S1 that is an intersection set between the first region of the first image and P.

13. The apparatus according to claim 11, wherein the fourth obtaining module is configured to:

obtain a second physical design parameter of the second camera, wherein the second physical design parameter indicates that, in any image photographed by the second camera, definition of an image in a third region is higher than definition of the image in a fourth region and satisfies the preset definition standard, and the fourth region is a complementary set of the third region in the image photographed by the second camera;

obtain a third region of the second image based on the second physical design parameter;

obtain an image receiving region Q of an image sensor in the second camera; and

determine, as the second sub-image, an image of a region S2 that is an intersection set between the third region of the second image and Q.

14. The apparatus according to claim 11, wherein the first physical design parameter indicates that:

a modulation transfer function (MTF) value corresponding to an image photographed by the first camera in a field-of-view range of [0, θ

₁] at a preset spatial frequency is greater than a preset first threshold; and

an MTF value corresponding to an image photographed by the first camera in another field-of-view range at the preset spatial frequency is not greater than the preset first threshold, wherein θ

₁ is less than ½

of a field of view of the first camera; and

wherein the second physical design parameter indicates that;

an MTF value corresponding to an image photographed by the second camera in a field-of-view range of [θ

₂, θ

₃] at the preset spatial frequency is greater than a preset second threshold; and

an MTF value corresponding to an image photographed by the second camera in another field-of-view range at the preset spatial frequency is not greater than the preset second threshold, wherein θ

₃ is less than ½

of a field of view of the second camera, and 0<

θ

₂<

θ

₁<

θ

₃.

15. The apparatus according to claim 13, wherein the image splicing module is configured to implement any one of the following three manners:

manner 1;

determining an image of a region S3 that is an intersection set between an image of a region S1 and an image of a region S2;

determine an image of a region S32 that is a complementary set of S3 in S2; and

perform fusion processing on the image of S1 and the image of S32, to obtain the target image;

or

manner 2;

determining an image of a region S3 that is an intersection set between an image of region S1 and an image of region S2;

determining an image of a region S31 that is a complementary set of S3 in S1; and

perform fusion processing on the image of S31 and the image of S2, to obtain the target image;

or

manner 3;

determining an image of a region S3 that is an intersection set between an image of region S1 and an image of region S2;

determining an image of a region S31 that is a complementary set of S3 in S1;

determining an image of a region S32 that is a complementary set of S3 in S2;

performing, based on S1 and S2, enhancement processing on S3 according to a preset enhancement algorithm to obtain an image of S4; and

performing fusion processing on the image of S31, the image of S32, and the image of S4 to obtain the target image.

16. The apparatus according to claim 11, further comprising an adjusting module to adjust the spacing between the first camera and the second camera.

17. A terminal device, comprising:

a first camera;

a second camera;

a memory;

a processor; and

a bus, wherein the first camera, the second camera, the memory, and the processor are coupled via the bus,

wherein optical axes of the first camera and the second camera are parallel to each other, and a spacing between the first camera and the second camera is less than a preset distance, and wherein f-numbers of the first camera and the second camera are less than 1.6, and neither of quantities of lenses of the first camera and the second camera is greater than 6;

wherein the each of the first and second cameras is configured to collect an image signal under a control of the processor;

the memory is configured to store a computer program and an instruction; and

the processor is configured to invoke the computer program and the instruction stored in the memory to perform a method, the method including

obtaining a first image of an object photographed by the first camera;

obtaining a second image of the object photographed by the second camera;

obtaining a first sub-image of the first image, wherein definition of the first sub-image satisfies a preset definition standard;

obtaining a second sub-image of the second image, wherein definition of the second sub-image satisfies the preset definition standard, there is an image intersection set between the first sub-image and the second sub-image, and an image union set between the first sub-image and the second sub-image can represent the object; and

performing fusion processing on the first sub-image and the second sub-image to obtain a target image.

18. The terminal device according to claim 17, further comprising an antenna system, wherein the antenna system receives or sends a wireless communication signal under the control of the processor, to implement wireless communication with a mobile communications network;

and wherein the mobile communications network comprises one or more of the following;

a GSM network, a CDMA network, a 3G network, an FDMA network, a TDMA network, a PDC network, a TACS network, an AMPS network, a WCDMA network, a TD-SCDMA network, a Wi-Fi network, and an LTE network.