Terminal provided with camera and shooting method

• US 11,082,547 B2
• Filed: 07/07/2017
• Issued: 08/03/2021
• Est. Priority Date: 07/07/2017
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Xu, Gang, Sun, Tuo, Jau, Hung-yuan, Chu, Chiaching, Liu, Kangchung, Hou, Hsienchang, Li, Xin
• Primary Examiner(s): Dagnew, Mekonnen D
• Application Number:
  US16/629,237
• Publication Number:
  US 20200195764A1
• Time in Patent Office: 1,488 Days
• CPC Class Codes: H04M 1/0264 for a camera module assembly
• CPC Class Codes: H04M 1/0266 for a display module assembly
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H05K 5/0017 with operator interface units
• CPC Class Codes: H10K 59/121 characterised by the geometry or disposition of pixel elements

Abstract

A terminal, comprising at least two front cameras located behind a display panel, wherein an area corresponding to a position of the at least one camera on the display panel is a first area, and other than the first area on the display panel is a second area, wherein each pixel unit of the first area and the second area respectively comprises a transparent portion and a non-transparent portion and a proportion of the transparent portion in the first area is greater than that of the second area. The terminal further comprises a processor configured to obtain N pictures taken by the at least two cameras, and process the N pictures by complementing the occluded areas based on pictures taken by the at least two cameras to obtain a picture in which the occluded area is completely or partially recovered.

Claims

1. A terminal, comprising:

a display panel comprising;

a first area comprising first pixel units, wherein the first pixel units comprise first transparent portions and first non-transparent portions; and

a second area comprising second pixel units, wherein the second pixel units comprise second transparent portions and second non-transparent portions, and wherein a proportion of the first transparent portions in the first area is greater than a proportion of the second transparent portions in the second area;

two front cameras located behind the display panel, wherein the first area corresponds to a position corresponding to the two front cameras, and wherein the two front cameras are configured to take pictures comprising occluded areas resulting from occlusion by the first non-transparent portions; and

a processor coupled to the display panel and the two front cameras and configured to;

obtain the pictures from the two front cameras; and

process the pictures by complementing the occluded areas based on the pictures to obtain a picture in which the occluded areas are completely or partially recovered.

2. The terminal according to claim 1, wherein the first transparent portions or the second transparent portions comprise transparent glass or a transparent plastic film, and wherein the first non-transparent portions or the second non-transparent portions comprise a thin film transistor and a metal wire in each first pixel unit or second pixel unit.

3. The terminal according to claim 1, wherein a quantity of pixels per inch (PPI) of the first area is 1/M of a PPI of the second area, and wherein M is greater than 1.

4. The terminal according to claim 1, wherein the first area comprises a pixel circuit without internal compensation, and wherein the second area comprises a pixel circuit with internal compensation.

5. The terminal according to claim 1, wherein the processor is further configured to perform an interpolation operation on an occluded pixel in the picture using information of adjacent pixels of the occluded pixel.

6. The terminal according to claim 1, wherein the processor is further configured to:

recognize physical information of an occluded pixel in the picture; and

recover information of the occluded pixel in the picture based on content of the physical information.

7. The terminal according to claim 1, wherein the processor is further configured to increase an overall brightness of the first area so that a similarity between brightness curves of the first area and the second area is greater than a preset value.

8. The terminal according to claim 1, wherein the processor is further configured to insert a black frame into the display panel during shooting.

9. A shooting method implemented by a terminal, wherein the method comprises:

obtaining pictures taken by at least two cameras of the terminal that are located behind a display panel of the terminal, wherein the pictures comprise occluded areas resulting from occlusion by first non-transparent portions of a first area of the display panel that corresponds to a position of the at least two cameras, wherein the first area comprises first pixel units that comprise first transparent portions and the first non-transparent portions, wherein a second area of the display panel comprises second pixel units comprising second transparent portions and second non-transparent portions, and wherein a proportion of the first transparent portions in the first area is greater than a proportion of the second transparent portions in the second area; and

processing the pictures by complementing the occluded areas based on the pictures to obtain a first picture in which the occluded areas are completely or partially recovered.

10. The shooting method according to claim 9, wherein when the occluded areas are partially recovered in the first picture, the method further comprises processing, using an image processing algorithm, the first picture to obtain a second picture in which the occluded areas are completely recovered.

11. The shooting method according to claim 10, wherein processing the first picture comprises performing an interpolation operation on an occluded pixel in the first picture using information of adjacent pixels of the occluded pixel.

12. The shooting method according to claim 10, wherein processing the first picture comprises:

recognizing physical information of an occluded pixel in the first picture; and

recovering information of the occluded pixel in the first picture based on content of the physical information.

13. The shooting method according to claim 9, wherein a pixels per inch (PPI) of the first area is 1/M of a PPI of the second area, wherein M is greater than 1, and wherein the method further comprises compressing display data of M adjacent pixels in the first area into display data of one pixel for display.

14. The shooting method according to claim 13, wherein the method further comprises increasing overall brightness of the first area so that a similarity between brightness curves of the first area and the second area is greater than a preset value.

15. The shooting method according to claim 9, wherein the first area comprises a pixel circuit without internal compensation, and wherein the method further comprises performing external compensation on the first pixel units.

16. The shooting method according to claim 9, wherein the method further comprises inserting a black frame into the display panel during shooting.

17. A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable medium that, when executed by a processor, cause a terminal to:

obtain pictures taken by at least two cameras of the terminal that are located behind a display panel of the terminal, wherein the pictures comprise occluded areas resulting from occlusion by first non-transparent portions of a first area of the display panel that corresponds to a position of the at least two cameras, wherein the first area comprises first pixel units that comprise first transparent portions and the first non-transparent portions, wherein a second area of the display panel comprises second pixel units comprising second transparent portions and second non-transparent portions, and wherein a proportion of the first transparent portions in the first area is greater than a proportion of the second transparent portions in the second area; and

process the pictures by complementing the occluded areas based on the pictures to obtain a first picture in which the occluded areas are completely or partially recovered.

18. The computer program product of claim 17, wherein when the occluded areas are partially recovered in the first picture, the computer-executable instructions further cause the terminal to process, using an image processing algorithm, the first picture to obtain a second picture in which the occluded areas are completely recovered.

19. The computer program product of claim 18, wherein the computer-executable instructions further cause the terminal to process the first picture by causing the terminal to perform an interpolation operation on an occluded pixel in the first picture using information of adjacent pixels of the occluded pixel.

20. The computer program product of claim 18, wherein the computer-executable instructions further cause the terminal to process the first picture by causing the terminal to:

recognize physical information of an occluded pixel in the first picture; and

recover information of the occluded pixel in the first picture based on content of the physical information.