Mobile device, and image processing method for mobile device

• US 10,681,281 B2
• Filed: 08/28/2018
• Issued: 06/09/2020
• Est. Priority Date: 12/28/2017
• Status: Active Grant

• Current Assignee: HTC Corporation
• Original Assignee: HTC Corporation
• Inventors: Ho, Chia-Chu, Li, Chih-Hung, Lee, Ching-Hao
• Primary Examiner(s): Berhan, Ahmed A
• Application Number:
  US16/114,219
• Publication Number:
  US 20190208133A1
• Time in Patent Office: 651 Days
• CPC Class Codes: G09G 2300/023 Display panel composed of stacked panels
• CPC Class Codes: G09G 3/32 semiconductive, e.g. using light-emitting diodes [LED]
• CPC Class Codes: G09G 3/3208 organic, e.g. using organic light-emitting diodes [OLED]
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

A mobile device includes a see-through display unit, a back cover, a first optical unit and a processing unit. The first optical unit is arranged between the see-through display unit and the back cover, and is configured to capture a first optical result along a first optical path, which passing through an active region of the see-through display unit. The processing unit is coupled to the first optical unit and the see-through display unit. The processing unit is configured to determine a first portion on the see-through display unit according to the first optical path in response to the first optical unit is triggered to capture the first optical result, and control the first portion of the see-through display unit to be transparent while capturing the first optical detection result or adjusting the first optical detection result according to display contents within the first portion on the see-through display unit.

Claims

1. A mobile device comprising:

a see-through display unit;

a back cover;

a first optical unit arranged between the see-through display unit and the back cover, and configured to capture a first optical detection result along a first optical path, the first optical path passing through an active region of the see-through display unit; and

a processing unit coupled to the first optical unit and the see-through display unit, wherein the processing unit is configured to;

in response to the first optical unit is triggered to capture the first optical detection result, determine a first portion on the see-through display unit according to the first optical path;

control the first portion of the see-through display unit to be transparent while capturing the first optical detection result or adjusting the first optical detection result according to display contents within the first portion on the see-through display unit;

detect a plurality of pixel values or light strength of the display contents within the first portion on the see-through display unit; and

subtract the detected pixel values or the detected light strength of the display contents within the first portion on the see-through display unit from a plurality of pixel values of the first optical detection result respectively to form a first processed optical detection result.

2. The mobile device as claimed in claim 1, wherein the processing unit is further configured to:

input the adjusted first optical detection result to an application corresponding to the first optical unit.

3. The mobile device as claimed in claim 1, wherein the first processed optical detection result is equal to an optical detection result captured by the first optical unit without being affected by the display contents on the see-through display unit.

4. The mobile device as claimed in claim 1, further comprises:

a second optical unit arranged between the see-through display unit and the back cover, and configured to capture a second optical detection result along a second optical path, the second optical path passing through the active region of the see-through display unit,

wherein the processing unit is further configured to;

in response to the second optical unit is triggered to capture the second optical detection result, determining a second portion on the see-through display unit according to the second optical path; and

control the second portion of the see-through display unit to be transparent while capturing the second optical detection result or adjusting the second optical detection result according to display contents within the second portion on the see-through display unit.

5. The mobile device as claimed in claim 1, wherein the processing unit further configured to:

control the first portion of the see-through display unit to be transparent for a first time period when the first optical unit is in use, wherein the first time period is related to the first optical unit.

6. The mobile device as claimed in claim 5, wherein the first time period is equal to or longer than a time length of capturing an image, examining a light strength and detecting an infrared light by the first optical unit.

7. The mobile device as claimed in claim 5, wherein an area of the first portion of the see-through display unit is equal to or longer than an area of the first optical unit.

8. An image processing method, suitable for a mobile device comprising a see-through display unit and a first optical unit arranged behind the see-through display unit and configured to generate a first optical detection result along a first optical path through the see-through display unit, the image processing method comprising:

extracting a first portion of display contents on the see-through display unit when the first optical unit captures the first optical detection result;

forming a first processed optical detection result by processing the first optical detection result with the first portion of the display contents displayed on the see-through display unit; and

inputting the first processed optical detection result to an application corresponding to the first optical unit;

wherein the step of forming the first processed optical detection result by processing the first optical detection result with the first portion of the display contents displayed on the see-through display unit comprises;

detecting a plurality of pixel values or a light strength of the first portion of the display contents on the see-through display unit; and

subtracting the detected pixel values or the detected light strength of the first portion of the display contents on the see-through display unit from a plurality of pixel values of the first optical detection result respectively to form the first processed optical detection result.

9. The image processing method as claimed in claim 8, wherein the first processed optical detection result is equal to an optical detection result captured by the first optical unit without being affected by the display contents on the see-through display unit.

10. The image processing method as claimed in claim 8, wherein the first optical unit comprises a proximity sensor, a camera, a structure light transmitter, a structure light receiver and an ambient light sensor.

11. The image processing method as claimed in claim 8, wherein the mobile device further comprises a second optical unit arranged behind the see-through display unit and configured to generate a third optical detection result along a second optical path through the see-through display unit, the image processing method further comprises:

extracting a second portion of the display contents on the see-through display unit when the second optical unit captures the third optical detection result;

forming a fourth optical detection result by processing the third optical detection result with the second portion of the display contents displayed on the see-through display unit; and

inputting the fourth optical detection result to an application corresponding to the second optical unit.

12. An image processing method, suitable for a mobile device comprising a see-through display unit, a first optical unit arranged behind the see-through display unit and configured to generate a first optical detection result along a first optical path through the see-through display unit, and a second optical unit arranged behind the see-through display unit and configured to generate a third optical detection result along a second optical path through the see-through display unit, the image processing method comprising:

displaying display contents on the see-through display unit;

in response to the first optical unit is triggered to capture the first optical detection result, determining a first portion on the see-through display unit according to the first optical path;

controlling the first portion of the see-through display unit to be transparent while capturing the first optical detection result;

determining whether the first optical unit captures the first optical detection result or the second optical unit captures the third optical detection result;

in response to the first optical unit receives the first optical detection result, controlling the first portion of the see-through display unit to be transparent, and

in response to the second optical unit receives a second optical detection result, controlling a second portion of the see-through display unit to be transparent.

13. The image processing method as claimed in claim 12, further comprising:

controlling the first portion of the see-through display unit to be transparent for a first time period when the first optical unit is in use, wherein the first time period is related to the first optical unit.

14. The image processing method as claimed in claim 13, wherein the first time period is equal to or longer than a time length of capturing an image, examining a light strength and detecting an infrared light by the first optical unit.

15. The image processing method as claimed in claim 13, wherein an area of the first portion of the see-through display unit is equal to or larger than an area of the first optical unit.