Method of displaying information and displaying system thereof

• US 10,614,780 B2
• Filed: 09/06/2018
• Issued: 04/07/2020
• Est. Priority Date: 06/15/2018
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute, Intellectual Property Innovation Corporation
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chang, Chih-Chia, Lin, Yu-Hsin, Chen, Jian-Lung, Wang, Pi-Hsien, Kuo, Shin-Hong
• Primary Examiner(s): Doan, Phuc N
• Application Number:
  US16/122,894
• Publication Number:
  US 20190385573A1
• Time in Patent Office: 579 Days
• CPC Class Codes: B60K 2360/177 Augmented reality
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B61K 13/04 Passenger-warning devices attached to vehicles; Safety devices for preventing accidents to passengers when entering or leaving vehicles for preventing passengers from being injured by movements of doors B61D19/026
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/59 : inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 5/38 : with means for controlling the display position

Abstract

The present disclosure provides a method and system of displaying information, applied to a transparent display configured on a vehicle. The method includes: capturing an image including a target at a first time point; identifying the target in the image, so as to acquire a first relative position of the target; acquiring first identification information of the target; acquiring relative moving information between the vehicle and the target; estimating a first estimated position and first estimated time of the target that appears in a display region of the transparent display according to the first relative position of the target, the first time point and the relative moving information; and displaying prompting information corresponding to the target on the transparent display according to the first estimated time, the first estimated position of the target and the first identification information of the target.

Claims

1. A method of displaying information, applied to a transparent display configured at a side of a vehicle and implemented by a processor, the method of displaying information comprising:

identifying a user in the vehicle;

calculating a visual range of the user according to a face position or eye positions of the user;

capturing an image at a first time point, the image comprising a target;

identifying the target in the image, so as to acquire a first relative position of the target;

acquiring first identification information of the target;

acquiring relative moving information between the vehicle and the target;

estimating a first estimated position and first estimated time of the target that appears in a display region of the transparent display according to the visual range of the user, the first relative position of the target, the first time point and the relative moving information between the vehicle and the target before the target appears on the side of the vehicle;

calculating a position of the target that is projected to the display region of the transparent display according to the visual range of the user, the first relative position of the target, the first time point, the first estimated position of the target, the first estimated time and the relative moving information between the vehicle and the target; and

displaying prompting information corresponding to the target on the display region of the transparent display according to the calculated position and the first identification information of the target.

2. The method of displaying information according to claim 1, further comprising:

estimating a plurality of relative positions of the target in the display region of the transparent display;

displaying a plurality of pieces of prompting information corresponding to the target on the display region of the transparent display according to the relative positions;

judging whether an actual position of the target is overlapped with the relative positions or not;

not displaying the prompting information corresponding to the relative positions if it is judged that the actual position of the target is overlapped with the relative positions.

3. The method of displaying information according to claim 1, further comprising:

capturing another image at a second time point, the another image comprising the target;

identifying the target in the another image, so as to acquire a second relative position of the target;

acquiring second identification information of the target;

estimating a second estimated position and second estimated time of the target that appears in a display region of another transparent display according to the second relative position of the target, the second time point and the relative moving information between the vehicle and the target; and

displaying the prompting information corresponding to the target on the display region of the another transparent display according to the second estimated time, the second estimated position of the target and the second identification information of the target.

4. The method of displaying information according to claim 1, further comprising:

capturing another image at a second time point, the another image comprising the target;

identifying the target in the another image, so as to acquire a second relative position of the target;

estimating a second estimated position and second estimated time of the target that appears in a display region of another transparent display according to the second relative position of the target, the second time point and the relative moving information between the vehicle and the target; and

displaying the prompting information corresponding to the target on the display region of the another transparent display according to the second estimated time, the second estimated position of the target and the first identification information of the target.

5. The method of displaying information according to claim 1, further comprising:

estimating a second estimated position and second estimated time of the target that appears in a display region of another transparent display according to the first relative position of the target, the first time point and the relative moving information between the vehicle and the target; and

displaying the prompting information corresponding to the target on the display region of the another transparent display according to the second estimated time, the second estimated position of the target and the first identification information of the target.

6. The method of displaying information according to claim 1, further comprising:

comparing the first relative position of the target at the first time point with the relative position of the target at another time point, so as to forecast a path of the target.

7. The method of displaying information according to claim 1, further comprising:

acquiring relative positions of the target in a plurality of continuous frames of images; and

respectively comparing the relative position of the target of the previous frame in the plurality of continuous frames of images with the relative position of the target of the next frame, so as to judge a motion mode of the target and forecast a path of the target.

8. A system of displaying information, comprising:

a transparent display, configured at a side of a vehicle;

a first external image capturing device, configured to identify a user in the vehicle;

a speed sensing apparatus;

a processor, coupled to the transparent display, the first external image capturing device and the speed sensing apparatus, and configured to calculate a visual range of the user according to a face position or eye positions of the user,

wherein the first external image capturing device circuit is configured to capture an image at a first time point, and the image comprises a target;

the processor is configured to identify the target in the image;

the first external image capturing device is configured to acquire a first relative position of the target;

the speed sensing apparatus is configured to capture relative moving information between the vehicle and the target;

the processor is configured to estimate a first estimated position and first estimated time of the target that appears in a display region of the transparent display according to the visual range of the user, the first relative position of the target, the first time point and the relative moving information between the vehicle and the target before the target appears on the side of the vehicle;

the processor is further configured to calculate a position of the target that is projected to the display region of the transparent display according to the visual range of the user, the first relative position of the target, the first time point, the first estimated position of the target, the first estimated time and the relative moving information between the vehicle and the target; and

the processor is configured to display prompting information corresponding to the target on the display region of the transparent display according to the calculated position and the first identification information of the target.

9. The system of displaying information according to claim 8, wherein the processor is further configured to estimate a plurality of relative positions of the target in the display region of the transparent display;

the transparent display is further configured to display a plurality of pieces of prompting information corresponding to the target on the display region of the transparent display according to the relative positions;

the processor is further configured to judge whether an actual position of the target is overlapped with the relative positions or not; and

the transparent display is further configured to not display the prompting information corresponding to the relative positions if it is judged that the actual position of the target is overlapped with the relative positions.

10. The system of displaying information according to claim 8, further comprising a second external image capturing device,

wherein the second external image capturing device is configured to capture another image at a second time point, the another image comprising the target;

the processor is further configured to identify the target in the another image, so as to acquire a second relative position of the target;

the processor is further configured to acquire second identification information of the target;

the processor is further configured to estimate a second estimated position and second estimated time of the target that appears in a display region of another transparent display according to the second relative position of the target, the second time point and the relative moving information between the vehicle and the target; and

the processor is further configured to display the prompting information corresponding to the target on the display region of the another transparent display according to the second estimated time, the second estimated position of the target and the second identification information of the target.

11. The system of displaying information according to claim 8, further comprising a second external image capturing device,

wherein the second external image capturing device is configured to capture another image at a second time point, the another image comprising the target;

the processor is further configured to identify the target in the another image, so as to acquire a second relative position of the target;

the processor is further configured to estimate a second estimated position and second estimated time of the target that appears in a display region of another transparent display according to the second relative position of the target, the second time point and the relative moving information between the vehicle and the target; and

the processor is further configured to display the prompting information corresponding to the target on the display region of the another transparent display according to the second estimated time, the second estimated position of the target and the first identification information of the target.

12. The system of displaying information according to claim 8, wherein

the processor is further configured to estimate a second estimated position and second estimated time of the target that appears in a display region of another transparent display according to the first relative position of the target, the first time point and the relative moving information between the vehicle and the target;

and

the processor is further configured to display the prompting information corresponding to the target on the display region of the another transparent display according to the second estimated time, the second estimated position of the target and the first identification information of the target.

13. The system of displaying information according to claim 8, wherein the processor is further configured to compare the first relative position of the target at the first time point with the relative position of the target at another time point, so as to forecast a path of the target.

14. The system of displaying information according to claim 8, wherein

the first external image capturing device is further configured to acquire relative positions of the target in continuous frames of images;

and

the processor is further configured to respectively compare the relative position of the target of the previous frame in the plurality of continuous frames of images with the relative position of the target of the next frame, so as to judge a motion mode of the target and forecast a path of the target.

15. The system of displaying information according to claim 8, wherein the prompting information comprises at least one of an image of the target, a position message, a moving path and time that prompts appearing of the target;

and the image of the target, the position message, the moving path and the time that prompts appearing of the target are expressed by using words or patterns.

16. The system of displaying information according to claim 8, wherein the transparent display is configured on the vehicle;

the speed sensing apparatus is configured to acquire vehicle moving information of the vehicle to capture the relative moving information between the vehicle and the target according to the vehicle moving information; and

the vehicle moving information comprises a relative acceleration, a relative speed and a moving direction.