DISPLAY SYSTEM FOR DISPLAYING AUGMENTED REALITY IMAGE AND CONTROL METHOD FOR THE SAME

• US 20150234508A1
• Filed: 03/24/2014
• Published: 08/20/2015
• Est. Priority Date: 02/17/2014
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: CHO, Eunhyung, KIM, Jongho, LEE, Doyoung, KIM, Jihwan, CHUN, Sinae
• Granted Patent: US 9,176,618 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04102 Flexible digitiser, i.e. constructional details for allowing the whole digitising part of a device to be flexed or rolled like a sheet of paper
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

The display system displays, at a first position on the transparent flexible display unit, a first augmented reality object corresponding to a first real object forming a first angle relative to the camera unit, and displays, at a second position on the transparent flexible display unit, a second augmented reality object corresponding to a second real object forming a second angle relative to the camera unit, wherein, upon detecting bending of the transparent flexible display unit, the processor displays the first augmented reality object at a third position spaced a first distance apart from the first position, and displays the second augmented reality object at a fourth position spaced a second distance apart from the second position, wherein, when the first angle is greater than the second angle, the first distance is set to be greater than the second distance.

Claims

1. A display system comprising:

a camera unit configured to detect a real object positioned within a viewing angle;

a transparent flexible display unit configured to display an augmented reality object based on the real object;

a bending sensor unit configured to detect bending of the transparent flexible display unit;

a processor configured to control the camera unit, the transparent flexible display unit and the bending sensor unit,

wherein the processor is further configured to;

display a first augmented reality object corresponding to a first real object forming a first angle relative to the camera unit at a first position on the transparent flexible display unit, and

display a second augmented reality object corresponding to a second real object forming a second angle relative to the camera unit at a second position on the transparent flexible display unit,

wherein, if the bending of the transparent flexible display unit is detected, display the first augmented reality object at a third position spaced a first distance apart from the first position, and display the second augmented reality object at a fourth position spaced a second distance apart from the second position,

wherein, when the first angle is greater than the second angle, the first distance is set to be greater than the second distance.

2. The display system according to claim 1, wherein:

the first position is determined based on a point of intersection between a line of gaze of a user viewing the first real object and the transparent flexible display unit; and

the second position is determined based on a point of intersection between a line of gaze of the user viewing the second real object and the transparent flexible display unit.

3. The display system according to claim 1, wherein the processor is further configured to determine the third position and the fourth position based on an angle of the bending of the transparent flexible display unit.

4. The display system according to claim 3, wherein:

the third position is determined based on a point of intersection between a line of gaze of a user viewing the first real object and the bent transparent flexible display unit; and

the fourth position is determined based on a point of intersection between a line of gaze of the user viewing the second real object and the bent transparent flexible display unit.

5. The display system according to claim 3, wherein the processor is further configured to:

when the angle of the bending of the transparent flexible display unit increases, increase the first distance and the second distance based on the angle of the bending.

6. The display system according to claim 1, further comprising a sensor unit configured to detect an angle of the real object relative to the camera unit,

wherein the camera unit is positioned on a vertical axis passing through a center point of the transparent flexible display unit,

wherein the first angle and the second angle are determined based on the center point of the camera unit.

7. The display system according to claim 6, wherein the first angle and the second angle are determined further based on a line connecting the center point of the transparent flexible display unit with the line of gaze of the user.

8. The display system according to claim 1, wherein:

the first angle is further determined based on a center point between the user'"'"'s left eye and right eye and a position of the first real object; and

the second angle is further determined based on center point between the user'"'"'s left eye and right eye and a position of the second real object.

9. The display system according to claim 8, wherein the processor is further configured to:

when the first augmented reality object is a three-dimensional object, determine the first position and the third position based on a distance and angle of the first real object relative to the user'"'"'s left eye and a distance and angle of the first real object relative to the user'"'"'s right eye.

10. The display system according to claim 6, wherein the sensor unit further detects a distance and angle of the transparent flexible display unit relative to a user'"'"'s eyes,

wherein the processor is further configured to determine the third position and the fourth position based on at least one of the distance and the angle of the transparent flexible display unit relative to the user'"'"'s eyes.

11. The display system according to claim 10, wherein:

the third position is located outside the first position when the distance of the transparent flexible display unit relative to the user'"'"'s eyes is greater than or equal to a threshold distance; and

the third position is located at inside the first position when the distance of the transparent flexible display unit relative to the user'"'"'s eyes is less than the threshold distance,

wherein the threshold distance is determined based on the first angle.

12. The display system according to claim 11, wherein the processor is further configured to determine the inside position and the outside position based on the center point of the transparent flexible display unit,

wherein the inside position is a position shifted closer to the center point of the transparent flexible display unit, and the outside position is a position shifted closer to an edge of the transparent flexible display unit.

13. The display system according to claim 2, wherein the processor is further configured to:

when the point of intersection between the line of gaze of the user viewing the first real object and the transparent flexible display unit does not exist, not display the first augmented reality object corresponding to the first real object.

14. The display system according to claim 13, wherein the processor is further configured to:

if the bending of the transparent flexible display unit is detected, display the first augmented reality object corresponding to the first real object at the point of intersection when the point of intersection between the line of gaze of the user viewing the first real object and the transparent flexible display unit exists.

15. The display system according to claim 1, wherein, when a distance between a user'"'"'s eyes and the transparent flexible display unit is within a threshold distance, the angle of the user'"'"'s field of view is greater than the viewing angle,

wherein the processor is further configured to:

when the first real object is positioned outside of the viewing angle and within the angle of the user'"'"'s field of view, not display the first augmented reality object corresponding to the first real object.

16. The display system according to claim 15, wherein the processor is further configured to:

display an indicator on the transparent flexible display unit based on the viewing angle,

wherein the indicator indicates a limit of a region for display of the augmented reality object on the transparent flexible display unit.

17. The display system according to claim 16, wherein the indicator is displayed based on an angle of bending of the transparent flexible display unit.

18. The display system according to claim 1,

comprising a head mounted display (HMD), wherein, the HMD includes the camera unit and the bending sensor unit;

and

comprising a display unit, wherein, the display unit includes the transparent flexible display unit.

19. The display system according to claim 18, wherein:

the first angle is an angle formed relative to the camera unit of the HMD by the first real object; and

the second angle is an angle formed relative to the camera unit of the HMD by the second real object.

20. A control method for a display system comprising:

detecting a real object,

wherein the detecting the real object comprises;

detecting a first real object forming a first angle relative to a user'"'"'s eyes and detecting a second real object forming a second angle relative to the user'"'"'s eyes,

displaying an augmented reality object corresponding to the real object on a transparent flexible display unit,

wherein the displaying the augmented reality object corresponding to the real object on a transparent flexible display unit comprises;

displaying a first augmented reality object corresponding to the first real object at a first position on the transparent flexible display unit, and

displaying a second augmented reality object corresponding to the second real object at a second position on the transparent flexible display unit;

detecting bending of the transparent flexible display unit; and

displaying the augmented reality object on the bent transparent flexible display unit,

wherein the displaying of the augmented reality object comprises;

displaying the first augmented reality object at a third position spaced a first distance apart from the first position based on the bending and displaying the second augmented reality object at a fourth position spaced a second distance apart from the second position based on the bending,

wherein, when the first angle is greater than the second angle, the first distance is set to be greater than the second distance.