MIXED REALITY DISPLAY PLATFORM FOR PRESENTING AUGMENTED 3D STEREO IMAGE AND OPERATION METHOD THEREOF

• US 20120146894A1
• Filed: 12/09/2011
• Published: 06/14/2012
• Est. Priority Date: 12/09/2010
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: YANG, Ung-Yeon, LEE, Gun A., JO, Dong-Sik, KIM, Ki-Hong, KIM, YONG WAN
• Application Number:
  US13/315,815
• Publication Number:
  US 20120146894A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/373 for tracking forward-backward translational head movements, i.e. longitudinal movements
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

Various 3D image display devices divide and share a physical space for expressing a 3D image, and real-time contents information is generated based on user information and information on the divided space and displayed together using various 3D image display devices to present a 3D image naturally in a deeper, wider, and higher space. A mixed reality display platform includes an input/output controller controlling display devices including 3D display devices, an advance information manager establishing 3D expression space for each display device to divide or share a physical space by collecting spatial establishment of the display device, and a real-time information controller generating real-time contents information using user information and 3D contents for a virtual space. The input/output controller distributes the real-time contents information to each display device based on the 3D expression spatial information.

Claims

1. A mixed reality display platform for presenting an augmented 3D stereo image, comprising:

an input/output controller controlling a plurality of display devices including at least one 3D display device, which are associated with each other;

an advance information manager establishing a 3D expression space for each display device to divide or share a physical space for expressing a 3D stereo image for each display device by collecting spatial establishment of the display device; and

a real-time information controller generating real-time contents information using user information including a user'"'"'s gaze information and 3D contents for a virtual space,

wherein the input/output controller distributes the real-time contents information to each display device on the basis of the 3D expression spatial information established for each display device and the user information.

2. The mixed reality display platform for presenting an augmented 3D stereo image of claim 1, wherein when the user is multiple,

the real-time information controller generates the real-time contents information by further using the interrelationship of the plurality of users, and

the input/output controller distributes the real-time contents information to the display device on the basis of the 3D expression spatial information, the user information, and the interrelationship of the plurality of users.

3. The mixed reality display platform for presenting an augmented 3D stereo image of claim 1, further comprising:

an input sensor device installed in a physical space around the user and another information outputting device other than the display device,

wherein the advance information manager further collects installation status information of the input sensor device and the information outputting device.

4. The mixed reality display platform for presenting an augmented 3D stereo image of claim 1, wherein the advance information manager authors information on the established physical space and information on the virtual space in an interplacement relationship in a 3D space using the 3D expression spatial information and 3D contents for the virtual space and stores the authorizing result as optimal 3D expression space establishing information for each display device.

5. The mixed reality display platform for presenting an augmented 3D stereo image of claim 4, wherein the authoring is performed by the user or automatically performed from space establishment of the display device collected by the advance information manager.

6. The mixed reality display platform for presenting an augmented 3D stereo image of claim 1, wherein the real-time information controller modifies the 3D expression spatial information on the basis of personal information of the user which is collected in advance.

7. The mixed reality display platform for presenting an augmented 3D stereo image of claim 1, wherein the 3D expression spatial information for each display device is established to have an overlapped area with each other.

8. The mixed reality display platform for presenting an augmented 3D stereo image of claim 1, wherein the advance information manager includes:

a space establishment collecting unit collecting space establishment of the display device;

a virtual space 3D contents database storing 3D contents for the virtual space;

an authoring unit authoring information of the established physical space and the information of the virtual space as an interplacement relationship in a 3D space; and

an optimal space establishment information database storing the authoring result.

9. The mixed reality display platform for presenting an augmented 3D stereo image of claim 8, wherein the real-time information controller includes:

a user information extracting unit extracting the user information;

a multi-user participation supporting unit managing an interrelationship of a plurality of users when the user is multiple;

a real-time contents information generating unit generating the real-time contents information on the basis of 3D contents for the user information, the interrelationship of the plurality of users, and the virtual space; and

a user adaptive device and image parameter controlling unit managing the user information and modifying the optimal 3D expression space establishment information for each display device on the basis of the personal information of the user which is collected in advance.

10. The mixed reality display platform for presenting an augmented 3D stereo image of claim 9, wherein the input/output controller distributes the real-time contents information to the display device on the basis of the optimal 3D expression space establishment information for each display device modified by the user adaptive device and image parameter controlling unit, the user information, and the interrelationship of the plurality of users.

11. A mixed reality display platform for presenting an augmented 3D stereo image, comprising:

an input/output controller controlling a plurality of display devices including at least one 3D display devices, which are associated with each other;

an advance information manager including a space establishment collecting unit collecting information on an optimal 3D space which is expressible by the display device, a virtual space 3D contents database storing 3D contents for the virtual space, an authoring unit authoring information of a physical space collected by the space establishment collecting unit and information of the virtual space as an interplacement relationship in a 3D space, and an optimal space establishment information database storing the authoring result as optimal 3D expression space establishment information for each display device; and

a real-time information controller including a user information extracting unit extracting user information, a multi-user participation supporting unit managing an interrelationship of a plurality of users when the user is multiple, a real-time contents information generating unit generating real-time contents information on the basis of the user information, the interrelationship of the plurality of users, and the 3D contents for the virtual space, and a user adaptive device and image parameter controlling unit managing the user information and modifying optimal 3D expression space establishment information for each display device on the basis of personal information of the user which is collected in advance.

12. The mixed reality display platform for presenting an augmented 3D stereo image of claim 11, wherein the user information includes binocular 6 degree-of-freedom information, a gaze direction, and a focusing direction of the user, and information on an input/output device including the display device which interacts with the user.

13. The mixed reality display platform for presenting an augmented 3D stereo image of claim 11, wherein the input/output controller distributes the real-time contents information to the display device on the basis of the optimal 3D expression space establishment information for each display device modified by the user adaptive device and image parameter controlling unit, the user information, and the interrelationship of the plurality of users.

14. An operation method of a mixed reality display platform for presenting an augmented 3D stereo image, comprising:

collecting information on an optimal 3D space which is expressible from a plurality of display devices including at least one 3D display device;

establishing a 3D expression space for each display device to divide or share a physical space for expressing a 3D stereo image for each display device on the basis of the collected information on the optimal 3D space;

collecting user information including binocular 6 degree-of-freedom information, a gaze direction, and focusing information of a user;

generating real-time contents information using 3D contents for a virtual space and the user information; and

distributing the real-time contents information to each display device on the basis of the user information and the 3D expression spatial information established for each display device.

15. The method of claim 14, wherein:

the real-time contents information includes information for virtual 3D contents, and

further comprising displaying the virtual 3D contents in the virtual space for each display device after the distributing.

16. The method of claim 15, wherein:

the plurality of display devices include at least one portable 3D display device assigned for each of two or more of the users, and

the two or more users aquire 3D images through the portable 3D display device assigned for themselves.

17. The method of claim 15, wherein the plurality of display devices include two or more 3D display device including a portable 3D display device of the user, thereby the virtual object is diplayed through display regions of the two or more 3D display devices.

20. The method of claim 14, further comprising:

including interactive advertisement contents, which reacts to the real-time contents information in real time, to the real-time contents information for the virtual space, and

demonstrating virtual simulation using input and output sensors included in the mixed reality display platform for presenting an augmented 3D stereo image and knowledge database.

18. The method of claim 18, further comprising including contents of a 3D virtual space around the user, which is owned and controlled by the user, to the real-time contents information for the virtual space, thereby the contents of a 3D virtual space around the user is displayed to another user.

19. The method of claim 18, wherein the contents of a 3D virtual space around the user includes UCC (User Created Contents).