Method for remotely controlling virtual content and apparatus for the same

• US 10,599,213 B2
• Filed: 05/16/2018
• Issued: 03/24/2020
• Est. Priority Date: 06/09/2017
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Yang, Ung-Yeon, Kim, Ki-Hong, Kim, Jin-Ho
• Primary Examiner(s): Cheng, Joe H
• Application Number:
  US15/981,526
• Publication Number:
  US 20180356879A1
• Time in Patent Office: 678 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

Disclosed herein are a method for remotely controlling virtual content and an apparatus for the method. The method for remotely controlling virtual content includes acquiring spatial data about a virtual space, creating at least one individual space by transforming the virtual space in accordance with a user interaction area that corresponds to a user based on the spatial data, visualizing the at least one individual space in the user interaction area and providing an interactive environment which enables an interaction between the user'"'"'s body and a virtual object included in the at least one individual space, and controlling the virtual object in response to a user interaction event occurring in the interactive environment.

Claims

1. A method for remotely controlling virtual content, comprising:

acquiring spatial data about a virtual space;

creating at least one individual space by transforming the virtual space in accordance with a user interaction area that corresponds to a user based on the spatial data;

visualizing the at least one individual space in the user interaction area and providing an interactive environment which enables an interaction between the user'"'"'s body and a virtual object included in the at least one individual space; and

controlling the virtual object in response to a user interaction event occurring in the interactive environment,

wherein providing the interactive environment comprises comparing a size of the user'"'"'s body with a size of the virtual object and if it is determined that a virtual interaction tool is required as a result of comparing the size of the user'"'"'s body with the size of the virtual object, providing a virtual interaction tool corresponding to the size of the virtual object to the user interaction area.

2. The method of claim 1, wherein the user interaction area is an intersection area between a user-workable area in which the user'"'"'s body is taken into consideration and a displayable range of a wearable display worn by the user.

3. The method of claim 2, wherein providing the interactive environment is configured to cause the at least one individual space to be included in the user-workable area and visualize a portion of the at least one individual space corresponding to the intersection area.

4. The method of claim 3, wherein providing the interactive environment comprises:

detecting a line of sight of the user based on a scheme corresponding to at least one of head tracking and eye tracking; and

sensing the displayable range based on the line of sight, and when the displayable range is changed, extracting the intersection area in real time in accordance with the changed displayable range.

5. The method of claim 2, wherein the user-workable area is generated in consideration of at least one of body size information of the user, motion information of the user collected for a preset period, and a body-reachable range measured in accordance with the user.

6. The method of claim 1, wherein providing the interactive environment is configured to provide the interactive environment based on a multimodal interaction scheme using the user'"'"'s body.

7. The method of claim 1, wherein providing the interactive environment further comprises, if it is determined that adjustment of a size of the at least one individual space is required as a result of comparing the size of the user'"'"'s body with the size of the virtual object, adjusting the size of the at least one individual space in accordance with the size of the user'"'"'s body.

8. The method of claim 1, wherein creating the at least one individual space is configured to create the at least one individual space by transforming the virtual space in accordance with at least one of rotation, translation, scale conversion, and projection into N dimensions.

9. The method of claim 1, wherein creating the at least one individual space comprises:

rearranging the virtual space into multiple division groups based on a scenario of the virtual content; and

creating the at least one individual space by merging the multiple division groups in a sequence of rearrangement.

10. The method of claim 1, wherein providing the interactive environment further comprises outputting an event effect that occurs due to the virtual content to the user based on at least one output module.

11. The method of claim 1, wherein the virtual space is at least one of a virtual space defined as a three-dimensional (3D) space and a logical virtual space defined as a multidimensional space having three or more dimensions.

12. The method of claim 1, wherein acquiring the spatial data is configured to acquire the spatial data based on any one of Internet Process Communication (IPC) and Remote Process Communication (RPC).

13. An apparatus for remotely controlling virtual content, comprising:

a communication unit for acquiring spatial data about a virtual space;

a transform unit for creating at least one individual space by transforming the virtual space in accordance with a user interaction area that corresponds to a user based on the spatial data;

a processing unit for visualizing the at least one individual space in the user interaction area and providing an interactive environment which enables an interaction between the user'"'"'s body and a virtual object included in the at least one individual space; and

a control unit for controlling the virtual object in response to a user interaction event occurring in the interactive environment,

wherein the processing unit is configured to compare a size of the user'"'"'s body with a size of the virtual object, and if it is determined that a virtual interaction tool is required as a result of comparing the size of the user'"'"'s body with the size of the virtual object, provide a virtual interaction tool corresponding to the size of the virtual object to the user interaction area.

14. The apparatus of claim 13, wherein the user interaction area is an intersection area between a user-workable area in which the user'"'"'s body is taken into consideration and a displayable range of a wearable display worn by the user.

15. The apparatus of claim 14, wherein the processing unit causes the at least one individual space to be included in the user-workable area and visualizes a portion of the at least one individual space corresponding to the intersection area.

16. The apparatus of claim 13, wherein the processing unit provides the interactive environment based on a multimodal interaction scheme using the user'"'"'s body.

17. The apparatus of claim 13, wherein the processing unit is configured to, if it is determined that adjustment of a size of the at least one individual space is required as a result of comparing the size of the user'"'"'s body with the size of the virtual object, adjust the size of the at least one individual space in accordance with the size of the user'"'"'s body.

18. The apparatus of claim 13, wherein the transformation unit creates the at least one individual space by transforming the virtual space in accordance with at least one of rotation, translation, scale conversion, and projection into N dimensions.