METHOD FOR PROVIDING TELEPRESENCE USING AVATARS, AND SYSTEM AND COMPUTER-READABLE RECORDING MEDIUM USING THE SAME

• US 20160205352A1
• Filed: 11/30/2015
• Published: 07/14/2016
• Est. Priority Date: 01/09/2015
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Center of Human-centered Interaction for Coexistence (Government of South Korea), Korea Advanced Institute of Science and Technology
• Inventors: Lee, Sung Hee, You, Bum Jae
• Granted Patent: US 9,843,772 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06V 10/759 Region-based matching
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

A method for providing telepresence by employing avatars is provided. The method includes steps of: (a) a corresponding location searching part determining a location in a first space where an avatar Y′ corresponding to a human Y in a second space will be placed, if a change of a location of the human Y in the second space is detected from an initial state, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′ in the first and the second spaces; and (b) an avatar motion creating part creating a motion of the avatar Y′ by referring to information on the determined location where the avatar Y′ will be placed.

Claims

1. A telepresence system by using avatars, comprising:

a corresponding location searching part for determining a location in a first space as a present space where an avatar Y′

corresponding to a human Y in a second space as a remote place will be placed, if a change of a location of the human Y in the second space is detected from an initial state in which an initial location of a human X in the first space, that of the human Y in the second space, that of the avatar Y′

in the first space corresponding to the human Y, and that of an avatar X′

in the second space corresponding to the human X are determined, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′

in the first and the second spaces; and

an avatar motion creating part for creating a motion of the avatar Y′

by referring to information on the determined location where the avatar Y′

will be placed.

2. The system of claim 1, wherein the corresponding location searching part determines the location in the first space where the avatar Y′

will be placed by further referring to (iii) information on gazes of the humans X and Y in the first and the second spaces.

3. The system of claim 2, wherein the corresponding location searching part estimates a destination where the human Y intends to reach in the second space by further referring to (iii) the information on gazes of the humans X and Y in the first and the second spaces and then determines the location in the first space where the avatar Y′

will be placed by referring to the estimated destination.

4. The system of claim 3, wherein the corresponding location searching part estimates the destination in a unit of patch by dividing the first and the second spaces into patches, and determines the location where the avatar Y′

will be placed.

5. The system of claim 4, wherein the first and the second spaces are divided into patches in a unit of plane and a plane not more than a predetermined minimum area is excluded from the patch and a plane not less than a predetermined maximum area is subdivided into multiple patches.

6. The system of claim 4, wherein the corresponding location searching part determines the location in the first space where the avatar Y′

will be placed by referring to at least some of information on types of objects corresponding to the patches included in the first and the second spaces, information on geometrical characteristics of the patches, and information on a degree of spatial similarity between the first and the second spaces.

7. The system of claim 6, wherein the information on the geometrical characteristics of the patches includes at least some of sizes, heights, normal vectors, and aspect ratios thereof.

8. The system of claim 6, wherein the corresponding location searching part acquires the information on the degree of spatial similarity by referring to information on a relative location and a relative direction between the human X and the avatar Y′

in the first space and those between the human Y and the avatar X′

in the second space.

9. The system of claim 1, wherein the corresponding location searching part determines a destination in the second space where the human Y intends to reach from the initial state by referring to (1) information on a function which increases as cumulative values of respective probabilities of gazing at the respective patches during a certain time section increase, (2) information on a function for evaluating degrees of appropriateness of locations for the human Y to be located by considering types of objects to which the respective patches correspond and geometrical characteristics of the respective patches, and (3) information on a function which increases as frequencies of the respective patches being the destination actually reached by the human Y become higher.

10. The system of claim 9, wherein the corresponding location searching part provides the human Y with information on the determined destination and offers user interface capable of receiving feedback from the human Y about whether the determined destination is appropriate.

11. The system of claim 2, further comprising:

a human gaze recognition part for acquiring information on gazes of the humans X and Y in the first and the second spaces.

12. The system of claim 1, further comprising:

a space recognition part for acquiring information on the first and the second spaces.

13. The system of claim 12, wherein the space recognition part recognizes at least one of types, locations, and orientations of objects existing in the first and the second spaces.

14. The system of claim 1, further comprising:

a human motion recognition part for acquiring information on a human'"'"'s location by using a sensor.

15. The system of claim 1, wherein, if the human X exists in a specified direction at a specified location of the first space and the human Y exists in a specific direction at a specific location of the second space, the corresponding location searching part allows the avatar Y′

to be placed in the first space to have a direction of 180 degrees (or near 180 degrees) from a direction of the human X and have a location within a predetermined distance from the human X and allows the avatar X′

to be placed in the second space to have a direction of 180 degrees (or near 180 degrees) from a direction of the human Y and have a location within a preset distance from the human Y.

16. The system of claim 15, wherein, if the first and the second spaces are divided into multiple patches, the corresponding location searching part allows the avatar Y′

to be placed in a patch with a high degree of appropriateness for a human to be located in the first space while having the direction of 180 degrees (or near 180 degrees) from a direction of the human X and having a location within the predetermined distance from the human X and allows the avatar X′

to be placed in a patch with a high degree of appropriateness for a human to be located in the second space while having the direction of 180 degrees (or near 180 degrees) from a direction of the human Y and having a location within the preset distance from the human Y.

17. The system of claim 1, wherein the avatar motion creating part creates a motion of the avatar Y′

by searching a route to avoid interpenetration in the first space.

18. The system of claim 17, wherein the avatar motion creating part creates the motion of the avatar Y′

by referring to (i) a degree of difference between a pose of the human Y and that of the avatar Y′ and

(ii) a degree of penetration of the avatar Y′

into an environment of the first space.

19. A method for providing telepresence by employing avatars, comprising steps of:

(a) a corresponding location searching part determining a location in a first space as a present space where an avatar Y′

corresponding to a human Y in a second space as a remote place will be placed, if a change of a location of the human Y in the second space is detected from an initial state in which an initial location of a human X in the first space, that of the human Y in the second space, that of the avatar Y′

in the first space corresponding to the human Y, and that of an avatar X′

in the second space corresponding to the human X are determined, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′

in the first and the second spaces; and

(b) an avatar motion creating part creating a motion of the avatar Y′

by referring to information on the determined location where the avatar Y′

will be placed.

20. The method of claim 19, wherein, at the step of (a), the location in the first space where an avatar Y′

will be placed is determined by further referring to (iii) information on gazes of the humans X and Y in the first and the second spaces.

21. The method of claim 20, wherein, at the step of (a), a destination where the human Y intends to reach in the second space is estimated by further referring to (iii) the information on gazes of the humans X and Y in the first and the second spaces and then the location in the first space where the avatar Y′

will be placed is determined by referring to the estimated destination.

22. The method of claim 21, wherein, at the step of (a), the destination in a unit of patch is estimated by dividing the first and the second spaces into patches, and the location where the avatar Y′

will be placed is determined.

23. The method of claim 21, wherein, at the step of (a), the location in the first space where the avatar Y′

will be placed is determined by referring to at least some of information on types of objects corresponding to the patches included in the first and the second spaces, information on geometrical characteristics of the patches, and information on a degree of spatial similarity between the first and the second spaces.

24. The method of claim 19, wherein, at the step of (a), wherein, a destination in the second space where the human Y intends to reach from the initial state is determined by referring to (1) information on a function which increases as cumulative values of respective probabilities of gazing at the respective patches during a certain time section increase, (2) information on a function for evaluating degrees of appropriateness of locations for the human Y to be located by considering types of objects to which the respective patches correspond and geometrical characteristics of the respective patches, and (3) information on a function which increases as frequencies of the respective patches being the destination actually reached by the human Y become higher.

25. The method of claim 19, wherein, at the step of (a), if the human X exists in a specified direction at a specified location of the first space and the human Y exists in a specific direction at a specific location of the second space, the avatar Y′

is allowed to be placed in the first space to have a direction of 180 degrees (or near 180 degrees) from a direction of the human X and have a location within a predetermined distance from the human X and the avatar X′

is allowed to be placed in the second space to have a direction of 180 degrees (or near 180 degrees) from a direction of the human Y and have a location within a preset distance from the human Y.

26. The method of claim 19, wherein, at the step of (b), a motion of the avatar Y′

is created by searching a route to avoid interpenetration in the first space.

27. The method of claim 26, wherein, at the step of (b), the motion of the avatar Y′

is created by referring to (i) a degree of difference between a pose of the human Y and that of the avatar Y′ and

(ii) a degree of penetration of the avatar Y′

into an environment of the first space.

28. One or more computer-readable recording media having stored thereon a computer program that, when executed by one or more processors, causes the one or more processors to perform acts including:

(a) a corresponding location searching part determining a location in a first space as a present space where an avatar Y′

corresponding to a human Y in a second space as a remote place will be placed, if a change of a location of the human Y in the second space is detected from an initial state in which an initial location of a human X in the first space, that of the human Y in the second space, that of the avatar Y′

in the first space corresponding to the human Y, and that of an avatar X′

in the second space corresponding to the human X are determined, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′

in the first and the second spaces; and

(b) an avatar motion creating part creating a motion of the avatar Y′

by referring to information on the determined location where the avatar Y′

will be placed.