APPARATUS AND METHOD FOR GENERATING TELEPRESENCE

• US 20150138301A1
• Filed: 11/20/2014
• Published: 05/21/2015
• Est. Priority Date: 11/21/2013
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KIM, Yong-Wan, JO, Dong-Sik, KIM, Hye-Mi, KIM, Jin-Ho, KIM, Ki-Hong
• Application Number:
  US14/548,801
• Publication Number:
  US 20150138301A1
• Time in Patent Office: Days
• US Class Current: 348/14.04
• CPC Class Codes: G06F 2203/012 Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

An apparatus and method for generating telepresence are disclosed. The apparatus for generating telepresence includes a visual information sensing unit that is mounted on a robot which is located in a remote location, and senses visual information corresponding to a view of the robot, a tactile information sensing unit that is mounted on the robot, and senses tactile information in the remote location, an environmental information sensing unit that is mounted on the robot, and senses environmental information which is information for a physical environment of the remote location, a robot communication unit that receives movement information for movement performed by a user who is located in a space separated from the remote location in correspondence to the visual information, the tactile information, and the environmental information, and a robot control unit that drives the robot based on the movement information.

Claims

1. An apparatus for generating telepresence comprising:

a visual information sensing unit mounted on a robot located in a remote location, and configured to sense visual information corresponding to a view of the robot;

a tactile information sensing unit mounted on the robot, and configured to sense tactile information in the remote location;

an environmental information sensing unit mounted on the robot, and configured to sense environmental information which is information for a physical environment of the remote location;

a robot communication unit configured to receive movement information for movement performed by a user who is located in a space separated from the remote location in correspondence to the visual information, the tactile information, and the environmental information; and

a robot control unit configured to drive the robot based on the movement information.

2. The apparatus of claim 1, wherein the visual information sensing unit is a stereoscopic camera having a wide angle, which stereoscopically captures images of the remote location at a wide angle by controlling a viewing direction of the robot in correspondence to a movement direction of the user'"'"'s head.

3. The apparatus of claim 1, wherein the environmental information sensing unit senses the environmental information including at least one of winds, sounds, smells, and smoke in the remote location.

4. The apparatus of claim 1, wherein the robot communication unit receives the movement information from a movement information sensing unit that senses the movement performed by the user in correspondence to the visual information, the tactile information, and the environmental information.

5. The apparatus of claim 4, wherein the movement information sensing unit comprises a gyro-based sensing unit for sensing the movement information through at least one of gyro sensors which are mounted on a body of the user.

6. The apparatus of claim 4, wherein the movement information sensing unit comprises a camera-based sensing unit for sensing the movement information through an infrared camera or a depth camera located in the vicinity of the user.

7. The apparatus of claim 4, wherein the movement information sensing unit comprises an interface unit for acquiring a space for movement of feet of the user through a treadmill located beneath the feet of the user.

8. The apparatus of claim 1, wherein the robot communication unit transmits the visual information, the tactile information, and the environmental information.

9. An apparatus for generating telepresence comprising:

a visual information acquisition unit worn by a user, and configured to acquire visual information corresponding to a view of a robot from the robot which is located in a remote location separated from a place where the user is present;

a tactile information acquisition unit worn by a user, and configured to acquire tactile information in the remote location from the robot; and

an environmental information acquisition unit configured to be present in a space where the user is located, and to acquire environmental information which is information for a physical environment of the remote location from the robot,

wherein the user understands a status of the remote location based on the visual information, the tactile information, and the environmental information.

10. The apparatus of claim 9, further comprising:

a fusion information generation unit configured to generate fusion information which is acquired by fusing at least one of the visual information, the tactile information, and the environmental information with virtual information generated based on a simulated virtual object,

wherein the user understands the status of the remote location based on the fusion information.

11. The apparatus of claim 9, wherein the visual information acquisition unit is a wide field-of-view Head Mounted Display (HMD).

12. The apparatus of claim 9, wherein the visual information acquisition unit transmits movement information for a movement direction of a user'"'"'s head, thus allowing the robot to control a viewing direction of the robot based on the movement information.

13. The apparatus of claim 9, wherein the environmental information acquisition unit acquires the environmental information including at least one of winds, sounds, smells, and smoke in the remote location.

14. A method for generating telepresence comprising:

sensing, by a visual information sensing unit, visual information corresponding to a view of a robot which is located in a remote location;

sensing, by a tactile information sensing unit, tactile information in the remote location;

sensing, by an environmental information sensing unit, environmental information which is information for a physical environment of the remote location;

receiving, by a robot communication unit, movement information for movement performed by a user who is located in a space separated from the remote location in correspondence to the visual information, the tactile information, and the environmental information; and

driving, by a robot control unit, the robot based on the movement information.

15. The method of claim 14, wherein sensing the visual information comprises stereoscopically capturing images of the remote location at a wide angle by controlling a viewing direction of the robot in correspondence to a movement direction of the user'"'"'s head.

16. The method of claim 14, wherein sensing the environmental information comprises sensing the environmental information including at least one of winds, sounds, smells, and smoke in the remote location.

17. The method of claim 14, wherein receiving the movement information comprises receiving, by the robot communication unit, the movement information from a movement information sensing unit that senses the movement performed by the user in correspondence to the visual information, the tactile information, and the environmental information.

18. The method of claim 17, wherein receiving the movement information comprises receiving the movement information from the movement information sensing unit that includes a gyro-based sensing unit for sensing the movement information through at least one of gyro sensors mounted on a body of the user.

19. The method of claim 17, wherein receiving the movement information comprises receiving the movement information from the movement information sensing unit that includes a camera-based sensing unit for sensing the movement information through an infrared camera or a depth camera located in the vicinity of the user.

20. The method of claim 14, wherein receiving the movement information comprises receiving the movement information from the movement information sensing unit that includes an interface unit for acquiring a space for movement of feet of the user through a treadmill located beneath the feet of the user.