Mutually-immersive mobile telepresence with gaze and eye contact preservation

• US 20040117067A1
• Filed: 12/14/2002
• Published: 06/17/2004
• Est. Priority Date: 12/14/2002
• Status: Active Grant

• Current Assignee: Baker Manufacturing Company, LLC
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Jouppi, Norman Paul
• Granted Patent: US 6,889,120 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A method and system are provided for user and participant telepresencing including viewing a user from a plurality of views to obtain an image of a user, communicating the image of the user to a surrogate, and displaying the image of the user on the surrogate with eye contact between the user and the participant preserved.

Claims

1. A method for user and participant telepresencing comprising:

viewing the user from a plurality of views to obtain a plurality of user'"'"'s images;

communicating the plurality of user'"'"'s images to a surrogate; and

displaying the plurality of user'"'"'s images on surrogate'"'"'s displays with gaze of the user preserved.

2. The method of telepresencing as claimed in claim 1 additionally comprising:

viewing the participant from the surrogate from at least one of a plurality of views to obtain at least one participant'"'"'s image;

communicating the at least one participant'"'"'s image to a user'"'"'s display; and

displaying the at least one participant'"'"'s image on the user'"'"'s display with gaze of the participant preserved.

3. The method of telepresencing as claimed in claim 1 wherein:

viewing the user includes viewing from an user'"'"'s eye level; and

displaying the plurality of images on the surrogate'"'"'s displays includes having the user'"'"'s eye level proximate center levels of the surrogate'"'"'s displays; and

additionally comprising;

viewing around the surrogate from views proximate the center levels of the surrogate'"'"'s displays to create images of views around the surrogate;

communicating the images of views around the surrogate; and

displaying images of views around the surrogate to the user at the user'"'"'s eye level on the user'"'"'s displays.

4. The method of telepresencing as claimed in claim 1 wherein:

viewing the user from a plurality of views uses a number of views; and

displaying the plurality of user'"'"'s images uses the same number of surrogate'"'"'s displays as the number of views.

5. The method of telepresencing as claimed in claim 1 additionally comprising:

viewing a plurality of views around the surrogate to create a plurality of images from the surrogate;

communicating the plurality of images from the surrogate to a user'"'"'s display;

displaying the plurality of images from the surrogate on the user'"'"'s display; and

displaying the plurality of images from the surrogate on the user'"'"'s display also on the surrogate'"'"'s display.

6. The method of telepresencing as claimed in claim 1 additionally comprising:

providing directional audio between the surrogate and the user.

7. A method of user and participant telepresencing comprising:

viewing a user from a plurality of views to obtain images of a user from 360 degrees around the user;

processing the images of the user to provide cropped video images of the user'"'"'s head;

communicating the cropped video images of the user'"'"'s head to a surrogate; and

displaying the cropped video images on surrogate'"'"'s displays with horizontal gaze of the user preserved.

8. The method of telepresencing as claimed in claim 7 additionally comprising:

viewing 360 degrees around the surrogate, the viewing including at least one view of the participant to obtain at least one participant'"'"'s image; and

communicating the at least one participant'"'"'s image to a user'"'"'s display cube; and

displaying the at least one participant'"'"'s image on the user'"'"'s display cube with horizontal gaze of the participant preserved and user and participant proximate eye contact preserved.

9. The method of telepresencing as claimed in claim 7 wherein:

viewing the user from a plurality of views uses a number of views;

displaying the plurality of user'"'"'s images uses the same number of surrogate'"'"'s displays as the number of views; and

viewing 360 degrees around the surrogate uses a plurality of surrogate views as the number of views.

10. The method of telepresencing as claimed in claim 7 wherein:

viewing the user includes viewing from a first and second heights when the user changes between first and second heights; and

additionally comprising;

viewing the user at the first or the second height when the user is respectively at the first or second height.

11. The method of telepresencing as claimed in claim 7 additionally comprising:

viewing a plurality of views 360 degrees around the surrogate to create a plurality of images from the surrogate;

communicating the plurality of images from the surrogate to a user'"'"'s display cube;

displaying the plurality of images from the surrogate on the user'"'"'s display cube; and

displaying the plurality of images from the surrogate on the user'"'"'s display cube behind the user on the surrogate'"'"'s display.

12. The method of telepresencing as claimed in claim 7 additionally comprising.

viewing a plurality of views 360 degrees around the surrogate to create a plurality of images from the surrogate;

communicating the plurality of images from the surrogate to a user'"'"'s display cube;

displaying the plurality of images life-size; and

transmitting sounds from 360 degrees around the surrogate in directions corresponding to the plurality of views 360 degrees around the user'"'"'s display cube.

13. A user and participant telepresence system comprising:

user'"'"'s cameras for viewing the user from a plurality of views to obtain a plurality of user'"'"'s images;

a communication system for communicating the plurality of images from the user'"'"'s cameras; and

a surrogate in communication with the communication system, the surrogate including surrogate'"'"'s displays for displaying the plurality of images of the user with the gaze of the user preserved.

14. The user and participant telepresence system as claimed in claim 13 wherein:

the number of user'"'"'s cameras equal the number of surrogate'"'"'s displays.

15. The user and participant telepresence system as claimed in claim 13 wherein:

the user'"'"'s cameras are positioned at an user'"'"'s eye level; and

the surrogate'"'"'s displays display the plurality of images of the user with the user'"'"'s eye level proximate center levels of the surrogate'"'"'s displays; and

additionally comprising;

surrogate'"'"'s cameras proximate the center levels of the surrogate'"'"'s displays to obtain images of views around the surrogate;

communicating the images of the views around the surrogate; and

user'"'"'s displays for displaying images of views around the surrogate to the user at the user'"'"'s eye level.

16. The user and participant telepresence system as claimed in claim 13 wherein:

the user'"'"'s cameras include further cameras for determining an eye level of the user; and

the user'"'"'s cameras at a height proximate the eye level of the user obtain the plurality of user'"'"'s images.

17. The user and participant telepresence system as claimed in claim 13 additionally comprising:

a plurality of surrogate'"'"'s cameras for obtaining an image of a participant, the plurality of surrogate'"'"'s cameras view 360 degrees around the surrogate.

surrogate'"'"'s cameras positioned adjacent to the surrogate'"'"'s displays for viewing a plurality of views around the surrogate to create a plurality of images from the surrogate;

a communication system for communicating the plurality of images from the surrogate;

user'"'"'s displays in communication with the communication system, the users displays for displaying the plurality of images from the surrogate on the user'"'"'s display; and

wherein;

the surrogates displays display the plurality of images from the surrogate on the user'"'"'s display also on the surrogate'"'"'s displays.

18. The user and participant telepresence system as claimed in claim 13 additionally comprising:

a plurality of microphones and speakers on the surrogate and around the user to obtain directional audio between the user and the participant.

19. A user and participant telepresence system comprising:

user'"'"'s cameras for viewing a user from a plurality of views to obtain images of a user from 360 degrees around the user;

a processing system for processing the images of the user to provide cropped video images of the user'"'"'s head;

a communication system for communicating the cropped video images of the user'"'"'s head; and

a surrogate in communication with the communication system, the surrogate having surrogate'"'"'s displays for displaying the cropped video image with horizontal gaze of the user preserved.

20. The user and participant telepresence system as claimed in claim 19 additionally comprising:

surrogate'"'"'s cameras for viewing 360 degrees around the surrogate and including at least one view of the participant to obtain at least one participant'"'"'s image; and

a further communication system for communicating the at least one participant'"'"'s image;

a user'"'"'s display cube for displaying the at least one participant'"'"'s image with the horizontal gaze of the participant preserved and user and participant proximate eye contact preserved.

21. The user and participant telepresence system as claimed in claim 19 wherein:

the number of user'"'"'s cameras equal the number of surrogate'"'"'s displays; and

additionally comprising;

a plurality of surrogate'"'"'s cameras, the number of surrogate'"'"'s cameras equal the number of user'"'"'s cameras.

22. The user and participant telepresence system as claimed in claim 19 additionally comprising:

surrogate'"'"'s cameras for viewing a plurality of views 360 degrees around the surrogate to create a plurality of images from the surrogate;

a further communication system for communicating the plurality of images from the surrogate;

a user'"'"'s display cube for displaying the plurality of images from the surrogate; and

wherein;

the surrogate'"'"'s displays display the plurality of images from the surrogate on the user'"'"'s display cube behind the user on the surrogate'"'"'s display.

23. The user and participant telepresence system as claimed in claim 19 additionally comprising:

surrogate'"'"'s cameras for viewing a plurality of views 360 degrees around the surrogate to create a plurality of images from the surrogate;

microphones for receiving sounds from 360 degrees around the surrogate;

a further communication system for communicating the plurality of images and sounds from the surrogate;

a display cube for viewing the plurality of images and hearing the sounds from the surrogate in directions corresponding to the plurality of views 360 degrees around the user'"'"'s display cube.