Telepresence system with simultaneous automatic preservation of user height, perspective, and vertical gaze

• US 7,593,546 B2
• Filed: 03/11/2003
• Issued: 09/22/2009
• Est. Priority Date: 03/11/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Jouppi, Norman Paul
• Primary Examiner(s): Mariam; Daniel G
• Assistant Examiner(s): Seth; Manav
• Application Number:
  US10/386,984
• Publication Number:
  US 20040179714A1
• Time in Patent Office: 2,387 Days
• Field of Search: 382/103, 382/153, 348/211.12, 700/259, 700/245
• US Class Current: 382/103
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

A method for mutually-immersive telepresencing is provided with a view of a surrogate'"'"'s location. An image of the surrogate'"'"'s location is displayed at a user'"'"'s location. A user'"'"'s eye level and perspective are sensed. The height of the camera and image of the user'"'"'s eyes at the surrogate'"'"'s location are adjusted to match the height of the user'"'"'s eyes. The user'"'"'s perspective and, hence, gaze are preserved on the image while the user'"'"'s eye level changes.

Claims

1. A method for mutually-immersive telepresencing comprising:

providing a view of a surrogate'"'"'s location using a surrogate'"'"'s camera;

displaying at a user'"'"'s location an image of the view;

sensing a user having a user'"'"'s eye level and having a user'"'"'s perspective;

moving the surrogate'"'"'s camera to about match the user'"'"'s eye level and perspective; and

preserving the user'"'"'s perspective and, hence, gaze on the image while the user'"'"'s eye level changes, wherein preserving the user'"'"'s gaze includes tilting the view of the surrogate'"'"'s location while the user'"'"'s eye level changes.

2. The method as claimed in claim 1 wherein:

displaying the image includes displaying a life-size image of the surrogate'"'"'s location.

3. The method as claimed in claim 1 wherein:

preserving the user'"'"'s gaze includes;

displaying the image at a first height; and

moving the image from the first height while the user'"'"'s eye level changes.

4. The method as claimed in claim 1 wherein:

preserving the user'"'"'s gaze includes;

displaying a first portion of the image for a first range of the user'"'"'s eye level; and

displaying a second portion of the image for second range of the user'"'"'s eye level.

5. A method for mutually-immersive telepresencing comprising:

providing a view of a surrogate'"'"'s location;

displaying at a user'"'"'s location an image of the view;

sensing a user'"'"'s eye level of a user having a user'"'"'s vertical perspective;

moving the view of the surrogate'"'"'s location between a first and a second surrogate'"'"'s height as the user'"'"'s eye level changes between a first and second user'"'"'s height; and

preserving the user'"'"'s vertical perspective and, hence, gaze on the image while the user'"'"'s eye level changes between the first and second user'"'"'s height.

6. The method as claimed in claim 5 wherein:

displaying the image includes;

providing a life-size image of the surrogate'"'"'s location with a perspective distortion;

unwarping the perspective distortion of the life-size image to form an unwarped image with excess image; and

cropping the excess image to display the image life-size with perspective distortion reduced.

7. The method as claimed in claim 5 wherein:

preserving the user'"'"'s gaze includes;

displaying the image with the image having a vertical center; and

moving the vertical center in direct proportion to changes in the user'"'"'s eye level.

8. The method as claimed in claim 5 wherein:

preserving the user'"'"'s gaze includes tilting the view of the surrogate'"'"'s location in degrees in inverse direction to changes in the user'"'"'s eye level.

9. The method as claimed in claim 5 wherein:

preserving the user'"'"'s gaze includes;

texture mapping the image at the surrogate'"'"'s location or the user'"'"'s location to provide a texture-mapped image;

displaying a first portion of the texture mapped image for a first range of the user'"'"'s eye level;

displaying a second portion of the texture mapped image for a second range of the user'"'"'s eye level; and

discarding any remaining portion of the texture mapped image not included in the first and second portions of the texture mapped image.

10. A mutually-immersive telepresencing system comprising:

a surrogate having a surrogate'"'"'s camera for providing a view of a surrogate'"'"'s location;

a user'"'"'s display for displaying at a user'"'"'s location an image of the view;

a user height sensor system for sensing a user having a user'"'"'s eye level and having a user'"'"'s perspective, and changing the height of the surrogate camera to match the user'"'"'s eye level as height of the user changes; and

a user gaze preservation system for preserving the user'"'"'s gaze on the image while the height of the user and the user'"'"'s eye level changes.

11. The mutually-immersive telepresencing system as claimed in claim 10 wherein:

the user'"'"'s display displays a life-size image of the surrogate'"'"'s location.

12. The mutually-immersive telepresencing system as claimed in claim 10 wherein:

the user gaze preservation system includes;

a projection system for displaying the image at a first height; and

a mechanism for mechanically or optically moving the image from the first height while the user'"'"'s eye level changes.

13. The mutually-immersive telepresencing system as claimed in claim 10 wherein:

the user gaze preservation system includes a rotary mechanism for tilting the view of the surrogate'"'"'s location while the user'"'"'s eye level changes.

14. The mutually-immersive telepresencing system as claimed in claim 10 wherein:

the user gaze preservation system includes;

an electronic system for displaying a first portion of the image for a first range of the user'"'"'s eye level and for displaying a second portion of the image for second range of the user'"'"'s eye level.

15. A mutually-immersive telepresencing system comprising:

a surrogate having a surrogate'"'"'s camera for providing a view of a surrogate'"'"'s location;

a user'"'"'s display for displaying at a user'"'"'s location an image of the view;

a user height sensor system for sensing a user'"'"'s eye level of the user having a user'"'"'s vertical gaze;

a system for changing the view of the surrogate'"'"'s location between a first and a second surrogate'"'"'s height as the user'"'"'s eye level changes between a first and a second user'"'"'s height; and

a user gaze preservation system for preserving the user'"'"'s vertical gaze on the image while the user'"'"'s eye level changes between the first and second user'"'"'s height.

16. The mutually-immersive telepresencing system as claimed in claim 15 wherein:

the user'"'"'s display includes;

an electronic system for providing a life-size image of the surrogate'"'"'s location with a perspective distortion, unwarping the perspective distortion of the life-size image to form an unwarped image with excess image, and cropping the excess image to display the image life-size with perspective distortion reduced.

17. The mutually-immersive telepresencing system as claimed in claim 15 wherein:

the user gaze preservation system includes;

a projection system for displaying the image with the image having a vertical center; and

a mechanism for mechanically or optically moving the vertical center in direct proportion to changes in the user'"'"'s eye level.

18. The mutually-immersive telepresencing system as claimed in claim 15 wherein:

the user gaze preservation system includes a rotary mechanism for tilting the view of the surrogate'"'"'s location in degrees in inverse proportion to changes in the user'"'"'s eye level.

19. The mutually-immersive telepresencing system as claimed in claim 15 wherein:

the user gaze preservation system includes;

an electronic system for texture mapping the image at the surrogate'"'"'s location or the user'"'"'s location to provide a texture mapped image, for displaying a first portion of the texture mapped image for a first range of the user'"'"'s eye level, for displaying a second portion of the texture mapped image for a second range of the user'"'"'s eye level, and for discarding any remaining portion of the texture mapped image not included in the first and second portions of the texture mapped image.

20. The mutually-immersive telepresencing system as claimed in claim 15 wherein:

the surrogate'"'"'s camera is tilted from 22 degrees up to a horizontal plane to 27 degrees down to the horizontal plane.