Telepresence system

• US 9,794,516 B1
• Filed: 11/16/2016
• Issued: 10/17/2017
• Est. Priority Date: 11/16/2016
• Status: Active Grant

• Current Assignee: Raja Singh Tuli
• Original Assignee: Raja Singh Tuli
• Inventors: Tuli, Raja Singh
• Primary Examiner(s): Smith, Creighton
• Application Number:
  US15/352,783
• Time in Patent Office: 335 Days
• Field of Search: None
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 5/655 Construction or mounting of chassis, e.g. for varying the elevation of the tube
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A system for telepresencing is provided with a user station at a remote user'"'"'s location and a telepresence device at a local subject'"'"'s location. Distance of objects around the local subject is sensed such that a selected distance between the telepresence device and the local subject is maintained. Eye features and gazes of the local subject and the remote user are sensed. The height of the telepresence device'"'"'s display is adjusted automatically such that the eye level of the local subject matches with the eye level of the user'"'"'s image in the display. When an intended eye contact is sensed, positions of the telepresence device, its display and a sensor hub that houses cameras and other sensors are adjusted, and a face image may be recreated at a different viewing angle to resemble an eye contact between the local subject and the remote user while preserving all other facial expressions.

Claims

1. A telepresence system comprising a telepresence device and a user station for conducting video conferencing between a remote user at a first location and a local subject at a second location, wherein the user station is at the first location and the telepresence device is at the second location, the telepresence device comprising:

a display unit having an adjustable height to display to the local subject a video image of the first location;

a processing unit tracking characteristics of the remote user in the image displayed on the display unit, wherein the characteristics of the remote user identify an eye level of the remote user in the image displayed on the display unit;

at least one sensor unit to sense characteristics of the local subject at the second location, wherein the characteristics of the local subject identify an eye level of the local subject; and

a motorized support unit that automatically adjusts the height of the display unit to match the eye level of the remote user identified in the image displayed on the display unit with the identified eye level of the local subject.

2. The telepresence system as claimed in claim 1, wherein the motorized support unit automatically adjusts the height of the display unit to match the eye level of the remote user identified in the image displayed on the display unit with the identified eye level of the local subject in response to a change in the characteristics of the local subject identifying the eye level of the local subject.

3. The telepresence system as claimed in claim 1, wherein the display unit is translated automatically to orient the display unit toward the local subject in response to a physical movement of the local subject.

4. The telepresence system as claimed in claim 1, wherein the motorized support unit is adapted so that the display unit covers height-levels ranging from a seated short person to a standing tall person.

5. The telepresence system as claimed in claim 1, wherein

the sensed characteristics of the local subject identify eye positions of the local subject, and

a sensor hub that is translatable, tiltable or rotatable, or a combination thereof, comprises at least one sensor unit, wherein the sensor hub is electronically controlled to automatically be oriented toward the positions of the local subject.

6. The telepresence system as claimed in claim 1, wherein the user station comprises at least one sensor unit to perform 3D image mapping.

7. The telepresence system as claimed in claim 1, wherein the telepresence device further comprises:

at least one sensor unit to sense spatial characteristics of objects near either the local subject or the telepresence device at the second location;

a motorized mobile unit to move the telepresence device; and

a processing unit to control speed, displacement and direction of the motorized mobile unit, configured to maintain a selected distance between the telepresence device and the local subject, wherein the selected distance is based on the spatial characteristics of objects near either the local subject or the telepresence device.

8. The telepresence system as claimed in claim 7 wherein the motorized mobile unit is configured to maintain the selected distance between the telepresence device and the local subject in response to a change in position of the local subject.

9. The telepresence system as claimed in claim 7 wherein:

movements of the telepresence device at the second location are controlled by the remote user at the first location; and

the motorized mobile unit is configured to avoid potential obstacles when movements of the telepresence device are controlled by the remote user.

10. The telepresence system as claimed in claim 7, wherein

the sensed characteristics of the local subject identify eye positions of the local subject, and

a sensor hub that is translatable, tiltable or rotatable, or a combination thereof, comprises at least one sensor unit, wherein the sensor hub is electronically controlled to automatically be oriented toward the eye positions of the local subject.

11. The telepresence system as claimed in claim 1, wherein:

the sensed characteristics of the local subject identify eye positions of the local subject or gaze direction of the local subject, or both; and

the telepresence device further comprises a processing unit to identify a gaze point of the local subject;

a processing unit to detect an intended eye contact between the remote user and the local subject; and

a processing unit to execute a gaze matching system based on the sensed characteristics of the local subject and the tracked characteristics of the remote user in the image displayed on the display unit when an intended eye contact between the remote user and the local subject is detected.

12. The telepresence system as claimed in claim 11, wherein a sensor hub that is translatable, bitable or rotatable, or a combination thereof, comprises at least one sensor unit, wherein the sensor hub is electronically controlled to automatically be oriented toward the eye positions of the local subject.

13. The telepresence system as claimed in claim 11 wherein:

the tracked characteristics of the remote user in the image displayed on the display unit identify eye positions of the remote user in the image displayed on the display unit, and

the eye positions of the remote user identified in the image displayed on the display unit are substantially centered when no intention of eye contact between the remote user and the local subject is detected.

14. The telepresence system as claimed in claim 11 wherein:

the tracked characteristics of the remote use in the image displayed on the display unit identify eye positions of the remote user in the image displayed on the display unit, and

the eye positions of the remote user identified in the image displayed on the display unit are substantially centered when an intention of eye contact between the remote user and the local subject is detected.

15. The telepresence system as claimed in claim 11, wherein facial expressions of the remote user are preserved.

16. The telepresence system as claimed in claim 1, wherein the display unit is tilted automatically to orient the display unit toward the local subject in response to a physical movement of the local subject.

17. The telepresence system as claimed in claim 1, wherein the display unit is rotated automatically to orient the display unit toward the local subject in response to a physical movement of the local subject.

18. The telepresence system as claimed in claim 1, wherein the user station comprises at least one sensor unit to track eye positions of the remote user at the first location.

19. The telepresence system as claimed in claim 1, wherein the user station comprises at least one sensor unit to sense gaze direction of the remote user.

20. The telepresence system as claimed in claim 11, wherein:

the telepresence device further comprises at least one sensor unit to either sense audio gain characteristics or sense human gesture by the local subject, or both; and

the intended eye contact between the remote user and the local subject is detected based on either the sensed audio gain characteristics or on the sensed human gesture by the local subject, or on both.