System and method for displaying images using foveal video
First Claim
Patent Images
1. A robotic telepresence system, comprising:
- a user station at a first geographic location, the user station being responsive to a user, communicating information to the user, and communicating information from the user, the user station including;
a processor, a display device coupled to the processor, and a memory; and
a robot at a second geographic location, responsive to the user station, the robot displaying a three dimensional representation of the user transmitted from the user station, the robot sensing predetermined types of information and communicating that information back to the user, the robot including;
at least one first sensor to sense information at a first predetermined high resolution level, and at least one second sensor to sense information at a predetermined low resolution level, wherein the robot transmits the sensed information from the at least one first sensor and from the at least one second sensor to the user station, wherein the memory of the user station stores instructions that cause the processor to;
display on the display device a high resolution image and at least two low resolution images combined to produce a single image, partially high resolution, partially low resolution, the high resolution image at least partially overlapping at least one of the low resolution images on the display, the high resolution image being sensed by the at least one first sensor of the robot, the low resolution images being sensed by the at least one second sensor of the robot.
1 Assignment
0 Petitions
Accused Products
Abstract
A high resolution image and at least two low resolution images are combined to produce a single image, partially high resolution, partially low resolution on a display. The high resolution image at least partially overlaps at least one of the low resolution images. This method of displaying images is referred to as foveal video. In another aspect of the invention, in a robotic telepresence system, a user station displays information received from a robot using foveal video.
-
Citations
28 Claims
-
1. A robotic telepresence system, comprising:
-
a user station at a first geographic location, the user station being responsive to a user, communicating information to the user, and communicating information from the user, the user station including;
a processor, a display device coupled to the processor, and a memory; and
a robot at a second geographic location, responsive to the user station, the robot displaying a three dimensional representation of the user transmitted from the user station, the robot sensing predetermined types of information and communicating that information back to the user, the robot including;
at least one first sensor to sense information at a first predetermined high resolution level, and at least one second sensor to sense information at a predetermined low resolution level, wherein the robot transmits the sensed information from the at least one first sensor and from the at least one second sensor to the user station, wherein the memory of the user station stores instructions that cause the processor to;
display on the display device a high resolution image and at least two low resolution images combined to produce a single image, partially high resolution, partially low resolution, the high resolution image at least partially overlapping at least one of the low resolution images on the display, the high resolution image being sensed by the at least one first sensor of the robot, the low resolution images being sensed by the at least one second sensor of the robot. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
scale the high resolution image to the saint scale as at least one of the low resolution images.
-
-
3. The robotic telepresence system of claim 1 wherein the memory further includes instructions that cause the processor to:
align the high resolution image with at least one of the low resolution images.
-
4. The robotic telepresence system of claim 1 wherein the memory further includes instructions that cause the processor to:
display a border around the high resolution image.
-
5. The robotic telepresence system of claim 1 wherein the memory further includes instructions that cause the processor to:
blend edges of the high resolution image with at least one of the displayed low resolution images whereby a seamless combined image is provided.
-
6. The robotic telepresence system of claim 1 wherein the at least two low resolution images include first and second low resolution images, the first and second low resolution images each having an overlapping subset of image information at an edge;
- and wherein the memory further includes instructions that cause the processor to;
blend the overlapping subset of image information of the first and second low resolution images.
- and wherein the memory further includes instructions that cause the processor to;
-
7. The robotic telepresence system of claim 1 wherein the at least two low resolution images include first and second low resolution images, the high resolution image having an overlapping subset of image information with a portion of the at least one low resolution image, and wherein the memory further includes instructions that cause the processor to:
-
align the high resolution image with the portion of the at least one low resolution image; and
blend an edge portion of the high resolution image with a corresponding portion of the low resolution image, whereby a seamless image is produced.
-
-
8. The robotic telepresence system of claim 1 and wherein the memory further includes instructions that cause the processor to:
magnify a portion of the single image.
-
9. The robotic telepresence system of claim 1 wherein the robot is mobile.
-
10. The robotic telepresence system of claim 1 wherein the user station includes an immersion room including:
-
a curved screen;
projectors for projecting a low resolution image of a received low resolution image sent from the robot;
a projector for projecting a high resolution image of a received high resolution image sent from the robot onto a midpoint area of the curved screen;
a user station positioned substantially at the center point facing the midpoint area of the curved screen; and
a set of mirrors, disposed between the user station and the curved screen, for receiving the projected low resolution images, each mirror receiving one projected low resolution image from one low resolution projector, and reflecting the projected low resolution image onto a portion of the curved screen.
-
-
11. The robotic telepresence system of claim 10 wherein the immersion room further comprises:
a control computer system for receiving the low resolution images as low resolution image data from the robot and also receiving die high resolution image as high resolution image data from the robot, the control computer processing the low resolution image data to compensate for a keystone effect in the projected low resolution image on the curved screen, wherein the alignment of the low resolution projectors with respect to the mirrors and the curved screen produces the keystone effect.
-
12. The robotic telepresence system of claim 11 wherein the low resolution projectors and the high resolution projectors are arranged to provide for standing worn behind the user station.
-
13. A robotic telepresence system, comprising:
-
a user station at a first geographic location, the user station being responsive to a user, communicating information to the user and communicating information from the user, the user station including;
a processor, a display device coupled to the processor, and a memory; and
a robot at a second geographic location, responsive to the user station, the robot displaying a three dimensional representation of the user transmitted from the user station, the robot sensing predetermined types of information and communicating that information back to the user, the robot including;
at least one first sensor to sense high resolution video information, and at least one second sensor to sense low resolution video information, wherein the robot transmits the sensed high resolution video information and the sensed low resolution video information to the user station, wherein the memory to the user station stores instructions that cause the processor to;
display on the display device the high resolution video information ant the low resolution video information to produce unified video information that is partially high resolution and partially low resolution. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20)
scale the high resolution video information at the same scale as of the low resolution video information.
-
-
15. The robotic telepresence system of claim 13 wherein the memory further includes instructions that cause the processor to:
align the high resolution video information with the low resolution video information.
-
16. The robotic telepresence system of claim 13 wherein the memory further includes instructions that cause the processor to:
display a border around to high resolution video information.
-
17. The robotic telepresence system of claim 13 wherein the memory further includes instructions that cause the processor to:
blend edges of the high resolution video information with at least one of the displayed low resolution video information.
-
18. The robotic telepresence system of claim 13 and wherein the memory further includes instructions that cause the processor to:
magnify a portion of the unified video information.
-
19. The robotic telepresence system of claim 13 wherein the robot is mobile.
-
20. The robotic telepresence system of claim 13 wherein the display device comprises:
-
a curved screen;
projectors for projecting the low resolution video information;
a projector for projecting a high resolution video information onto a midpoint area of the curved screen.
-
-
21. A robotic telepresence system, comprising:
-
a robot having high and low resolution robot cameras directed radially outward from the robot with the radial outward directions not intersecting, the robot having robot displays facing radially outward from the robot;
a user station having user cameras directed radially inward towards a center region and having a user display for providing images from the robot cameras, the images facing radially inward towards the center region; and
a computer communication connection for providing;
bi-directional communication between the robot and the user station, the robot with images of the center region on the robot displays, and the user station with high and low resolution images of views radially outward from the robot, the high and low resolution images on the user display. - View Dependent Claims (22, 23, 24, 25, 26, 27, 28)
the user camera image of the center region having a user at the user station to the robot display; and
the image of the robot camera radially behind the robot display to the robot display as background to the user.
-
-
25. The robotic telepresence system as claimed in claim 21 wherein the computer communication connection is for providing the user display with a foveal image.
-
26. The robotic telepresence stem as claimed in claim 21 wherein the computer communication connection is for providing the user display with an anamorphic image.
-
27. The robotic telepresence system as claimed in claim 21 wherein the computer communication connection is for providing the user display wit a compensated image to provide a user with a sense of being present at a robot location.
-
28. The robotic telepresence system as claimed in claim 21 wherein the computer communication connection is for providing the robot displays with views of a user to provide the appearance of the user being three-dimensional.
Specification