Dynamic perspective video window

• US 8,194,101 B1
• Filed: 04/01/2009
• Issued: 06/05/2012
• Est. Priority Date: 04/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mann, Samuel A., Bertolami, Joseph, Bronder, Matthew L., Dougherty, Michael A., Craig, Robert M., Tardif, John A.
• Primary Examiner(s): Brier, Jeffrey A
• Application Number:
  US12/416,505
• Time in Patent Office: 1,161 Days
• Field of Search: 345/633
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Systems and methods are disclosed for generating an image for a user based on an image captured by a scene-facing camera or detector. The user'"'"'s position relative to a component of the system is determined, and the image captured by the scene-facing detector is modified based on the user'"'"'s position. The resulting image represents the scene as seen from the perspective of the user. The resulting image may be further modified by augmenting the image with additional images, graphics, or other data.

Claims

1. A method comprising:

capturing a first image with a first detector;

capturing a second image with a second detector;

receiving a remote image from a remote device;

determining a position of a user based on the second image;

determining a crop area based on a position of the user;

cropping the first image based on the crop area;

compositing the remote image with the first cropped image; and

rendering the first cropped image on a display.

2. The method of claim 1, further comprising compositing graphical elements with the first cropped image.

3. The method of claim 1, further comprising augmenting the first cropped image with a supplemental image.

4. The method of claim 1, wherein determining the position of the user based on the second image comprises determining at least one of a horizontal orientation of the user, a vertical orientation of the user, and a distance of the user from the display.

5. The method of claim 1, wherein determining the crop area based on the position of the user comprises determining a field of view of the user.

6. The method of claim 1, wherein determining the position of the user based on the second image comprises determining a position of a portion of the user.

7. The method of claim 6, wherein the portion of the user is at least one of a head of the user, an eye of the user, a torso of the user, and a body of the user.

8. A computer-readable medium that is not a transient signal, the computer-readable medium comprising computer-readable instructions for:

capturing a first image with a first camera;

capturing a second image with a second camera;

receiving a remote image from a remote device;

determining a position of a user based on the second image;

determining a crop area based on the position of the user;

cropping the first image based on the crop area;

compositing the remote image with the first cropped image;

transmitting the first cropped image to a display.

9. The computer-readable medium of claim 8 wherein instructions for determining the position of the user based on the second image comprise instructions for determining a position of a portion of the user.

10. The computer-readable medium of claim 9, wherein instructions for determining the crop area based on the position of the user comprise instructions for determining a field of view of the user.

11. The computer-readable medium of claim 8, further comprising instructions for transmitting a request for data to the remote device, receiving the data from the remote device, and, based on the received data, augmenting the first cropped image.

12. The computer-readable medium of claim 11, further comprising instructions for formatting the data in a text format, and wherein augmenting the first cropped image comprises overlaying the data in the text format on the first cropped image.

13. A system for providing a dynamic perspective video window, comprising:

a first detector configured to capture a first image;

a second detector configured to capture a second image;

a receiver configured to receive a remote image from a remote device;

a processor configured to;

determine a position of a user based on the second image,

determine a crop area based on the position of the user;

crop the first image based on the crop area; and

generate a first display image by compositing the remote image with the first cropped image; and

a first display configured to render the first display image.

14. The system of claim 13, further comprising a second display, wherein the processor is further configured to determine a second position of the user based on the second image and generate a second display image based on the first image and the second position of the user, and wherein the second display is configured to render the second display image.

15. The system of claim 14, wherein the processor configured to determine the first position of the user based on the second image comprises the processor configured to determine a first position of a first eye of the user, and

wherein the processor configured to determine the second position of the user based on the first user image comprises the processor configured to determine a second position of a second eye of the user.

16. The system of claim 13, wherein the processor is further configured to generate a first supplemental image and generate the first display image comprising the first supplemental image.

17. The system of claim 13, further comprising a communications component configured to transmit data to the remote device and receive data from the remote device.

18. The system of claim 13, wherein the first detector is configured in a first housing and the processor is configured in a second housing.

19. The system of claim 18, wherein the first detector is further configured to wirelessly transmit the first to the processor.

20. The system of claim 13, wherein second detector is configured in a first housing and the processor is configured in a second housing.