Dynamic perspective video window

• US 8,379,057 B2
• Filed: 05/14/2012
• Issued: 02/19/2013
• Est. Priority Date: 04/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mann, Samuel A., Bertolami, Joseph, Bronder, Matthew L., Dougherty, Michael A., Craig, Robert M., Tardif, John A.
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US13/470,957
• Publication Number:
  US 20120223967A1
• Time in Patent Office: 281 Days
• Field of Search: 345/633
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Systems and methods are disclosed for generating an image for a user based on an image captured by a scene-facing camera or detector. The user'"'"'s position relative to a component of the system is determined, and the image captured by the scene-facing detector is modified based on the user'"'"'s position. The resulting image represents the scene as seen from the perspective of the user. The resulting image may be further modified by augmenting the image with additional images, graphics, or other data.

Claims

1. A method of providing a dynamic perspective video window, comprising:

capturing a first image with a first detector;

capturing a second image with a second detector;

determining a position of a user based on the second image;

determining a crop area based on a position of the user;

cropping the first image based on the crop area;

determining an augmentation image;

augmenting the first cropped image by compositing the augmentation image with the first cropped image to generate a composite image; and

rendering the composite image on a display.

2. The method of claim 1, further comprising augmenting the first cropped image by compositing graphical elements with the first cropped image to generate the composite image.

3. The method of claim 1, further comprising augmenting the first cropped image with a supplemental image to generate the composite image.

4. The method of claim 1, wherein determining the position of the user based on the second image comprises determining at least one of a horizontal orientation of the user, a vertical orientation of the user, and a distance of the user from the display.

5. The method of claim 1, wherein determining the crop area based on the position of the user comprises determining a field of view of the user.

6. The method of claim 1, wherein determining the position of the user based on the second image comprises determining a position of a portion of the user.

7. The method of claim 6, wherein the portion of the user is at least one of a head of the user, an eye of the user, a torso of the user, and a body of the user.

8. A computer-readable medium that is not a transient signal, the computer-readable storage medium comprising computer-readable instructions for providing a dynamic perspective video window, the computer-readable instructions comprising instructions for:

capturing a first image with a first camera;

capturing a second image with a second camera;

determining a position of a user based on the second image;

determining a crop area based on the position of the user;

cropping the first image based on the crop area;

determining an augmentation image;

augmenting the first cropped image by compositing the augmentation image with the first cropped image to generate a composited image; and

transmitting the composited image to a display.

9. The computer-readable medium of claim 8 wherein instructions for determining the position of the user based on the second image comprise instructions for determining a position of a portion of the user.

10. The computer-readable medium of claim 9, wherein instructions for determining the crop area based on the position of the user comprise instructions for determining a field of view of the user.

11. The computer-readable medium of claim 8, further comprising instructions for transmitting a request for data to a remote device, receiving the data from the remote device, and, based on the received data, further augmenting the first cropped image.

12. The computer-readable medium of claim 11, further comprising instructions for formatting the received data in a text format, and wherein further augmenting the first cropped image comprises overlaying the received data in the text format on the first cropped image to generate the composited image.

13. A system for providing a dynamic perspective video window, comprising:

a first detector configured to capture a first image;

a second detector configured to capture a second image;

a processor configured to;

determine a position of a user based on the second image;

determine a crop area based on the position of the user;

crop the first image based on the crop area;

determine an augmentation image; and

generate a first display image by augmenting the first cropped image by compositing the augmentation image with the first cropped image; and

a first display configured to render the first display image.

14. The system of claim 13, further comprising a second display, wherein the processor is further configured to determine a second position of the user based on the second image and generate a second display image based on the first image and the second position of the user, and wherein the second display is configured to render the second display image.

15. The system of claim 14, wherein the processor configured to determine the first position of the user based on the second image comprises the processor configured to determine a first position of a first eye of the user, and

wherein the processor configured to determine the second position of the user based on the first user image comprises the processor configured to determine a second position of a second eye of the user.

16. The system of claim 13, wherein the processor is further configured to generate a first supplemental image and generate the first display image by augmenting the first cropped image with the first supplemental image.

17. The system of claim 13, further comprising a communications component configured to transmit data to a remote device and receive data from the remote device.

18. The system of claim 13, wherein the first detector is configured in a first housing and the processor is configured in a second housing.

19. The system of claim 18, wherein the first detector is further configured to wirelessly transmit the first to the processor.

20. The system of claim 13, wherein the second detector is configured in a first housing and the processor is configured in a second housing.