PARALLAX COMPENSATION

• US 20130155180A1
• Filed: 12/14/2011
• Published: 06/20/2013
• Est. Priority Date: 12/14/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Barnett, Donald, Wantland, Timothy, Ramos, Gonzalo Alberto
• Granted Patent: US 10,008,021 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Among other things, one or more techniques and/or systems are disclosed for rendering imagery that compensates for parallax. Image data may comprise a sequence of images of a location, collected at known intervals (e.g., distances), such as along a path. The image data can comprise a first layer comprising a first depth, and a second layer comprising a second depth, that is different from the first depth. The image data can be composed into resulting imagery, where the first layer is composed at the first depth and the second layer is composed at the second depth. The resulting imagery can be rendered to compensate for, or rather make use of, parallax, such that the first layer is rendered at a first movement speed, based at least upon the first depth, and the second layer is rendered at a second movement speed, based at least upon the second depth.

Claims

1. A computer-based method for rendering imagery that compensates for parallax, comprising:

receiving image data comprising a first layer, comprising a first depth, and a second layer, comprising a second depth;

composing the first layer at the first depth in resulting imagery and composing the second layer at the second depth in the resulting imagery; and

rendering the resulting imagery to compensate for parallax, comprising rendering the first layer at a first movement speed and the second layer at a second movement speed, the first movement speed and the second movement speed based at least upon the first depth and the second depth, at least a portion of the rendering performed by a computer-based processor.

2. The method of claim 1, the resulting imagery comprising a lateral panorama created from a plurality of combined consecutive images obtained from corresponding image data.

3. The method of claim 1, comprising determining at least one of the first depth and the second depth, comprising one or more of:

identifying a first object depth for an object in the image data from a first perspective in a first image; and

identifying a second object depth for the object in the image data from a second perspective in a second image.

4. The method of claim 1, at least one of the first layer and the second layer comprising an object tag layer, the object tag layer comprising information associated with an object identified in the image data.

5. The method of claim 4, the information associated with the object comprising geo-location data for the object received from one or more online sources.

6. The method of claim 4, composing comprising annotating the information associated with the object to the object in the resulting imagery.

7. The method of claim 1, the first layer comprising foreground image data and the second layer comprising background image data.

8. The method of claim 7:

receiving image data comprising receiving image data comprising an object tag layer, the object tag layer comprising information associated with an object identified in the image data;

composing comprising composing the object tag layer at a third depth in the resulting imagery; and

rendering the resulting imagery comprising rendering the object tag layer in the resulting imagery at a third movement speed, the third movement speed based at least upon the third depth.

9. The method of claim 8, the third movement speed based at least upon an object weighting factor.

10. The method of claim 1, comprising:

receiving an indication of a panning speed for the resulting imagery; and

determining the first movement speed and the second movement speed based at least upon the panning speed.

11. The method of claim 1, comprising:

identifying a non-desired view of an object in the image data; and

composing the resulting imagery to mitigate the non-desired view of the object based at least upon the first depth and the second depth.

12. A system for rendering imagery that compensates for parallax, comprising:

a computer-based processor configured to process data for the system;

an imagery composing component, operably coupled with the processor, configured to compose received image data into resulting imagery, the resulting imagery comprising a first layer disposed at a first depth and a second layer disposed at a second depth;

a movement speed determination component, operably coupled with the processor, configured to determine a first movement speed for the first layer and a second movement speed for the second layer, based at least upon the first depth and the second depth; and

an imagery rendering component, operably coupled with the processor, configured to render the first layer in the resulting imagery at the first movement speed and render the second layer in the resulting imagery at the second movement speed, to compensate for parallax.

13. The system of claim 12, comprising a depth determination component configured to determine at least one of the first depth and the second depth, based upon one or more of:

a first object depth for an object in the image data from a first perspective in a first image; and

a second object depth for the object in the image data from a second perspective in a second image.

14. The system of claim 13, comprising an object depth determination component configured to perform one or more of:

identify the object in the image data; and

determine at least one of the first object depth and the second object depth based at least in part on a distance between the object and an image capture location.

15. The system of claim 12, the movement speed determination component configured to determine at least one of the first movement speed and the second movement speed based at least upon a received indication of a panning speed for the resulting imagery.

16. The system of claim 12, comprising an object tag layer component configured to identify an object tag layer for the image data, the identifying comprising one or more of:

receiving object tag layer data, the object tag layer data comprising information associated with an object in the image data; and

annotating at least a portion of the information associated with the object to the object in the resulting imagery.

17. The system of claim 16, the information associated with the object comprising one or more of:

geo-location data identifying a location of the object;

identification data that identifies the object; and

descriptor information that provides information about the object.

18. The system of claim 16, the movement speed determination component configured to determine a third movement speed for the object tag layer, based at least upon a third depth associated with the object tag layer.

19. The system of claim 12, comprising a non-desired view mitigating component configured to mitigate a non-desired view of an object in the image data, comprising one or more of:

identifying the non-desired view of the object in the image data; and

adjusting the resulting imagery to mitigate the non-desired view of the object based at least upon the first depth and the second depth.

20. A computer readable medium comprising computer executable instructions that when executed via a processor perform a method for rendering imagery that compensates for parallax, comprising:

receiving image data comprising a first layer and a second layer, at least one of the first layer and the second layer comprising an object tag layer, the object tag layer comprising information associated with an object identified in the image data;

determining at least one of a first depth for the first layer and a second depth for the second layer, comprising one or more of;

identifying a first object depth for the object in the image data from a first perspective in a first image; and

identifying a second object depth for the object in the image data from a second perspective in a second image;

composing the first layer at the first depth in resulting imagery and composing the second layer at the second depth in the resulting imagery, the resulting imagery comprising a lateral panorama created from a plurality of combined consecutive images obtained from corresponding image data;

receiving an indication of a panning speed for the resulting imagery;

determining a first movement speed for the first layer and a second movement speed for the second layer based at least upon the panning speed, the first depth and the second depth; and

rendering the resulting imagery to compensate for parallax, comprising rendering the first layer at a first movement speed and the second layer at a second movement speed.