System and method for immersive and interactive multimedia generation
First Claim
1. An apparatus, comprising a processor and a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the apparatus to perform a method, the method comprising:
- determining a first set of coordinates associated with a first feature in one or more first images, wherein the first images are captured from a first position or a first orientation relative to a physical environment;
determining a second set of coordinates associated with a second feature in one or more second images, wherein the second images are captured from a second position or a second orientation relative to the physical environment;
re-projecting the first set of coordinates to one or more 2D spaces associated with the second images;
comparing the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features;
determining a change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features;
determining a rendering of the physical environment based on the one or more first and second images and on the determined change in the orientation or position,providing data related to the rendering of the physical environment to a display system;
wherein comparing the re-projected first set of coordinates with the second set of coordinates in at least one of the position closeness, feature closeness, or stereo constraints comprises determining a distance between locations of the re-projected first set of coordinates and locations of the second set of coordinates in the one or more 2D spaces; and
wherein determining the change between the first and second orientations or the change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features comprises determining the change between the first and second orientations or the change between the first and second positions with respect to the physical environment based on minimizing the distance in the one or more 2D spaces.
2 Assignments
0 Petitions
Accused Products
Abstract
An apparatus comprising a processor and a non-transitory computer-readable storage medium storing instructions, when executed by the processor, cause the apparatus to perform a method comprising determining a first and second set of coordinates, associated with a first and second feature, respectively, in one or more first and second images, respectively, the first and second images are captured from a first and second position and/or orientation relative to the physical environment, respectively, re-projecting the first set of coordinates to one or more 2D spaces associated with the second images, comparing the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features and determining a change between the first and second orientations and/or positions with respect to the physical environment based on the determined correspondence.
-
Citations
15 Claims
-
1. An apparatus, comprising a processor and a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the apparatus to perform a method, the method comprising:
-
determining a first set of coordinates associated with a first feature in one or more first images, wherein the first images are captured from a first position or a first orientation relative to a physical environment; determining a second set of coordinates associated with a second feature in one or more second images, wherein the second images are captured from a second position or a second orientation relative to the physical environment; re-projecting the first set of coordinates to one or more 2D spaces associated with the second images; comparing the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features; determining a change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features; determining a rendering of the physical environment based on the one or more first and second images and on the determined change in the orientation or position, providing data related to the rendering of the physical environment to a display system; wherein comparing the re-projected first set of coordinates with the second set of coordinates in at least one of the position closeness, feature closeness, or stereo constraints comprises determining a distance between locations of the re-projected first set of coordinates and locations of the second set of coordinates in the one or more 2D spaces; and wherein determining the change between the first and second orientations or the change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features comprises determining the change between the first and second orientations or the change between the first and second positions with respect to the physical environment based on minimizing the distance in the one or more 2D spaces. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method comprising:
-
determining, by a processor, a first set of coordinates associated with a first feature in one or more first images, wherein the first images are captured from a first position or a first orientation relative to a physical environment; determining, by the processor, a second set of coordinates associated with a second feature in one or more second images, wherein the second images are captured from a second position or a second orientation relative to the physical environment; re-projecting, by the processor, the first set of coordinates to one or more 2D spaces associated with the second images; comparing, by the processor, the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features; determining, by the processor, a change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features; rendering, by the processor, of the physical environment based on the one or more first and second images and on the determined change in orientation or position; and providing, by the processor, data related to the rendering of the physical environment to a display system, wherein the comparing, by the processor, the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features comprises determining a distance between locations of the re-projected first set of coordinates and locations of the second set of coordinates in the one or more 2D spaces; and wherein the determining, by the processor, a change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features comprises determining the change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on minimizing the distance in the one or more 2D spaces. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method, the method comprising:
-
determining a first set of coordinates associated with a first feature in one or more first images, wherein the first images are captured from a first position or a first orientation relative to a physical environment; determining a second set of coordinates associated with a second feature in one or more second images, wherein the second images are captured from a second position or a second orientation relative to the physical environment; re-projecting the first set of coordinates to one or more 2D spaces associated with the second images; comparing the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features; determining a change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features; determining a rendering of the physical environment based on the one or more first and second images and on the determined change in the orientation or position; and providing data related to the rendering of the physical environment to a display system, wherein the comparing the re-projected first set of coordinates with the second set of coordinates in at least one of position closeness, feature closeness, or stereo constraints to determine a correspondence between the first and second features comprises determining a distance between locations of the re-projected first set of coordinates and locations of the second set of coordinates in the one or more 2D spaces; and wherein the determining the change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on the determined corresponding first and second features comprises determining the change between the first and second orientations or a change between the first and second positions with respect to the physical environment based on minimizing the distance in the one or more 2D spaces.
-
Specification