Systems and methods for reconstructing 3-dimensional model based on vertices

• US 10,042,672 B2
• Filed: 12/01/2014
• Issued: 08/07/2018
• Est. Priority Date: 03/12/2014
• Status: Active Grant

• Current Assignee: Live Planet LLC
• Original Assignee: Live Planet LLC
• Inventors: Arun, Rohan Maroly kovumal
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US14/557,238
• Publication Number:
  US 20150262420A1
• Time in Patent Office: 1,345 Days
• Field of Search: 345419, 345420, 345421
• CPC Class Codes: G06F 3/1438 using more than one graphics controller
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2210/52 Parallel processing
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers

Abstract

Systems and methods are described for reconstructing a 3-dimensional model of a target space, including, but not limited to: partitioning a model of a target space into a plurality of vertices; determining at least one display attribute associated with each of the plurality of vertices based on output data observed by a plurality of data sources; and selecting one of the at least one display attribute for each of the plurality of vertices.

Claims

1. A method comprising:

partitioning a model of a target space into a plurality of vertices;

determining two or more display attributes associated with each of at least one of the plurality of vertices based on output data provided by a plurality of data sources, wherein each of the two or more display attributes is captured by at least one data source of plurality of data sources, the at least one data source being a different data source relative to each data source that captures each of the other display attributes of the two or more display attributes; and

selecting one of the two or more display attributes for each of the at least one of the plurality of vertices;

wherein the one of the two or more display attributes is selected based on weights associated with each of the two or more display attributes; and

wherein the weights associated with each of the two or more display attributes are based on a number of the plurality of data sources capturing each of the two or more display attributes.

2. The method of claim 1, further comprising providing a plurality of data sources, each of the plurality of data sources output data corresponding to a current frame as the output data comprising a plurality of areas.

3. The method of claim 2, wherein each of the plurality of areas corresponds to one of the plurality of vertices.

4. The method of claim 3, wherein determining the at least one display attribute associated with each of the plurality of vertices comprises determining a display attribute associated with each of the plurality of areas of the current frame for each of the plurality of data sources.

5. The method of claim 2, wherein each of the plurality of data sources provides the output data corresponding to at least some of the plurality of vertices.

6. The method of claim 5, wherein each of the plurality of data sources comprises at least one digital video camera arranged at a camera position and orientation that is different from the camera position and orientation of another one of the plurality of data sources.

7. The method of claim 5, wherein the output data corresponding to at least one of the plurality of vertices is outputted by two or more of the plurality of data sources.

8. The method of claim 2, wherein:

the plurality of data sources are arranged in two or more levels; and

the at least one display attribute for a first vertex of the plurality of vertices is determined based on the output data outputted by the at least one of the plurality of data sources in a first level of the two or more levels.

9. The method of claim 1, further comprising projecting at least one of the plurality of vertices onto one of the plurality of areas.

10. The method of claim 1, wherein partitioning a model of a target space into a plurality of vertices comprises:

receiving an exterior boundary of the model of the target space;

determining density of the plurality of vertices; and

sampling a volume defined by the exterior boundary of the model based on the density of the plurality of vertices.

11. The method of claim 10, wherein the density of the plurality of vertices is determined based on at least one of resolution desired, processing power available, and network conditions.

12. The method of claim 1, wherein the target space is a 3-dimensional space.

13. The method of claim 1, wherein the output data comprises one or more of an image or video frame.

14. The method of claim 1, wherein:

the at least one display attribute comprises two or more colors; and

selecting one of the at least one display attribute for each of the plurality of vertices comprises selecting one of the two or more colors for each of the plurality of vertices.

15. The method of claim 1, wherein the at least one display attribute is at least one of a texture, frequency transform, wavelet transform, average, or standard deviation.

16. The method of claim 1, wherein:

the model of the target space comprises at least one object; and

the model of the target space is partitioned based on a size of discrete volumes making up an overall volume of the model of the target space.

17. The method of claim 1, wherein determining the two or more display attributes associated with each of the at least one of the plurality of vertices based on the output data provided by the plurality of data sources comprises:

determining a first display attribute of the two or more display attributes associated one of the at least one of the plurality of vertices based on first output data provided by a first data source of the plurality of data sources; and

determining a second display attribute of the two or more display attributes associated the one of the at least one of the plurality of vertices based on second output data provided by a second data source of the plurality of data sources; and

wherein selecting the one of the two or more display attributes comprises selecting one of the first display attribute and the second display attribute.

18. A method comprising:

partitioning a model of a target space into a plurality of vertices;

determining two or more display attributes associated with each of at least one of the plurality of vertices based on output data provided by a plurality of data sources, wherein each of the two or more display attributes is captured by at least one data source of plurality of data sources, the at least one data source being a different data source relative to each data source that captures each of the other display attributes of the two or more display attributes;

selecting one of the two or more display attributes for each of the at least one of the plurality of vertices; and

providing a plurality of data sources, each of the plurality of data sources output data corresponding to a current frame as the output data comprising a plurality of areas;

wherein the plurality of data sources are arranged in two or more levels; and

wherein the at least one display attribute for a first vertex of the plurality of vertices is determined based on the output data outputted by the at least one of the plurality of data sources in a first level of the two or more levels; and

wherein the at least one display attribute for the plurality of vertices other than the first vertex is determined based on the output from the plurality of data sources associated with levels other than the first level when a weight associated with one of the at least one display attribute for the first vertex exceeds a predetermined threshold.

19. A method comprising:

partitioning a model of a target space into a plurality of vertices;

determining two or more display attributes associated with each of at least one of the plurality of vertices based on output data provided by a plurality of data sources, wherein each of the two or more display attributes is captured by at least one data source of plurality of data sources, the at least one data source being a different data source relative to each data source that captures each of the other display attributes of the two or more display attributes;

selecting one of the two or more display attributes for each of the at least one of the plurality of vertices; and

providing a plurality of data sources, each of the plurality of data sources output data corresponding to a current frame as the output data comprising a plurality of areas;

wherein the plurality of data sources are arranged in two or more levels; and

wherein the at least one display attribute for a first vertex of the plurality of vertices is determined based on the output data outputted by the at least one of the plurality of data sources in a first level of the two or more levels; and

wherein the at least one display attribute for the first vertex is determined based on the output data outputted by the at least one of the plurality of data sources associated with a second level of the two or more levels when the weights associated with any of the at least one display attribute is equal to or less than a predetermined threshold.

20. An apparatus, the apparatus configured to:

partition a model of a target space into a plurality of vertices;

determine two or more display attributes associated with each of at least one of the plurality of vertices based on output data observed by a plurality of data sources, wherein each of the two or more display attributes is captured by at least one data source of plurality of data sources, the at least one data source being a different data source relative to each data source that captures each of the other display attributes of the two or more display attributes; and

select one of the two or more display attributes for each of the at least one of the plurality of vertices;

wherein the one of the two or more display attributes is selected based on weights associated with each of the two or more display attributes; and

wherein the weights associated with each of the two or more display attributes are based on a number of the plurality of data sources capturing each of the two or more display attributes.

21. The apparatus of claim 20, wherein the apparatus is further configured to provide a plurality of data sources, each of the plurality of data sources output data corresponding to a current frame as the output data comprising a plurality of areas.

22. The apparatus of claim 21, wherein each of the plurality of areas corresponds to one of the plurality of vertices.

23. The apparatus of claim 22, wherein determining the at least one display attribute associated with each of the plurality of vertices comprises determining a display attribute associated with each of the plurality of areas of the current frame for each of the plurality of data sources.

24. The apparatus of claim 21, wherein each of the plurality of data sources provides the output data corresponding to at least some of the plurality of vertices.

25. The apparatus of claim 24, wherein:

each of the plurality of data sources comprises at least one digital video camera arranged at a camera position and orientation that is different from the camera position and orientation of another one of the plurality of data sources.

26. A non-transitory computer-readable storage medium storing program instructions that, when executed, causes a processor to:

partition a model of a target space into a plurality of vertices;

determine two or more display attributes associated with each of at least one of the plurality of vertices based on output data observed by a plurality of data sources, wherein each of the two or more display attributes is captured by at least one data source of plurality of data sources, the at least one data source being a different data source relative to each data source that captures each of the other display attributes of the two or more display attributes; and

select one of the two or more display attributes for each of the at least one of the plurality of vertices;

wherein the one of the two or more display attributes is selected based on weights associated with each of the two or more display attributes; and

wherein the weights associated with each of the two or more display attributes are based on a number of the plurality of data sources capturing each of the two or more display attributes.