3D model multi-reviewer system

• US 9,282,321 B2
• Filed: 08/17/2015
• Issued: 03/08/2016
• Est. Priority Date: 02/17/2011
• Status: Expired due to Fees

• Current Assignee: Legend3D, Inc.
• Original Assignee: Legend3D, Inc.
• Inventors: Sandrew, Jared, Baldridge, Tony, Lopez, Anthony, Jones, Scott, Schad, Thomas, McFarland, Jacqueline
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US14/827,482
• Publication Number:
  US 20150358613A1
• Time in Patent Office: 204 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.

Claims

1. A 3D model multi-reviewer system comprising

a 3D model of an environment;

a plurality of review stations, each operated by a reviewer from a plurality of reviewers, each coupled to said 3D model, each comprising

at least one review display;

a reviewer pose subsystem configured to measure or obtain a pose of said reviewer relative to said 3D model;

a viewpoint renderer configured to

receive said pose of said reviewer from said reviewer pose subsystem;

generate at least one image of said 3D model from a viewpoint corresponding to said pose of said reviewer;

display said at least one image on said at least one review display;

a reviewer focus subsystem configured to

calculate or obtain one or more reviewer focus regions within said 3D model, each corresponding to a portion of said 3D model viewed by said reviewer;

a coordinator station operated by a coordinator and coupled to said plurality of review stations and to said 3D model, comprising

at least one coordinator display;

a model overview renderer configured to

generate at least one overview image of said 3D model;

receive said one or more reviewer focus regions from at least one of said plurality of review stations;

add a graphical indicator to said at least one overview image for each of said one or more reviewer focus regions received from said at least one of said plurality of review stations;

display said at least one overview image on said at least one coordinator display.

2. The system of claim 1 wherein a location in said at least one overview image of said graphical indicator corresponds to a location in said 3D model of the reviewer focus region associated with said graphical indicator.

3. The system of claim 1 wherein said graphical indicator includes a differentiating characteristic identifying the reviewer or the review station associated with the graphical indicator.

4. The system of claim 3 wherein said differentiating characteristic comprises one or more of a color, a shape, a pattern, an icon, and a label.

5. The system of claim 1 wherein said one or more reviewer focus regions for each of said plurality of review stations comprise all or a portion of a field of view of the corresponding reviewer.

6. The system of claim 5 wherein said portion of said field of view comprises a center portion of said field of view of said reviewer.

7. The system of claim 6 wherein center portion of said field of view comprises a center point of said field of view of said reviewer.

8. The system of claim 1 wherein said reviewer focus subsystem is further configured to accept a focus input from said reviewer that defines or modifies said one or more reviewer focus regions.

9. The system of claim 1 wherein said coordinator station further comprises

a model update subsystem configured to

accept a 3D model update from said coordinator;

modify said 3D model based on said 3D model update, or record said 3D model update in a list of updates to be applied.

10. The system of claim 1 wherein said coordinator station is further configured to accept a selection of a reviewer focus region from said coordinator;

remove said graphical indicator associated with said reviewer focus region from said at least one overview image.

11. The system of claim 1 further comprising a 3D model generation subsystem configured to obtain a plurality of 2D images of a virtual reality environment from at least one camera;

stitch together said plurality of 2D images into one or more integrated 2D images of said virtual reality environment;

project said one or more integrated 2D images onto a spherical surface, yielding a spherical surface image;

unwrap said spherical surface image onto an unwrapped plane image;

divide said unwrapped plane image into a plurality of regions;

assign depth information to the points of each of said plurality of regions to create said 3D model.

12. The system of claim 9 wherein

said at least one review display comprises one or more stereoscopic 3D displays;

said viewpoint renderer is further configured to

render a left viewpoint image of said 3D model from said pose of said reviewer;

render a right viewpoint image of said 3D model from said pose of said reviewer;

said model update subsystem is further configured to

generate a spherical translation map from said 3D model, wherein

said spherical translation map assigns translation values to points on a spherical surface in said 3D model with a user located at a center of said spherical surface;

each of said translation values is a function of a distance of a closest object in said 3D model from said user along a ray between said user and each of said points on said spherical surface;

apply one or more translation modifications to said spherical translation map to adjust a depth of one or more objects in said three-dimensional model relative to said user;

said viewpoint renderer is further configured to

modify said left viewpoint image and said right viewpoint image to reflect said translation modifications without re-rendering said left viewpoint image and said right viewpoint image from said 3D model.

13. The method of claim 12 wherein said spherical translation map is a parallax map that comprises horizontal pixel offset values between a left and right stereoscopic pair of images.

14. The system of claim 12 wherein said spherical translation map is a depth map.

15. The system of claim 12 wherein said spherical translation map is a U map or a UV map.

16. The system of claim 1 wherein

said at least one coordinator display comprises one or more stereoscopic 3D displays;

said overview renderer is further configured to

generate a left viewpoint panoramic image of said 3D model;

generate a right viewpoint panoramic image of said 3D model.

17. The system of claim 16 wherein said left viewpoint panoramic image and said right viewpoint panoramic image each comprise a 360 degree horizontal field of view.

18. The system of claim 17 wherein said left viewpoint panoramic image and said right viewpoint panoramic image each comprise a 180 degree vertical field of view.

19. A 3D model multi-reviewer system comprising

a 3D model of an environment;

a plurality of review stations, each operated by a reviewer from a plurality of reviewers, each coupled to said 3D model, each comprising

at least one review display;

a reviewer pose subsystem configured to measure or obtain a pose of said reviewer relative to said 3D model;

a viewpoint renderer configured to

receive said pose of said reviewer from said reviewer pose subsystem;

generate at least one image of said 3D model from a viewpoint corresponding to said pose of said reviewer;

display said at least one image on said at least one review display;

a reviewer focus subsystem configured to

calculate or obtain one or more reviewer focus regions within said 3D model, each corresponding to a portion of said 3D model viewed by said reviewer;

wherein said one or more reviewer focus regions comprise all or a portion of a field of view of the said reviewer;

a coordinator station operated by a coordinator and coupled to said plurality of review stations and to said 3D model, comprising

at least one coordinator display;

a model overview renderer configured to

generate at least one overview image of said 3D model;

receive said one or more reviewer focus regions from at least one of said plurality of review stations;

add a graphical indicator to said at least one overview image for each of said one or more reviewer focus regions received from said at least one of said plurality of review stations;

display said at least one overview image on said at least one coordinator display;

a model update subsystem configured to

accept a 3D model update from said coordinator;

modify said 3D model based on said 3D model update, or record said 3D model update in a list of updates to be applied;

wherein

a location in said at least one overview image of said graphical indicator corresponds to a location in said 3D model of the reviewer focus region associated with said graphical indicator;

said graphical indicator includes a differentiating characteristic identifying the reviewer or the review station associated with the graphical indicator;

said differentiating characteristic comprises one or more of a color, a shape, a pattern, an icon, and a label.