Diminished reality

• US 9,514,573 B2
• Filed: 12/26/2013
• Issued: 12/06/2016
• Est. Priority Date: 12/26/2013
• Status: Active Grant

• Current Assignee: Dassault Systemes SA
• Original Assignee: Dassault Systemes SA
• Inventors: Grimaud, Jean-Jacques
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US14/141,243
• Publication Number:
  US 20150187136A1
• Time in Patent Office: 1,076 Days
• Field of Search: 345/419, 345/420, 345/631, 345/632
• US Class Current: 1/1
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 7/564 from contours

Abstract

A computer tool generates user-defined diminished reality images of a subject environment from source images. The diminished reality images display less real-world objects in the subject environment than that displayed in the source images. A 3D model of the subject environment in diminished reality is formed from the diminished reality images. The 3D model supports augmented reality user interaction with views of the subject environment diminished in reality and tidied/decluttered of user-selected objects.

Claims

1. A computer method of displaying a subject environment comprising:

digitally generating a set of one or more images that form a first view depicting a subject environment in an origination state instead of in a current state, such that the first view is a three-dimensional (3D) diminished reality of the subject environment by;

receiving a two-dimensional (2D) source image displaying the subject environment in the current state,

determining line segments defining the borders of the subject environment based on line segments in the 2D source image and line segments extrapolated from the 2D source image, the determined line segments being a background of the subject environment, and

digitally forming a 3D model of the 3D diminished reality of the subject environment, the 3D model being based on the background of the subject environment and having objects of the 2D source image that are separate from the background removed from the 3D model; and

enabling computer-based user interaction with the set of images forming the first view in a manner that generates a second view, the second view being a 3D augmented reality representation of the subject environment in its origination state having augmentations based on the enabled computer-based user interaction instead of the current state.

2. The computer method as claimed in claim 1 wherein the first view depicts a lesser amount of real-world objects in the subject environment than amount of real-world objects displayed in the 2D source image.

3. The computer method as claimed in claim 1 wherein the first view depicts the subject environment being bare of objects.

4. The computer method as claimed in claim 1 wherein enabling computer-based user interaction includes providing a user interface to an augmented reality subsystem.

5. The computer method as claimed in claim 1 wherein the generated second view superimposes one or more virtual elements on the first view, each virtual element representing a respective real-world object.

6. The computer method as claimed in claim 1 wherein the subject environment is a room;

and

the first view depicts the room being substantially empty contrary to the current state of the room.

7. The computer method as claimed in claim 6 wherein the second view displays an augmented reality representation of the room with one of:

one or more objects in the room that are absent from the current state of the room, and

one or more objects in the room that are different from the current state of the room.

8. The computer method as claimed in claim 7 wherein the one or more objects are any of:

furniture, furnishings, amenities and accessories.

9. The computer method as claimed in claim 1 wherein for each of a plurality of objects in the subject environment, display of the object in the first view is user-selectable.

10. The computer method as claimed in claim 1 further comprising:

creating a 3D model of the subject environment in diminished reality using the set of images forming the first view, wherein the 3D model is employed in the augmented reality representation of the subject environment.

11. A computer apparatus for displaying a subject environment, comprising:

a reality staging engine executable by a processor and configured to generate a set of one or more images that form a first view depicting a subject environment in an origination stage other than a current stage by;

receiving a two-dimensional (2D) source image displaying the subject environment in the current state, and

determining line segments defining the borders of the subject environment based on line segments in the 2D source image and line segments extrapolated from the 2D source image, the determined line segments being a background of the subject environment;

a 3D modeler coupled to receive the generated set of images from the reality staging engine, and the 3D modeler configured to form a 3D model of the subject environment in diminished reality by digitally forming a 3D model of the 3D diminished reality of the subject environment, the 3D model being based on the background of the subject environment and having objects of the 2D source image that are separate from the background removed from the 3D model; and

an augmented reality interface enabling user interaction with an augmented reality representation of the subject environment formed from the 3D model in its origination state having augmentations based on the enabled computer-based user interaction instead of the current state.

12. The computer apparatus as claimed in claim 11, wherein the first view depicts a lesser amount of real-world objects in the subject environment than amount of real-world objects displayed in the 2D source image, such that amount of real-world objects in the 3D model of the subject environment is lesser than amount of real-world objects displayed in the 2D source image.

13. The computer apparatus as claimed in claim 11 wherein the augmented reality interface superimposes one or more virtual elements on views of the 3D model, each virtual element representing a respective real-world object.

14. The computer apparatus as claimed in claim 11 wherein the subject environment is a room, and the first view depicts the room being substantially empty.

15. The computer apparatus as claimed in claim 14 wherein the augmented reality representation of the room is displayed with one of:

one or more objects in the room that are absent from the current stage of the room, and

one or more objects in the room that are different from the current stage of the room.

16. The computer apparatus as claimed in claim 15 wherein the objects include any of:

furniture, furnishings, amenities and accessories.

17. A computer program product for displaying a subject environment, comprising:

a non-transitory computer readable medium; and

computer code embodied on the computer readable medium and executable by a processor to display a subject environment by;

digitally generating a set of one or more images that form a first view depicting the subject environment in a user-selectable stage different from a current stage of the subject environment, such that the first view is a diminished reality of the subject environment by;

receiving a two-dimensional (2D) source image displaying the subject environment in the current state,

determining line segments defining the borders of the subject environment based on line segments in the 2D source image and line segments extrapolated from the 2D source image, the determined line segments being a background of the subject environment, and

digitally forming a 3D model of the 3D diminished reality of the subject environment, the 3D model being based on the background of the subject environment and having objects of the 2D source image that are separate from the background removed from the 3D model; and

providing an augmented reality interface applied to the set of images forming the first view in a manner that generates a second view, the second view being an augmented reality representation of the subject environment in its user-selectable stage having augmentations based on the enabled computer-based user interaction instead of the current stage.