IN-SCENE REAL-TIME DESIGN OF LIVING SPACES

• US 20140132595A1
• Filed: 11/14/2012
• Published: 05/15/2014
• Est. Priority Date: 11/14/2012
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Siddiqui, Matheen, Pradeep, Vivek, Dietz, Paul, Bathiche, Steven, Boulanger, Catherine N.
• Application Number:
  US13/676,151
• Publication Number:
  US 20140132595A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/60 of natural phenomena, e.g. rain, snow, water or plants
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 7/564 from contours
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A display that renders realistic objects allows a designer to redesign a living space in real time based on an existing layout. A computer system renders simulated objects on the display such that the simulated objects appear to the viewer to be in substantially the same place as actual objects in the scene. The displayed simulated objects can be spatially manipulated on the display through various user gestures. A designer can visually simulate a redesign of the space in many ways, for example, by adding selected objects, or by removing or rearranging existing objects, or by changing properties of those objects. Such objects also can be associated with shopping resources to enable related goods and services to be purchased, or other commercial transactions to be engaged in.

Claims

1. A computer-implemented process comprising:

receiving image data of a scene into memory, the image data being received from one or more cameras in a known relationship with a display;

generating models of actual objects in the scene, the models defining simulated objects corresponding to the actual objects in the scene;

detecting a viewer'"'"'s orientation with respect to the display; and

rendering the simulated objects on the display such that the simulated objects appear to the viewer to be in substantially a same place as the actual objects in the scene and such that the displayed simulated objects are can be manipulated spatially on the display through user gestures.

2. The computer-implemented process of claim 1, wherein generating models of actual objects comprises:

detecting contours of an actual object in the image data; and

converting the detected contours into a simulated object corresponding to the actual object.

3. The computer-implemented process of claim 2, wherein generating models of actual objects comprises:

receiving input from a user selecting the detected contours that define the actual object.

4. The computer-implemented process of claim 3, further comprising receiving data descriptive of the actual object and storing the received data in association with the corresponding simulated object.

5. The computer-implemented process of claim 1, wherein rendering the simulated objects on the display such that the simulated objects appear to the viewer to be in substantially a same place as the actual objects in the scene, comprises rendering the simulated objects according to a view dependent depth corrected gaze.

6. The computer-implemented process of claim 1, further comprising:

modifying a displayed simulated object in response to a user gesture with respect to the display.

7. The computer-implemented process of claim 1, further comprising:

associating items available for purchase with the simulated objects.

8. The computer-implemented process of claim 7, further comprising:

receiving information describing a purchase of an item associated with a simulated object.

9. The computer-implemented process of claim 1, wherein rendering the simulated objects on the display such that the simulated objects appear to the viewer to be in substantially a same place as the actual objects in the scene, comprises:

generating a three-dimensional model of the scene;

determining a pose of the display with respect to the scene;

determining a pose of the viewer with respect to the display; and

rendering the scene on the display according to the determined poses of the display and the viewer.

10. The computer-implemented process of claim 1, further comprising:

simulating changes in a simulated object over time.

11. The computer-implemented process of claim 1, further comprising:

adding a simulated object to the display in response to a user gesture.

12. The computer-implemented process of claim 1, further comprising:

scaling and rotating the simulated object to match size and orientation of objects in the scene.

13. The computer-implemented process of claim 1, further comprising:

simulating changes in lighting on the simulated objects displayed on the display.

14. The computer-implemented process of claim 1, further comprising:

simulating changes in surface properties of the simulated objects displayed on the display.

15. An article of manufacture comprising:

computer storage;

computer program instructions stored on the computer storage which, when processed by a processing device, instruct the processing device to perform a process comprising;

receiving image data of a scene into memory, the image data being received from one or more cameras in a known relationship with a display;

generating models of actual objects in the scene, the models defining simulated objects corresponding to the actual objects in the scene;

detecting a viewer'"'"'s orientation with respect to the display; and

rendering the simulated objects on the display such that the simulated objects appear to the viewer to be in substantially a same place as the actual objects in the scene and such that the displayed simulated objects can be manipulated spatially on the display through user gestures.

16. The article of manufacture of claim 15, wherein the process further comprises:

modifying a displayed simulated object in response to a user gesture with respect to the display.

17. The article of manufacture of claim 15, wherein the process further comprises:

associating items available for purchase with the simulated objects.

18. A design system comprising:

inputs for receiving image data of a scene;

an object modeling system having inputs for receiving data describing the scene and an output providing simulated objects corresponding to actual objects in the scene;

a rendering system having inputs for receiving the simulated objects and outputting to a display a rendering of the scene such that simulated objects appear to the viewer to be in substantially a same place as the actual objects in the scene; and

an input system enabling a user to spatially manipulate the displayed simulated objects on the display.

19. The design system of claim 18, further comprising:

an input system that processes user input with respect to one or more simulated objects and modifies one or more properties of the simulated objects.

20. The design system of claim 18, further comprising:

a purchasing system that processes user input with respect to one or more simulated objects and presents goods or services available for purchase related to the simulated objects.