Personal Media Landscapes in Mixed Reality

• US 20100208033A1
• Filed: 02/13/2009
• Published: 08/19/2010
• Est. Priority Date: 02/13/2009
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Edge, Darren K., Min, Kyungmin, Chang, Eric
• Application Number:
  US12/371,431
• Publication Number:
  US 20100208033A1
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

An exemplary method includes accessing geometrically located data that represent one or more virtual items with respect to a three-dimensional coordinate system; generating a three-dimensional map based at least in part on real image data of a three-dimensional space as acquired by a camera; rendering to a physical display a mixed reality scene that includes the one or more virtual items at respective three-dimensional positions in a real image of the three-dimensional space acquired by the camera; and re-rendering to the physical display the mixed reality scene upon a change in the field of view of the camera. Other methods, devices, systems, etc., are also disclosed.

Claims

1. An application, executable on a computing device, the application comprising:

a mapping module configured to access real image data of a three-dimensional space as acquired by a camera and to generate a three-dimensional map based at least in part on the accessed real image data;

a data module configured to access stored geometrically located data that represent one or more virtual items with respect to a three-dimensional coordinate system; and

a rendering module configured to render graphically the one or more virtual items of the geometrically located data, with respect to the three-dimensional map, along with real image data acquired by the camera of the three-dimensional space to thereby provide for a displayable mixed reality scene.

2. The application of claim 1 further comprising a tracking module configured to track field of view of the camera in real-time to thereby provide for three-dimensional navigation of the displayable mixed reality scene.

3. The application of claim 1 further comprising a screen capture module configured to capture a displayed screen for subsequent rendering in a mixed reality scene to thereby avoid a feedback loop between a camera and a screen.

4. The application of claim 1 further comprising an insertion module configured to insert and geometrically locate one or more virtual items in a mixed reality scene.

5. The application of claim 1 further comprising an edit module configured to edit or relocate one or more virtual items in a mixed reality scene.

6. The application of claim 1 further comprising a command module configured to receive commands from one or more input devices to thereby control operation of the application.

7. The application of claim 6 wherein the one or more input devices comprise at least one member selected from a group consisting of a keyboard, a camera, a microphone, a mouse, a trackball and a touch screen.

8. The application of claim 1 wherein the mapping module is configured to access real image data of a three-dimensional space as acquired by a camera selected from a group consisting of a webcam, a mobile phone camera, and a head-mounted camera.

9. The application of claim 1 wherein the mapping module is configured to access real image data of a three-dimensional space as acquired by a stereo camera.

10. The application of claim 1 further comprising a geography module configured to geographically locate the three-dimensional space.

11. The application of claim 1 wherein the data module is configured to access, via a network, geometrically located data stored a remote site.

12. A system comprising:

a camera with a changeable field of view;

a display; and

a computing device that comprises at least one processor, memory, an input for the camera, an output for the display and control logic to generate a three-dimensional map based on real image data of a three-dimensional space acquired by the camera via the input, to locate one or more virtual items with respect to the three-dimensional map, to render a mixed reality scene to the display via the output wherein the mixed reality scene comprises the one or more virtual items along with real image data of the three-dimensional space acquired by the camera and to re-render the mixed reality scene to the display via the output upon a change in the field of view of the camera.

13. The system of claim 12 wherein the camera comprises a field of view changeable by manual movement of the camera, by head movement of the camera or by sensing movement wherein the sensing comprises at least one member selected from a group consisting of sensing by computing optical flow, sensing by using one or more gyroscopes mounted on the camera, and by using position sensors that compute the relative position of the camera and the front of view of the camera.

14. The system of claim 12 wherein the camera comprises a field of view changeable by zooming.

15. The system of claim 12 further comprising control logic to store, as geometrically located data, data representing one or more virtual items located with respect to a three-dimensional coordinate system.

16. The system of claim 12 comprising a mobile computing device that comprises a built in camera and a built in display.

17. A method, implemented at least in part by a computing device, the method comprising:

accessing geometrically located data that represent one or more virtual items with respect to a three-dimensional coordinate system;

generating a three-dimensional map based at least in part on real image data of a three-dimensional space as acquired by a camera;

rendering to a physical display a mixed reality scene that comprises the one or more virtual items at respective three-dimensional positions in a real image of the three-dimensional space acquired by the camera; and

re-rendering to the physical display the mixed reality scene upon a change in the field of view of the camera.

18. The method of claim 17 further comprising issuing a command to target one of the one or more virtual items in the mixed reality scene.

19. The method of claim 17 further comprising locating another virtual item in the mixed reality scene and storing data representing the virtual item with respect to a location in a three-dimensional coordinate system.

20. One or more processor-readable media comprising processor executable-instructions for performing the method of claim 17.