MOBILE PLATFORM FOR AUGMENTED REALITY

• US 20120320033A1
• Filed: 06/18/2011
• Published: 12/20/2012
• Est. Priority Date: 06/18/2011
• Status: Active Grant

• Current Assignee: Oxbotica Limited
• Original Assignee: Microsoft Corporation
• Inventors: Papaefstathiou, Efstathios, Sanchez, Russell, Clinton, Nathaniel
• Granted Patent: US 8,902,255 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A mobile image processing manager may include an image data receiving engine configured to obtain a first set of three-dimensional (3-D) image data associated with an observation environment. The mobile image processing manager may also include a navigational plan engine configured to determine a navigational plan based on the first set. A navigation manager may be configured to initiate a navigation event based on the navigational plan. A scene determination engine may be configured to determine a first group of one or more graphical images. An image projection engine may be configured to initiate a display of the first group on a first surface, the display based on a light source.

Claims

1. A system comprising:

a mobile image processing manager including;

an image data receiving engine configured to obtain a first set of three-dimensional (3-D) image data associated with an observation environment;

a navigational plan engine configured to determine a navigational plan based on the first set;

a navigation manager configured to initiate a navigation event based on the navigational plan;

a scene determination engine configured to determine, via a device processor, a first group of one or more graphical images; and

an image projection engine configured to initiate a display of the first group on a first surface, the display based on a light source.

2. The system of claim 1, further comprising:

an audio output engine configured to initiate transmission of an audio output to an audio output device; and

an audio input engine configured to receive audio input.

3. The system of claim 1, wherein:

the scene determination engine is configured to determine the first group based on one or more predetermined graphical character images, and

the navigational plan engine is configured to determine the navigational plan based on the first set and a predetermined navigational control scenario associated with the one or more predetermined graphical character images.

4. The system of claim 1, wherein:

the first surface includes one or more of;

a display surface associated with a user display device, or

a transporting object surface that is located on a transporting object associated with transportation of a 3-D image input device associated with input of the first set, and an image projection device associated with the display of the first group, in a navigational path based on the navigational plan and the initiation of the navigation event.

5. The system of claim 4, wherein:

the scene determination engine is configured to determine the first group based on one or more predetermined graphical virtual environment images, and

the navigational plan engine is configured to determine the navigational plan based on one or more navigational user requests.

6. The system of claim 4, wherein:

the scene determination engine is configured to determine the first group based on combining one or more predetermined graphical virtual environment images and at least a portion of the first set, and

the navigational plan engine is configured to determine the navigational plan based on one or more navigational user requests.

7. The system of claim 4, wherein:

the user display device includes one or more of;

a television display, a mobile device display, a computer display device, or a remote projector display device.

8. The system of claim 1, wherein:

the image data receiving engine is configured to obtain a second set of three-dimensional 3-D image data associated with the observation environment,

the scene determination engine is configured to determine a second group of one or more graphical images, based on a reaction to an event associated with the obtained second set of three-dimensional 3-D image data,

the image projection engine is configured to initiate a display of the second group on the first surface, and

the first surface includes a portion of the observation environment.

9. The system of claim 8, wherein:

the scene determination engine is configured to determine the first group based on one or more predetermined graphical virtual environment images, and

the navigational plan engine is configured to determine the navigational plan based on one or more navigational user requests.

10. The system of claim 8, wherein:

the scene determination engine is configured to determine the first group based on image data received from an image data transmitter, and

the navigational plan engine is configured to determine the navigational plan based on one or more navigational requests received from a navigational request transmitter.

11. The system of claim 1, further comprising:

an orientation determination engine configured to determine a 3-D orientation of a first bounded 3-D area, based on the first set, wherein

the scene determination engine is configured to determine the first group of one or more graphical images based on the determined 3-D orientation, and

the image projection engine is configured to initiate the display of the first group to the first surface, the display based on the light source and the determined 3-D orientation.

12. The system of claim 11, further comprising:

a matching engine configured to determine a match between a portion of the first bounded 3-D area and one or more of the graphical images, based on the determined 3-D orientation, wherein

the image projection engine is configured to initiate the display of the first group to the first surface, the display based on the light source, the determined 3-D orientation, and the determined match.

13. The system of claim 12, wherein:

the matching engine is configured to determine the match between the portion of the first bounded 3-D area and one or more of the graphical images, based on the determined 3-D orientation, based on one or more 3-D geometric similarities between the one or more graphical images and the portion of the first bounded 3-D area, wherein

the image projection engine is configured to initiate the display of the first group to the first surface, the display based on a projection that is based on the light source, the determined 3-D orientation, and fitting the projection to the first surface based on the 3-D geometric similarities.

14. The system of claim 11, wherein:

the observation environment includes the first bounded 3-D area, and

the determined 3-D orientation includes one or more of;

a contour associated with the observation environment, a color associated with the observation environment, a theme associated with the observation environment, or a 3-D geometric configuration associated with an observation object located in the observation environment, wherein

the image projection engine is configured to initiate the display of the first group to the first surface, the display based on fitting a portion of the projection to a 3-D geometric configuration associated with a portion of the first surface.

15. A method comprising:

obtaining a first set of three-dimensional (3-D) image data associated with an environment and a 3-D image input device;

determining, via a device processor, a navigational plan based on the first set, the navigational plan associated with navigation of a transporting device associated with the 3-D image input device;

determining a first 3-D geometric arrangement associated with a plurality of objects included in the environment, based on the first set;

determining a first group of one or more graphical images, based on the first 3-D arrangement; and

initiating a projection display of the first group to one or more surfaces associated with one or more of the plurality of objects, the display based on a light source and the first 3-D geometric arrangement.

16. The method of claim 15, wherein:

the navigational plan is associated with navigation of a transporting device associated with the 3-D image input device and a projection device associated with the projection display of the first group.

17. A computer program product tangibly embodied on a computer-readable medium and including executable code that is configured to cause at least one data processing apparatus to:

obtain a first set of three-dimensional (3-D) image data associated with an observation environment and a 3-D image input device;

determine a navigational plan based on the first set, the navigational plan associated with navigation of a transporting device associated with the 3-D image input device;

determine a first 3-D geometric arrangement associated with a plurality of objects included in the environment, based on the first set;

determine a first group of one or more graphical images, based on the first 3-D arrangement; and

initiate a display of the first group to one or more surfaces, the display based on a light source and the first 3-D geometric arrangement.

18. The computer program product of claim 17, wherein the executable code is configured to cause the at least one data processing apparatus to:

initiate a navigation event associated with the transporting device, based on the navigational plan, wherein

the first set is obtained from the 3-D image input device, and

the initiating the display of the first group includes initiating the display via a projector device, wherein

the transporting device is associated with transportation of the 3-D image input device and the projector device in a navigational path based on the navigational plan and the initiation of the navigation event.

19. The computer program product of claim 17, wherein the executable code is configured to cause the at least one data processing apparatus to:

receive a request for a projection of a first scene based on a first scene description, wherein

the determining the first group includes determining the first group of one or more graphical images, based on the first 3-D geometric arrangement, and based on the first scene description.

20. The computer program product of claim 19, wherein:

the first scene description includes a description associated with environment attributes of a 3-D virtual scene that is based on a real-world environment.