Projection of graphical objects on interactive irregular displays

• US 8,884,883 B2
• Filed: 05/19/2008
• Issued: 11/11/2014
• Est. Priority Date: 01/25/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benko, Hrvoje, Wilson, Andrew
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): RICKS, DONNA J
• Application Number:
  US12/122,745
• Publication Number:
  US 20090189917A1
• Time in Patent Office: 2,367 Days
• Field of Search: None
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

A method for displaying images on a curved display surface is described herein. The method includes receiving a graphical object and distorting the graphical object at run-time such that an appearance of the graphical object on the curved display surface will be substantially similar regardless of a position of the graphical object on the curved display surface when viewed at a viewing axis that is approximately orthogonal to a plane that is tangential to the curved display surface at a center of the graphical object. The method may further include displaying the graphical object on the curved display surface.

Claims

1. A system that facilitates projecting images onto an exterior surface of a diffuse curved display, the diffuse curved display defining an interior region, the system comprising:

a processor; and

a memory, the memory comprising a plurality of components that are executed by the processor, the components comprising;

a receiver component that receives a graphical object that is to be displayed on the exterior surface of the diffuse curved display in a multi-touch interactive environment, the graphical object formatted for display on a two-dimensional display surface;

a distorter component that distorts the graphical object at run-time of an application that uses the graphical object, the distorter component distorts the graphical object for display on the exterior surface of the diffuse curved display; and

a detector component that detects a position on the exterior surface that a member is in physical contact with the exterior surface of the diffuse curved display, the detector component detecting the position of the member based upon an image captured by the detector component, the image depicting light emitted from the interior region by an emitter, that has passed through the diffuse curved display and been reflected from the member back through the diffuse curved display, wherein the distorter component distorts the graphical object for display on the exterior surface of the diffuse curved display based at least in part upon the detected position.

2. The system of claim 1, further comprising a projector that projects the at least one graphical object onto the exterior surface of the diffuse curved display.

3. The system of claim 2, wherein the diffuse curved display comprises an interior surface, and wherein the projector causes the graphical object to be displayed on the exterior surface of the diffuse curved display by having visible light initially contact the interior surface of the diffuse curved display.

4. The system of claim 2, further comprising the emitter, the light emitted by the emitter being infrared light.

5. The system of claim 1, wherein the diffuse curved display is a spherical display.

6. The system of claim 1, wherein the graphical object is one of a planar graphical object or a three-dimensional graphical object.

7. The system of claim 1, wherein the diffuse curved display is spherical, and the components further comprising:

a scaler component that determines a desired scale of the graphical object with respect to a representation of a sphere;

an orientor component that determines a desired orientation of the graphical object with respect to the representation of the sphere; and

a position determiner component that determines a position on the representation of the sphere that corresponds to where the graphical object is desirably displayed on the curved display surface, wherein the distorter component distorts the graphical object at run-time of the application that uses the graphical object based at least in part upon the determined scale, the determined orientation, and the determined position.

8. The system of claim 7, the components further comprising:

an illuminator component that orients the graphical object in accordance with the desired orientation of the graphical object on the representation of the sphere;

a projector component that projects the graphical object onto the representation of the sphere;

a rotator component that rotates the representation of the sphere until the graphical object is at the desired position; and

a flattener component that flattens the computer-implemented display surface onto a projection disk that is to be projected on the exterior surface of the diffuse curved display by a projector.

9. The system of claim 1, wherein the detector component detects multiple positions of multiple members of the user in physical contact with the exterior surface of the diffuse curved display, and wherein the distorter component distorts the graphical object for display on the exterior surface of the diffuse curved display based at least in part upon the detected multiple positions.

10. The system of claim 1, wherein the graphical object is a portion of a larger graphical object that includes multiple graphical objects, and wherein the distorter component distorts each of the graphical objects in the larger graphical object at runtime of the application.

11. A method for displaying images on an exterior surface of a diffuse curved display, the diffuse curved display defining an interior region, the method comprising:

receiving a graphical object that is formatted for display on a two-dimensional display;

at run time of an application that desirably displays the graphical object, distorting the graphical object for display on the exterior surface of the diffuse curved display, wherein distorting the graphical object comprises causing an appearance of the graphical object on the exterior surface of the diffuse curved display to be identical regardless of a position of the graphical object on the exterior surface of the diffuse curved display when viewed at a viewing axis that is orthogonal to a plane that is tangential to the diffuse curved display at a center of the graphical object;

displaying the graphical object on the curved display surface;

detecting a location on the exterior surface of the diffuse curved display where a member is in contact with the exterior surface of the diffuse curved display, the detecting comprising;

emitting infrared light from the interior region, the infrared light passes through the diffuse curved display; and

receiving an image of an interior surface of the diffuse curved display, the image comprising a portion where the infrared light has reflected from the member back through the diffuse curved display;

receiving user input by way of the member coming into contact with the exterior surface of the diffuse curved display;

receiving user input by way of a second member coming into contact with the exterior surface of the diffuse curved display simultaneously with the member coming into contact with the exterior surface of the diffuse curved display; and

displaying the graphical object on the exterior surface of the diffuse curved display based at least in part upon a position of the member on the exterior surface of the diffuse curved display and a position of the second member on the exterior surface of the diffuse curved display.

12. The method of claim 11, wherein the graphical object is rectangular, and wherein the graphical object is distorted such that lines of the graphical object appear to be straight on the exterior surface of the diffuse curved display when viewed at a viewing axis that is approximately orthogonal to a plane that is tangential to the exterior surface of the diffuse curved display at the center of the graphical object.

13. The method of claim 11, wherein the graphical object is tessellated.

14. The method of claim 11, wherein the graphical object comprises text.

15. The method of claim 11, wherein the diffuse curved display is a spherical display, and wherein the act of distorting comprises:

determining a scale of the graphical object with respect to a representation of a sphere;

determining a desired orientation of the graphical object with respect to the representation of the sphere;

determining a desired position of the graphical object with respect to the representation of the sphere;

orienting the graphical object with respect to the representation of the sphere such that a center of the graphical object corresponds to a pole of the representation of the sphere;

projecting the graphical object onto the representation of the sphere;

rotating the representation of the sphere such that the graphical object is positioned in accordance with the desired position; and

flattening the spherical representation and the graphical object onto a disk that is to be projected onto the spherical display.

16. The method of claim 11, wherein the graphical object is a portion of a larger graphical object that includes several other graphical objects, and wherein the acts of distorting and displaying are performed for each of the graphical objects in the larger graphical object.

17. The method of claim 11, wherein the diffuse curved display comprises an interior surface, and wherein the graphical object is displayed on the exterior surface of the diffuse curved display by projecting light onto the interior surface of the diffuse curved display.

18. The method of claim 11, further comprising:

emitting infrared light through the diffuse curved display;

detecting infrared light reflected from the member back through the diffuse curved display; and

detecting that the member has come into contact with the exterior surface of the diffuse curved display based at least in part upon the detecting that the infrared light has reflected from the member.

19. A display apparatus comprising a computer-readable memory, the memory comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

executing an application retained in the computer-readable memory, the application configured to display graphical objects on a two-dimensional display screen;

receiving, from the application, a graphical object that is to be displayed on an exterior surface of a diffuse spherical display in a multi-touch interactive environment, the graphical object formatted for display on the two-dimensional display screen;

distorting the graphical object at run-time of the application such that the graphical object is formatted for display on the exterior surface of the diffuse spherical display; and

detecting a position on the exterior surface of the diffuse spherical display that a member is in physical contact with the exterior surface of the diffuse spherical display, the graphical object is distorted for display based at least in part upon the position, and wherein detecting the position of the member comprises;

receiving an infrared image of an interior surface of the diffuse spherical display, the infrared image comprising a portion corresponding to a location on the exterior surface of the diffuse spherical display where the member is in physical contact with the exterior surface of the diffuse spherical display, the portion indicating where infrared light emitted by an emitter, has passed through the diffuse spherical display and been reflected from the member back through the diffuse spherical display.

20. The display apparatus of claim 19, the acts further comprising:

detecting a second position on the exterior surface of the diffuse spherical display that a second member is in physical contact with the exterior surface of the diffuse spherical display, the member and the second member simultaneously being in physical contact with the exterior surface of the diffuse spherical display; and

distorting the graphical object for display on the exterior surface of the diffuse spherical display based at least in part upon the position of the member on the exterior surface of the diffuse spherical display and the second position of the second member on the exterior surface of the diffuse spherical display.