Invoking applications with virtual objects on an interactive display

• US 20050251800A1
• Filed: 05/05/2004
• Published: 11/10/2005
• Est. Priority Date: 05/05/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Dehlin, Joel P., Kurlander, David Joshua
• Granted Patent: US 7,467,380 B2
• Time in Patent Office: Days
• US Class Current: 717/174
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

One or more objects placed on a display surface of an interactive display system are identified by a camera that receives infrared light reflected back by the object(s) through the display surface. The interactive display system displays images on the display surface that are produced by software applications, as well as sensing objects placed proximate to the display surface. If object(s) that were placed on the display surface are identified and determined to be associated with a predefined software application, the application is automatically executed. Objects can be identified based upon shape, a pattern applied to the objects, a size of the object, or a location on the display surface where the object is placed. The object may be used in the software application after it is executed or may simply be bear a functional relationship to the software application.

Claims

1. A method for executing a software application in response to at least one object being placed on a display surface, comprising the steps of:

(a) detecting at least one characteristic of each said at least one object placed on the display surface, to identify each said at least one object as a function of said at least one characteristic thereof; and

(b) determining if one object identified is uniquely associated with executing only one software application, and if so, executing said one software application if already installed for use by a current user of the display surface, and if not, determining if a plurality of objects that have been placed on the display surface and have been identified are uniquely associated with executing only one software application, and if so, executing said one software application if installed for use by the current user of the display surface.

2. The method of claim 1, wherein if the software application has not been installed for use, further comprising the steps of:

(a) connecting to a predefined remote storage over a network, said predefined storage storing the software application;

(b) downloading the software application from the predefined remote storage over the network;

(c) installing the software application for use with the display surface; and

(d) executing the software application.

3. The method of claim 2, wherein the step of installing includes the step of registering the software application.

4. The method of claim 2, further comprising the step of enabling a current user of the display surface to enter information relating to one of a debit and a credit account for use in making online purchases, said information being stored locally.

5. The method of claim 4, further comprising the step of enabling the current user of the display surface to purchase the software application prior to downloading it, by automatically transmitting the information relating to said one of the debit account and credit account to an entity associated with the remote storage.

6. The method of claim 1, wherein the step of detecting comprises the steps of:

(a) illuminating said at least one object placed on the display surface with light that passes through the display surface and is incident on said at least one object placed on the display surface;

(b) receiving light reflected from said at least one object placed on the display surface;

(c) producing a signal corresponding to the light that is received;

(d) processing the signal to determine said at least one characteristic for each object placed on the surface, to identify each said at least one object.

7. The method of claim 1, wherein said at least one characteristic comprises at least one of:

(a) a shape of the object;

(b) a size of the object;

(c) a color of the object;

(d) an electrical characteristic of the object;

(e) a radio frequency signature of the object;

(f) a location where the object is placed on the display surface; and

(g) a pattern disposed on the object.

8. The method of claim 1, further comprising the step of employing said at least one object within the software application after the software application has been executed.

9. The method of claim 1, wherein said at least one object is related to a function of the software application.

10. The method of claim 1, wherein said at least one object comprises a component provided with the packaging in which the software application was sold, further comprising the step of detecting indicia on the component that identify the software application to be executed, and in response, executing the software application.

13. A memory medium on which machine executable instructions are stored for executing the steps of claim 1.

11. The method of 1, wherein if said at least one object is associated with a plurality of software applications, further comprising the steps of:

(a) indicating the plurality of applications with which said at least one object is associated to the current user; and

(b) enabling the current user to select one of the plurality of applications to be executed.

12. The method of claim 11, further comprising the step of indicating a subset of the plurality of software applications when an additional object is placed on the display surface, said subset of the plurality of software application being associated with said at least object previously placed on the display surface and with said additional object just placed on the display surface.

14. A system for executing a software application in response to at least one object being identified, comprising:

(a) an interactive display system including a projector, an object detector, and a display surface on which images produced by the software application are displayed by the projector when the software application is executed;

(b) a memory in which are stored machine instructions;

(c) a processor that is coupled to the interactive display system and to the memory, said processor executing the machine instructions, which cause the processor to carry out a plurality of functions, including;

(i) using the object detector to detect at least one characteristic of each said at least one object after said at least one object is placed on the display surface, each said at least one object being identified as a function of said at least one characteristic thereof that is detected; and

(ii) determining if one object that is identified is uniquely associated with executing only one software application, and if so, executing the software application if the software application has been installed on the interactive display system, and if not, determining if a plurality of objects placed on the display surface and identified are uniquely associated with executing only one software application, and if so, executing the software application if the software application has been installed on the interactive display system.

15. The system of claim 14, wherein if the software application is not stored in the memory because it has not yet been installed on the interactive display, the machine instructions further cause the processor to:

(a) connect to a predefined remote storage over a network, said predefined remote storage storing the software application;

(b) download the software application from the predefined remote storage over the network;

(c) install the software application for use with the display surface; and

(d) execute the software application.

16. The system of claim 15, wherein the machine instructions further cause the processor to register the software application.

17. The system of claim 15, wherein the machine instructions further cause the processor to store information relating to one of debit account and a credit account in the memory, for use in making online purchases, including an online purchase of the software application.

18. The system of claim 17, wherein the machine instructions further cause the processor to enable a current user of the interactive display to purchase the software application prior to downloading it over the network, by automatically transmitting the information relating to said one of the debit account and credit account to an entity associated with the remote storage.

19. The system of claim 14, wherein the interactive display further includes a light source disposed under the display surface that illuminates said at least one object placed on the display surface with light that passes through the display surface and is incident on said at least one object, said object sensor comprising a camera that receives light reflected from said at least one object placed on the display surface and produces a signal corresponding to the light that is received, said signal being coupled to the processor, and said machine instructions further causing the processor to determine said at least one characteristic for each object placed on the surface, to identify each said at least one object.

20. The system of claim 14, wherein said at least one characteristic comprises at least one of:

(a) a shape of the object;

(b) a size of the object;

(c) a color of the object;

(d) an electrical characteristic of the object;

(e) a radio frequency signature of the object;

(f) a location where the object is placed on the display surface; and

(g) a pattern disposed on the object.

21. The system of claim 14, wherein said at least one object is employed within the software application after the software application has been executed.

22. The system of claim 14, wherein said at least one object is related to a function of the software application.

23. The system of claim 14, wherein said at least one object comprises a component provided with the packaging in which the software application was sold, further comprising the step of detecting indicia on the component that identify the software application to be executed, and in response, executing the software application.

24. The system of claim 14, wherein the light source emits infrared (IR) light, and wherein the object detector comprises an IR bandpass filter that transmits IR light, but substantially blocks other light, so that said at least one characteristic of each said at least one object is indicated in an image formed by the object detector using IR light reflected back by the object and received by the object detector after passing through the IR bandpass filter.

25. The system of claim 23, wherein the machine instructions cause the processor to determine a shape of each object placed on the display surface using the image formed by the object detector, said shape comprising a characteristic of the object used to identify the object.

26. The system of claim 24, wherein the machine instructions cause the processor to compare the shape of each object placed on the display surface with a corresponding shape stored in memory and associated with the software program that is to be executed.

27. The system of claim 14, wherein if said at least one object is associated with a plurality of software applications, the machine instructions further cause the processor to:

(a) indicate the plurality of applications with which said at least one object is associated; and

(b) enable a user to select one of the plurality of applications to be executed.

28. The system of claim 27, wherein the machine instructions cause the processor to indicate a subset of the plurality of software applications when an additional object is placed on the display surface, said subset of the plurality of software application being associated with said at least object previously placed on the display surface and with said additional object just placed on the display surface.