INTERACTIVE PROJECTION SYSTEM

• US 20110169778A1
• Filed: 01/07/2011
• Published: 07/14/2011
• Est. Priority Date: 01/08/2010
• Status: Active Grant

• Current Assignee: Crayola LLC (Hallmark Cards Incorporated)
• Original Assignee: Crayola LLC (Hallmark Cards Incorporated)
• Inventors: WEISS, STEPHEN, NUNGESTER, GREGORY R.
• Granted Patent: US 8,842,096 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

An interactive projection system, apparatus, and methods for their use are provided. The system includes a projection unit that projects an image of a user interface onto a projection surface. At least one vision sensor is provided that is configured to sense a location of an infrared light source. A computing device that is configured to execute applications to provide the user interface and to determine corresponding outputs based on inputs received by the vision sensor is also provided. An infrared input device, or stylus, is employed by a user to provide infrared light inputs at desired locations with respect to the projected user interface. Additionally, a plurality of styli, each emitting a different respective wavelength of infrared light is also useable to provide multiple simultaneous inputs to the system. An equal plurality of vision sensors may also be employed to separately sense inputs from a respective stylus.

Claims

1. An interactive projection system comprising:

a projection unit that projects at least one image onto a projection surface;

a vision sensor configured to sense the location of an infrared light source;

a computing device having one or more processors and a memory that executes a computer application that provides a user interface image that is projected by the projection unit onto the projection surface; and

an infrared input device that is configured to emit an infrared light upon activation by a user, the infrared light providing input to the computing device via the vision sensor,

wherein the computing device identifies a location of the infrared light with respect to the user interface image projected onto the projection surface and provides a corresponding output.

2. The projection system of claim 1, wherein the vision sensor is calibrated to an area inclusive of the user interface image projected onto the projection surface.

3. The projection system of claim 2, wherein the corresponding output includes one or more of updating the user interface image, providing a audible output, and performing an executable task.

4. The projection system of claim 1, further comprising a removable computer memory, the removable computer memory comprising one or more of a universal serial bus (USB) flash drive, a Secure Digital (SD) memory card, a CompactFlash memory card, a Memory Stick, and an xD memory card.

5. The projection system of claim 1, further comprising:

a first filter associated with the vision sensor that allows only infrared light of a first predetermined range of wavelengths to contact the vision sensor;

a second vision sensor capable of sensing the location of a second infrared light source;

a second filter associated with the second vision sensor that allows only infrared light of a second predetermined range of wavelengths to contact the second vision sensor,

wherein the first and second predetermined ranges of wavelengths are mutually exclusive.

6. The projection system of claim 5, wherein the vision sensor is a complementary metal-oxide-semiconductor (CMOS) sensor with a visible light filter.

7. The projection system of claim 5, further comprising:

a second infrared input device that is configured to emit an infrared light upon activation by a user, the infrared light providing input to the computing device via the second vision sensor, the second infrared input device emitting infrared light that has a wavelength within the second predetermined range,

wherein the infrared input device emits light that has a wavelength within the first predetermined range.

8. The projection system of claim 7, wherein an input is received by the projection system from both the input device and the second input device simultaneously.

9. The projection system of claim 7, wherein the computing device includes at least two processors, wherein a first processor determines the location of the infrared input device and a second processor determines a location of the second infrared device.

10. The projection system of claim 1, wherein at least one of the one or more computer applications is a drawing program useable by children to create an artwork.

11. A method for using a projected user interface comprising:

providing a projection system that includes a housing having disposed therein a projection unit that projects at least one image onto a projection surface, a vision sensor capable of sensing a location of an infrared light source, and a computing device having a processor and a memory that executes a software application that provides a user interface image;

projecting the user interface image onto a surface via the projection unit;

providing an infrared input device that is configured to provide an infrared input signal;

receiving the infrared input signal from the input device via the vision sensor;

determining a location of the infrared input signal with respect to the projected user interface image;

determining an input command to the software application based on the determined location of the infrared input signal;

updating the user interface image based on the input command; and

projecting an updated user interface image.

12. The method of claim 11, wherein the infrared input device includes one or more of a switch or a button that, when actuated by a user, causes the input device to emit the infrared input signal.

13. The method of claim 11, wherein the infrared input signal from the input device is reflected off of the projection surface toward the vision sensor.

14. The method of claim 11, wherein determining a location of the infrared input signal with respect to the projected user interface image further comprises:

analyzing pixel data collected by the vision sensor to identify pixels that indicate received infrared light;

filtering the pixel data to remove data not associated with infrared light emitted by the infrared input device;

identifying a largest grouping of pixels indicating received infrared light from the infrared input device;

determining a central pixel in the largest grouping; and

identifying coordinates of the central pixel.

15. The method of claim 11, further comprising;

projecting a calibration image onto the projection surface, the calibration image including one or more calibration points;

receiving an input from the infrared input device at each of the one or more calibration points;

determining a coordinate location of the received input at each of the one or more calibration points;

mapping the coordinate locations to the calibration image, thereby aligning an output of the projection unit with inputs received by the vision sensor.

16. The method of claim 11, wherein the infrared input received from the infrared input device is one or more of a short stationary signal or a prolonged non-stationary signal.

17. The method of claim 11, wherein more than one infrared input device is provided and more than one infrared input is received at one time, and wherein each infrared input device emits infrared light of a different wavelength.

18. The method of claim 17, wherein the projection system further comprises a second vision sensor, and wherein the vision sensor and the second vision sensor each sense infrared light within a respective, mutually exclusive range of wavelengths of infrared light.

19. An interactive projection unit comprising:

a projection unit that projects at least one image onto a projection surface;

a first vision sensor configured to sense the location of a first infrared light source, the first vision sensor including a first complementary metal-oxide-semiconductor (CMOS) sensor and a light filter that allows only a first predetermined range of wavelengths of infrared light to reach the CMOS sensor;

a second vision sensor configured to sense the location of a second infrared light source, the second vision sensor including a second CMOS sensor and a second light filter that allows only a second predetermined range of wavelengths of infrared light to reach the second CMOS sensor;

a computing device having a processor and a memory configured to execute one or more applications that provide a user interface image that is projected by the projection unit onto the projection surface;

a first infrared input device that is configured to provide a first input to the computing device by emitting infrared light of a first wavelength within the first predetermined range that is received by the first vision sensor; and

a second infrared input device that is configured to provide a second input to the computing device by emitting infrared light of a second wavelength within the second predetermined range that is received by the second vision sensor.

20. The interactive projection unit of claim 19, wherein the first infrared input device emits infrared light of a wavelength of approximately 850 nanometers and the second infrared input device emits infrared light of a wavelength of approximately 940 nanometers.