INTERACTIVE IMAGE PROJECTION

• US 20160306418A1
• Filed: 04/19/2016
• Published: 10/20/2016
• Est. Priority Date: 07/12/2012
• Status: Active Grant

• Current Assignee: Ulyte, Inc.
• Original Assignee: MEP Tech, Inc.
• Inventors: Davis, Mark L., Stoker, Matthew Lund, Swank, William Lorenzo, Mealing, Donald Roy, Hoole, Roger H., Taylor, Jeffrey F.
• Granted Patent: US 9,946,333 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2009/2463 on a screen, e.g. using a video projector
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607

Abstract

An accessory facilitating interaction with a projected image at least partially sourced by the image generation device. The accessory receives an input image from a device, and projects at least a derived image of the input image onto a surface on which the accessory sits. For instance, the accessory might project the input image itself, or perhaps some post-processed version of the input image. A camera system captures data representing user interaction with the projected image. Based on this data, an image input event is detected, and then communicated to the device. For instance, if the image generation device were a touch-sensitive device, the device may respond to the user contacting the projected image the same as it would if the user touched the image generation device at the same corresponding location. Embodiments described herein also relate color compensation of a displayed image.

Claims

1. A method for facilitating interaction with a projected image, the method comprising:

projecting an image onto a surface;

capturing interaction data representing user interaction with the image that is projected on the surface by scanning a three-dimensional space over the surface, the interaction data corresponding to reflected waves from a physical object and including;

information regarding a position, orientation and/or arrangement of the physical object relative to the surface and the image; and

movement of the physical object relative to the surface and the image;

identifying an image-interactive input event from the interaction data; and

communicating an interactive input corresponding to the image-interactive input event to an image generator.

2. The method of claim 1, wherein projecting the image onto the surface comprises receiving an image input from the image generator and projecting the image, comprising at least a derived image of the image input, onto the surface.

3. The method of claim 1, further comprising:

processing the interaction data or the interactive input to provide a modified image.

4. The method of claim 3, wherein processing the interaction data or the interactive input to provide the modified image comprises:

generating a secondary image; and

compositing the secondary image with the interactive input.

5. The method of claim 3, wherein processing the interaction data or the interactive input comprises:

performing color compensation of the interactive input to provide the derived image.

6. The method of claim 5, wherein performing color compensation comprises:

capturing an image of the projected image;

comparing the image of the projected image with a version of the interactive input that corresponds to the projected image; and

determining an adjustment to the version of the interactive input that would have resulted in a projected image that is closer to the version of the image input that corresponds to the projected image.

7. The method of claim 6, wherein determining the adjustment uses as input an amount of prior adjustment to the interactive input in response to a prior color compensation of the interactive input.

8. The method of claim 6, wherein the version of the interactive input that corresponds to the projected image is a prior frame of the interactive input as compared to a current frame of the interactive input to which color compensation is being applied.

9. The method of claim 3, wherein processing the interaction data or the interactive input to form the derived image comprises:

performing distortion correction on the interactive input.

10. The method of claim 3, wherein processing the interaction data or the interactive input to form the derived image comprises:

inserting one or more control visualizations into the projected image.

11. The method of claim 1, wherein capturing the interaction data comprises capturing the interaction data with a structured light system.

12. The method of claim 1, wherein capturing interaction data comprises capturing interaction data including information about a three-dimensional shape of a hand, information regarding a position, orientation or arrangement of the hand over the surface and the image and movement of the hand relative to the surface and the image.

13. The method of claim 1, wherein capturing interaction data comprises capturing interaction data including information about a three-dimensional shape of a physical object extending from a location outside of a periphery of the image and placed over the surface and the image.

14. The method of claim 1, wherein capturing interaction data comprises capturing interaction data further including information about contact between the physical object and the surface or information about contact between the physical object and another physical object placed on the surface.

15. An image generation device accessory, comprising:

a projector for projecting a projected image onto a surface;

a camera system for scanning a three-dimensional space over the surface upon which the projected image is displayed and for capturing interaction data representing user interaction with the projected image on the surface, the interaction data corresponding to reflected waves from a physical object and including;

information regarding a position, orientation or arrangement of the physical object over the surface and the image; and

movement of the physical object relative to the surface and the image;

a detection mechanism for detecting an image input event using captured data from the camera system; and

an output interface for communicating the image input event to an image generator.

16. The image generation device accessory of claim 15, further comprising:

an input interface for receiving an image input from the image generator when the image generator is interacting with the image generation device accessory.

17. The image generation device accessory of claim 15, further comprising:

a mechanical interface with which the input interface may dock.

18. The image generation device accessory of claim 15, wherein the camera system captures interaction data corresponding to a physical object that comprises a hand.

19. The image generation device accessory of claim 16, wherein the camera system captures interaction data corresponding to a physical object that extends from a location outside of a periphery of the image and placed over the surface and the image.

20. The image generation device accessory of claim 16, wherein the camera system also captures interaction data comprising information about contact between the physical object and the surface or information about contact between the physical object and another physical object placed on the surface.