Interactive image projection accessory

• US 9,317,109 B2
• Filed: 03/15/2013
• Issued: 04/19/2016
• Est. Priority Date: 07/12/2012
• Status: Active Grant

• Current Assignee: Ulyte, Inc.
• Original Assignee: MEP Tech, Inc.
• Inventors: Hoole, Roger H., Davis, Mark L., Taylor, Jeffrey F., Stoker, Matthew Lund, Swank, William Lorenzo, Mealing, Donald Roy
• Primary Examiner(s): Dulka, John P
• Application Number:
  US13/842,291
• Publication Number:
  US 20140015847A1
• Time in Patent Office: 1,131 Days
• Field of Search: 463/34
• US Class Current: 1/1
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2009/2463 on a screen, e.g. using a video projector
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607

Abstract

An accessory facilitating interaction with a projected image at least partially sourced by the image generation device. The accessory receives an input image from a device, and projects at least a derived image of the input image onto a surface on which the accessory sits. For instance, the accessory might project the input image itself, or perhaps some post-processed version of the input image. A camera system captures data representing user interaction with the projected image. Based on this data, an image input event is detected, and then communicated to the device. For instance, if the image generation device were a touch-sensitive device, the device may respond to the user contacting the projected image the same as it would if the user touched the image generation device at the same corresponding location. Embodiments described herein also relate color compensation of a displayed image.

Claims

1. A method for facilitating interaction with a projected image, the method comprising:

receiving an image input from an image generation device;

projecting an image comprising at least a derived image of the input image onto a surface on which the image generation device sits;

capturing interaction data representing user interaction with the image that is projected on the surface by scanning a three-dimensional space over the surface, the interaction data corresponding to reflected light from a physical object and including;

information about a three-dimensional shape of the physical object;

information regarding a position, orientation or arrangement of the physical object over the surface and the image; and

movement of the physical object relative to the surface and the image;

detecting an image-interactive input event using the interaction data; and

communicating an interactive input corresponding to the image-interactive input event to the image generation device.

2. The method in accordance with claim 1, further comprising:

processing the interaction data or the interactive input to provide a derived image.

3. The method in accordance with claim 2, wherein processing the interaction data or the interactive input image to provide the derived image comprises:

generating a secondary image; and

compositing the secondary image with the interactive input.

4. The method in accordance with claim 2, wherein processing the interaction data or the interactive input to provide the derived image comprises:

performing color compensation of the interactive input to provide the derived image.

5. The method in accordance with claim 4, wherein performing color compensation comprises:

capturing an image of the projected image;

comparing the image of the projected image with a version of the interactive input that corresponds to the projected image; and

determining an adjustment to the version of the interactive input that would have resulted in a projected image that is closer to the version of the image input that corresponds to the projected image.

6. The method in accordance with claim 5, wherein determining the adjustment uses as input an amount of prior adjustment to the interactive input in response to a prior color compensation of the interactive input.

7. The method in accordance with claim 5, wherein the version of the interactive input that corresponds to the projected image is a prior frame of the interactive input as compared to a current frame of the interactive input to which color compensation is being applied.

8. The method in accordance with claim 2, wherein processing the interaction data or the interactive input to form the derived image comprises:

performing distortion correction on the interactive input.

9. The method in accordance with claim 2, wherein processing the interaction data or the interactive input to form the derived image comprises:

inserting one or more control visualizations into the projected image.

10. The method in accordance with claim 1, wherein receiving the image input comprises receiving the image input from an image generation device comprising a smartphone.

11. The method in accordance with claim 1, wherein receiving the image input comprises receiving the image input from an image generation device comprising a tablet computing device.

12. The method in accordance with claim 1, wherein capturing the interaction data comprises capturing the interaction data with a structured light system.

13. The method of claim 1, wherein capturing interaction data comprises capturing interaction data including information about a three-dimensional shape of a hand, information regarding a position, orientation or arrangement of the hand over the surface and the image and movement of the hand relative to the surface and the image.

14. The method of claim 1, wherein capturing interaction data comprises capturing interaction data including information about a three-dimensional shape of a physical object extending from a location outside of a periphery of the image and placed over the surface and the image.

15. The method of claim 1, wherein capturing interaction data comprises capturing interaction data further including information about contact between the physical object and the surface or information about contact between the physical object and another physical object placed on the surface.

16. An image generation device accessory, comprising:

an input interface for receiving an image input from an image generation device when the image generation device is interacting with the image generation device accessory;

a projector for projecting a projected image comprising at least a derived image of the image input onto a surface by which the image generation device is supported;

a camera system for scanning a three-dimensional space over the surface upon which the projected image is displayed and for capturing interaction data representing user interaction with the projected image on the surface, the interaction data corresponding to reflected light from a physical object and including;

information about a three-dimensional shape of the physical object;

information regarding a position, orientation or arrangement of the physical object over the surface and the image; and

movement of the physical object relative to the surface and the image;

a detection mechanism for detecting an image input event using captured data from the camera system; and

an output interface for communicating the image input event to the image generation device.

17. The image generation device accessory in accordance with claim 16, further comprising:

a mechanical interface with which the input interface may dock.

18. The image generation device of claim 16, wherein the camera system captures interaction data corresponding to a physical object that comprises a hand.

19. The image generation device of claim 16, wherein the camera system captures interaction data corresponding to a physical object that extends from a location outside of a periphery of the image and placed over the surface and the image.

20. The image generation device of claim 16, wherein the camera system also captures interaction data comprising information about contact between the physical object and the surface or information about contact between the physical object and another physical object placed on the surface.