METHOD AND APPARATUS FOR INTERACTING WITH PROJECTED DISPLAYS USING SHADOWS

• US 20120139827A1
• Filed: 12/02/2010
• Published: 06/07/2012
• Est. Priority Date: 12/02/2010
• Status: Abandoned Application

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: LI, KEVIN A., Cowan, Lisa Gail
• Application Number:
  US12/959,231
• Publication Number:
  US 20120139827A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour

Abstract

A method, computer readable medium and apparatus for interacting with a projected image using a shadow are disclosed. For example, the method projects an image of a processing device to create the projected image, and detects a shadow on the projected image. The method interprets the shadow as a display formatting manipulation command and sends the display formatting manipulation command to an application of the processing device to manipulate a display format of the projected image.

Claims

1. A method for interacting with a projected image using a shadow, comprising:

projecting an image of a processing device to create the projected image;

detecting a shadow on the projected image;

interpreting the shadow as a display formatting manipulation command; and

sending the display formatting manipulation command to an application of the processing device to manipulate a display format of the projected image.

2. The method of claim 1, wherein the processing device comprises a mobile device.

3. The method of claim 1, wherein the detecting the shadow is performed by a camera coupled to the processing device.

4. The method of claim 1, wherein the detecting the shadow comprises:

detecting outer edges of the projected image;

performing thresholding to determine which pixels of the projected image are shadow pixels; and

detecting an area of connected shadow pixels, where the area is greater than a predetermined threshold.

5. The method of claim 1, wherein the interpreting the shadow comprises:

tracking a shape of the shadow;

tracking a position vector of the shadow;

tracking a velocity vector of the shadow; and

tracking an acceleration vector of the shadow.

6. The method of claim 5, wherein the tracking the shape of the shadow, the tracking the position vector of the shadow, the tracking the velocity vector of the shadow and the tracking the acceleration vector of the shadow are performed continuously over a pre-defined time period.

7. The method of claim 1, wherein the display formatting manipulation command comprises at least one of:

a pointing command, a panning command or a zooming command.

8. The method of claim 7, wherein the pointing command is correlated to the shadow having a convex hull that is separated by defects, wherein points on the convex hull that are separated by the defects are estimated as a location of a fingertip, wherein a location of the fingertip is stable for a predefined period of time.

9. The method of claim 8, wherein the pointing command causes a pop-up box with information about a selected point to appear on the projected image.

10. The method of claim 7, wherein the panning command is correlated to a centroid of the shadow having an average velocity over a predetermined time period above a predefined threshold in a direction.

11. The method of claim 10, wherein the direction is at least one of:

an up direction, a down direction, a left direction or a right direction.

12. The method of claim 10, wherein the panning command causes the projected image to move in the direction of the shadow.

13. The method of claim 7, wherein the zooming command is correlated to at least one of:

a change in an area of the shadow over a predetermined time period above a predefined threshold or detecting a transition of a number of fingertips.

14. The method of claim 13, wherein an increase in the change in the area of the shadow causes the projected image to zoom out and a decrease in the change in the area of the shadow causes the projected image to zoom in.

15. The method of claim 13, wherein the detecting the transition of the number of fingertips from a single fingertip to two fingertips causes the projected image to zoom out and detecting the transition from the two fingertips to the single fingertip causes the projected image to zoom in.

16. The method of claim 1, wherein the shadow is created by an object disposed in front of a projector.

17. The method of claim 1, wherein the projected image is displayed on a user'"'"'s hand.

18. The method of claim 1, further comprising:

projected a second display of a second processing device to create a second projected image, wherein the second projected image overlaps the projected image such that a portion of the projected image blocked by the shadow is re-displayed on top of the shadow via the second projected image.

19. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform a method for interacting with a projected image using a shadow, comprising:

projecting an image of a processing device to create the projected image;

detecting a shadow on the projected image;

interpreting the shadow as a display formatting manipulation command; and

sending the display formatting manipulation command to an application of the processing device to manipulate a display format of the projected image.

20. An apparatus for interacting with a projected image using a shadow, comprising:

means for projecting an image of a processing device to create the projected image;

means for detecting a shadow on the projected image;

means for interpreting the shadow as a display formatting manipulation command; and

means for sending the display formatting manipulation command to an application of the processing device to manipulate a display format of the projected image.