PROJECTOR SYSTEM EMPLOYING DEPTH PERCEPTION TO DETECT SPEAKER POSITION AND GESTURES

• US 20090168027A1
• Filed: 12/28/2007
• Published: 07/02/2009
• Est. Priority Date: 12/28/2007
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Dunn, Joseph Wesslund, Dunn, Gregory Joseph, Yu, Tianli, Super, Boaz
• Granted Patent: US 8,007,110 B2
• Time in Patent Office: Days
• US Class Current: 353/28
• CPC Class Codes: G03B 21/26 Projecting separately subsidiary matter simultaneously with main image

Abstract

A projection system that prevents projection light on a presenter and provides a gesture based interface for visual enhancements is disclosed. The system may include a projector camera system comprising a computer communicating an image to a projector, the projector projecting the image on a screen, a stereo camera unit mounted approximately in the plane of the screen or at the projector and directed toward the presenter and communicating with the computer, and software that interprets the stereo camera data to distinguish and manipulate foreground and background features. The foreground feature may be the presenter relative to the screen or to the audience and room, or the presenter'"'"'s hand relative to the presenter. The invention employs imaging stereo camera algorithms and techniques developed for foreground/background segmentation, image manipulation, and gesture recognition.

Claims

1. A projection system, comprising:

a projection device for projecting at least one image onto a viewing surface;

an input device for acquiring a depth image of at least one foreground object relative to the input device;

processing logic configured to;

determine from the acquired depth image the position of the foreground object; and

modify the projected at least one image based on the position of the foreground object.

2. The system of claim 1, the processing logic further configured to:

identify from at least one of the acquired depth image and an image acquired from a camera at least one command instructing the projection system to perform an action.

3. The system of claim 1, wherein the input device is at least one of a stereo camera and a system comprising two or more monocular cameras.

4. The system of claim 1, wherein the input device is positioned at the projection device or at the viewing surface.

5. The system of claim 2, wherein when identifying a command the processing logic is configured to create from the at least one of the acquired depth image and an image acquired from a camera an isolated image of a presenter'"'"'s gesturing hand.

6. The system of claim 1, the processing logic further configured to:

create from the acquired depth image a dynamic mask of the foreground object.

7. The system of claim 6, wherein modifying the projected at least one image is using the created dynamic mask so that the projected image is restricted to the viewing surface.

8. The system of claim 6, wherein modifying the projected at least one image is using the created dynamic mask so that at least one pixel of a projected image is modified according to its location with respect to the dynamic mask.

9. The system of claim 6, wherein modifying the projected at least one image is limiting the projection on the foreground object by reducing the intensity or by shining a non-image light.

10. The system of claim 2, wherein a command to perform an action is at least one of highlighting text, highlighting objects, invoking a magnifying glass visual effect, selecting a slide from a slide show, controlling the projection device, and other visual enhancements or presentation aids.

11. A method performed by a projection system, comprising:

projecting at least one image onto a viewing surface;

acquiring with an input device a depth image of at least one foreground object relative to the input device;

determining from the acquired depth image the position of the foreground object; and

modifying the projected at least one image based on the position of the foreground object.

12. The method of claim 11, the method further comprising:

identifying from at least one of the acquired depth image and an image acquired from a camera at least one command instructing the projection system to perform an action.

13. The method of claim 11, wherein the input device is at least one of a stereo camera and a system comprising two or more monocular cameras;

and

wherein the input device is positioned at the projection device or at the viewing surface.

14. The method of claim 12, wherein when identifying a command the projection system is configured to create from at least one of the acquired depth image and an image acquired from a camera an isolated image of a presenter'"'"'s gesturing hand;

and

wherein a command to perform an action is at least one of highlighting text, highlighting objects, invoking a magnifying glass, selecting from a slide show, controlling the projection device, and other visual enhancements or presentation aids.

15. The method of claim 11, the method further comprising:

creating from the acquired depth image a dynamic mask of the foreground object.

16. The method of claim 15, wherein modifying the projected at least one image is using the created dynamic mask so that the projected image is restricted to the viewing surface.

17. The method of claim 15, wherein modifying the projected at least one image is using the created dynamic mask so that at least one pixel of the projected image is modified according to its location with respect to the dynamic mask.

18. The method of claim 15, wherein modifying the projected at least one image is limiting the projection on the foreground object by reducing the intensity or by shining a non-image light.

19. An interactive presentation system comprising:

a processor;

a storage device coupled to the processor;

and software means operative on the processor for;

projecting at least one image onto a viewing surface;

acquiring with a vision based input device a depth image of at least one foreground object;

identifying from at least one of the acquired depth image and an image acquired from the vision based input device at least one command instructing the interactive presentation system to perform an action;

determining from the acquired depth image the position of the foreground object;

creating from at least one of the acquired depth image and an image acquired from the vision based input device a dynamic mask of the foreground object; and

modifying the projected at least one image based on the dynamic mask.

20. The interactive presentation system of claim 19, wherein the vision based input device is at least one of a stereo camera and a system comprising two or more monocular cameras;

wherein the vision based input device is positioned on the projection device or on the viewing surface;

wherein modifying the projected at least one image is using the created dynamic mask so that the projected image is restricted to the viewing surface; and

wherein a command to perform an action is at least one of highlighting text, highlighting objects, invoking a magnifying glass, selecting from a slide show, controlling the projection device, and other visual enhancements or presentation aids.