SYSTEM AND METHOD FOR INTERACTING WITH A DISPLAY

• US 20130002539A1
• Filed: 09/13/2012
• Published: 01/03/2013
• Est. Priority Date: 10/25/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: DENNARD, Mark D., McCULLOCH, Douglas J., SCHUNZEL, Mark J., TREVATHAN, Matthew B.
• Application Number:
  US13/614,200
• Publication Number:
  US 20130002539A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G03B 21/26 Projecting separately subsidiary matter simultaneously with main image

Abstract

A system and method of interacting with a display. The method comprises recognizing a disturbance in a display zone of a projected image and displaying a selected state in response to the recognized disturbance. The method further includes recognizing a gesture which interrupts a light source and is associated with an action to be taken on or associated with the displayed selected state. An action is executed in response to the recognized gesture. The system includes a server having a database containing data associated with at least one or more predefined gestures, and at least one of a hardware and software component for executing an action based on the at least one or more predefined gestures.

Claims

1. A method, comprising:

recognizing with a camera a disturbance of a disturbing object in a display zone of a projected image;

making a determination as to whether a shape of the disturbing object is a valid shape in a template;

displaying a selected state of a selected item when the disturbing object is a valid shape;

recognizing with the camera a movement comprising a combination of two substantially simultaneous actions, including a first action of the disturbing object and a second action of an object in an interaction zone between the camera and the projected image;

validating the movement comprising the combination of the first action and the second action by comparing the first action and the second action to a plurality of predefined gestures; and

executing a command based on the validating of the combination of the first action and the second action.

2. The method of claim 1, wherein the display zone is one or more areas on the image.

3. The method of claim 1, wherein the executed command is a menu driven command.

4. The method of claim 1, wherein the predefined combination of two substantially simultaneous actions includes a trajectory for defined patterns that indicate at least one of a type and place of motion.

5. The method of claim 1, wherein when the disturbing object is a valid shape, the selected state of a selected icon or menu item in a user interface of the projected image is displayed.

6. The method of claim 5, wherein a look-up table associates the plurality of predefined gestures with a respective plurality of commands, the plurality of pre-defined gestures comprising:

a single left click gesture that executes a single left mouse click selection;

a double left click gesture that executes double left mouse click selection;

a single right click gesture that executes a single right mouse click selection;

a copy gesture that executes a copy command;

a paste gesture that executes a paste command;

an undo gesture that executes an undo command; and

a redo gesture that executes a redo command.

7. The method of claim 6, wherein the plurality of pre-defined gestures further comprise:

a next gesture that takes a user to a next entry in a list;

a previous gesture that takes the user to a previous entry in the list;

a first entry gesture that takes the user to a first entry in the list;

a last entry gesture that takes the user to a last entry in the list;

a home gesture that takes the user to a home of an application; and

an exit gesture that executes an exit command and closes the application.

8. The method of claim 1, further comprising, before recognizing the disturbance in the display zone,

recognizing a user entering a field of vision of a camera;

and

determining the proximity of the user to the user interface.

9. A system comprising a server having a database containing data associated with at least one or more predefined gestures, and at least one of a hardware and software component for executing an action based on the at least one or more predefined gestures, the hardware and software:

comparing a first action of a disturbing object, detected by a camera, in an interaction zone between the camera and a projected image to a predefined template of a shape;

recognizing a gesture detected by the camera, which interrupts a light source between the camera and the projected image, the gesture comprising a combination of two substantially simultaneous hand motions;

determining that the gesture corresponds to a predefined command of the user interface; and

executing the command corresponding to the gesture.

10. The system of claim 9, wherein the system includes an architecture comprising a services layer, an integration layer and an application layer.

11. The system of claim 10, wherein the services layer comprises at least:

a vision interface module responsible for recognizing the gesture and converting an appropriate action to an application;

a geometric reasoning module which enables automatic selection of a display based on proximity of the interaction zone to a user and non-occlusion of the interaction zone by the user or other object; and

a user localization module to determine a position of the user.

12. The system of claim 11, wherein one of the geometric reasoning module and user localization module recognize the first action and the second action.

13. The system of claim 9, wherein the at least one of a hardware and software component resides on a server provided by a service provider.

14. A computer program product comprising a computer usable storage medium having readable program code embodied in the storage medium, the computer program product includes at least one component to:

recognize a disturbance caused by a first finger of a first hand of the user in a display zone of a user interface image projected on a surface in the field of vision of the camera;

display a selected state in response to the recognized disturbance;

recognize a gesture interrupting a light source and associated with an action to be taken on or associated with the displayed selected state, the gesture comprising a combination of two substantially simultaneous motions by the first hand of the user and a second hand of the user;

determine that the gesture corresponds to a command of the user interface based on one or more look-up tables populated with a plurality of pre-defined hand motions and a respective plurality of commands of the user interface; and

execute the associated command of the user interface in response to the gesture.

15. The computer program product of claim 14, wherein the recognized gesture includes a motion of a non-tethered object between the source and the projected image.

16. The computer program product of claim 14, wherein the display zone is one or more areas on the projected image which are associated with an application program.

17. The computer program product of claim 14, wherein when the disturbance is a valid shape, the selected state of a selected item is displayed.

18. The computer program product of claim 14, wherein the executed function is a menu driven command.