System and method for gesture interface

• US 7,095,401 B2
• Filed: 10/31/2001
• Issued: 08/22/2006
• Est. Priority Date: 11/02/2000
• Status: Expired due to Term

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Liu, Che-Bin, Ramesh, Visvanathan, Geiger, Bernhard
• Primary Examiner(s): Nguyen, Jimmy H.
• Application Number:
  US10/001,553
• Publication Number:
  US 20020118880A1
• Time in Patent Office: 1,756 Days
• Field of Search: 345/156, 382/103, 382/107, 382/128, 382/153, 382/181, 382/190, 382/224, 348169-172, 715/863
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method for determining a gesture includes determining a change in a background of an image from a plurality of images, determining a object in the image, determining a trajectory of the object through the plurality of images, and classifying a gesture according to the trajectory of the object.

Claims

1. A method for automatically remotely issuing commands to a medical imaging workstation comprising the steps of:

determining a change in a background of an image from a plurality of images;

determining an object in the image,

identifying a gesture according to a trajectory motion pattern of the object;

determining if the motion pattern corresponds to a valid command by classifying the motion pattern along windows in time; and

if the motion pattern corresponds to a valid command, the workstation automatically executing the command resulting in translational and rotational manipulation of a virtual visualization tool based on the command.

2. The method of claim 1, wherein the step of determining the change in the background further comprises the steps of:

determining a gradient intensity map for the background from a plurality of images;

determining a gradient intensity map for the current image;

determining, for a plurality of pixels, a difference between the gradient intensity map and the gradient intensity map for the background;

determining a comparison between the difference and a threshold; and

determining a pixel to be a background pixel according to the comparison.

3. The method of claim 1, wherein the object includes a user'"'"'s hand.

4. The method of claim 1, wherein the step of identifying a gesture further comprises the steps of:

determining a reference point;

determining a correspondence between the trajectory and the reference point; and

translating the trajectory according to one of a plurality of commands.

5. The method of claim 1 wherein the step of determining an object in the image further comprising the steps of:

obtaining a normalized color representation for a plurality of colors in each image;

determining from training images an estimate of a probability distribution of normalized color values for an object class; and

determining, for each pixel, a likelihood according to an estimated probability density of normalized color values for the object class.

6. The method of claim 1 wherein the step of executing the command further comprises the steps of:

determining the duration of the gesture; and

correlating the duration of the gesture to an intensity and scale in which the command is executed.

8. The method of claim 1 wherein the virtual visualization tool is a virtual endoscope.

9. The method of claim 8 wherein the command corresponds to rotation of the virtual endoscope.

10. The method of claim 9 wherein the gesture is rotation of a user'"'"'s hand.

11. The method of claim 8 wherein the command corresponds to moving the virtual endoscope in a right to left direction.

12. The method of claim 11 wherein the gesture is waving of a user'"'"'s hand from right to left.

13. The method of claim 1, wherein the step of determining the trajectory of the object through the plurality of images further comprises the steps of:

determining, for each pixel, a temporal likelihood across a plurality of images; and

determining a plurality of moments according to the temporal likelihoods.

14. The method of claim 1 wherein the identified gesture is a valid command if it detects object pixels moving in a predefined pattern.

7. The method of 6 wherein repetition of a command increases the intensity of the corresponding command response.

15. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for automatically remotely issuing commands to a medical imaging workstation, the method steps comprising:

determining a change in a background of an image from a plurality of images;

determining an object in the images;

identifying a gesture according to a trajectory and motion pattern of the object;

determining if the motion pattern corresponds to a valid command by classifying the motion pattern along window in time; and

if the motion pattern corresponds to a valid command, the workstation automatically executing the command resulting in translational and rotational manipulation of a virtual visualization tool based on the command.

16. The method of claim 15, wherein the step of determining the change in the background further comprises the steps of:

determining a gradient intensity map for the background from a plurality of images;

determining a gradient intensity map for the current image;

determining, for a plurality of pixels, a difference between the gradient intensity map and the gradient intensity map for the background;

determining a comparison between the difference and a threshold; and

determining a pixel to be a background pixel according to the comparison.

17. The method of claim 15, wherein the object includes a user'"'"'s hand.

18. The method of claim 15, wherein the step of identifying a gesture further comprises the steps of:

determining a reference point;

determining a correspondence between the trajectory and the reference point; and

translating the trajectory according to one of a plurality of commands.

19. The method of claim 15, wherein the step of determining the object in the image further comprises the steps of:

obtaining a normalized color representation for a plurality of colors in each image;

determining from training images an estimate of a probability distribution of normalized color values for an object class; and

determining, for each pixel, a likelihood according to an estimated probability density of normalized color values for the object class.