Interface using pattern recognition and tracking

• US 6,363,160 B1
• Filed: 01/22/1999
• Issued: 03/26/2002
• Est. Priority Date: 01/22/1999
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Yeo, Boon-Lock, Yeung, Minerva M., Bradski, Gary R.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US09/235,139
• Time in Patent Office: 1,159 Days
• Field of Search: 382/100, 382/103, 382/155, 382/159, 382/168, 382/170, 382/171, 382/172, 382/181, 382/190, 382/209, 382/216, 382/218, 382/312, 348/77, 348/169, 345/339, 345/340, 345/156, 345/157
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method for interfacing with an electronic system is provided. The method includes receiving images from a camera. A portion of at least one image from the camera is tracked. A pattern in the tracked portion of the at least one image is recognized. And, the electronic system is controlled based on the recognized pattern.

Claims

1. A method for interfacing with an electronic system, the method comprising:

receiving images from a camera;

tracking a portion of at least one image from the camera using a mean shift method on a probability distribution;

recognizing a pattern in the tracked portion of the at least one image; and

selectively controlling the electronic system based on the recognized pattern.

2. The method of claim 1, wherein tracking a portion of the at least one image comprises tracking a portion of the at least one image that corresponds to a hand of the user.

3. The method of claim 1, wherein recognizing a pattern comprises recognizing a pattern that corresponds to a selected hand gesture.

4. The method of claim 1, wherein recognizing a pattern comprises recognizing a hand gesture based on a gradient orientation histogram.

5. The method of claim 1, wherein recognizing a pattern comprises:

generating at least one histogram from the tracked portion of the at least one image from the camera;

comparing the at least one histogram with histograms created based on known patterns; and

selecting among the known patterns based on the comparisons.

6. The method of claim 5, wherein comparing the at least one histogram comprises comparing the at least one histogram using a Bayesian probability routine.

7. The method of claim 5, wherein comparing the at least one histogram comprises comparing the at least one histogram using a chi-square routine to determine the most likely pattern, and a minimum intersection routine to determine if the pattern exceeds a threshold.

8. The method of claim 5, wherein comparing the at least one histogram comprises comparing the at least one histogram using a correlation routine.

9. The method of claim 5, wherein generating at least one histogram comprises generating a gradient orientation histogram.

10. The method of claim 1, and further comprising training on a number of known patterns prior to recognizing a pattern.

11. The method of claim 1, wherein selectively controlling the electronic system comprises generating signals that control the display of video data on a display, wherein the video data includes a graphical representation of the content of a video source.

12. The method of claim 11, wherein the graphical representation of the content of the video source is generated by detecting shots within the video source and grouping the shots in the video source into scenes.

13. A user interface comprising:

a camera for capturing images of at least a portion of the user; and

a processor based device, coupled to the camera, that is operable to receive images from the camera, to track a portion of at least one image using a mean shift method on a probability distribution, to recognize a pattern in the tracked portion of the at least one image, and to generate signals to control the operation of an electronic device.

14. The user interface of claim 13, wherein the processor based device recognizes a pattern using a gradient orientation histogram of the tracked portion of the image.

15. The user interface of claim 13, wherein the processor based device selects a portion of the at least one image that corresponds to a hand of the user.

16. The user interface of claim 13, wherein the processor based device recognizes a pattern by generating at least one histogram from the tracked portion of the at least one image, comparing the at least one histogram with histograms created based on known patterns, and selecting among the known patterns based on the comparisons.

17. The user interface of claim 16, wherein the processor based device compares the at least one histogram using a chi-square routine to determine the most likely pattern, and a minimum intersection routine to determine if the pattern exceeds a threshold.

18. The user interface of claim 13, wherein the processor based device selectively controls the electronic system by generating signals that control the display of video data on a display, wherein the video data includes a graphical representation of the content of a video source.

19. A system with a pattern recognition interface, the system comprising:

a video source;

a camera that captures images at least a portion of a user;

a display for providing video information to the user; and

a processor based device, coupled to the video source, the camera and the display, that is operable to receive an image from the camera, to track a portion of the image using a mean shift method on a probability distribution, to recognize a pattern in the tracked portion of the image, and to selectively drive the display based on the recognized pattern.

20. The system of claim 19, wherein the processor based device recognizes a pattern using a gradient orientation histogram of the tracked portion of the image.

21. The system of claim 19, wherein the processor based device selects a portion of the at least one image that corresponds to a hand of the user.

22. The system of claim 19, wherein the processor based device recognizes a pattern by generating at least one histogram from the tracked portion of the at least one image, comparing the at least one histogram with histograms created based on known patterns, and selecting among the known patterns based on the comparisons.

23. The system of claim 22, wherein the processor based device compares the at least one histogram using a chi-square routine to determine the most likely pattern, and a minimum intersection routine to determine if the pattern exceeds a threshold.

24. The system of claim 19, wherein the processor based device selectively controls the electronic system by generating signals that control the display of video data on a display, wherein the video data includes a graphical representation of the content of a video source.

25. A method for navigating through video data from a video source, the method comprising:

displaying a graphical representation of the content of a video source on a display, the graphical representation of the content of the video source being generated based on an analysis of the frames of the video source;

receiving images from a camera;

tracking a portion of at least one image from the camera that corresponds to at least one hand of the user by using a mean shift method on a probability distribution;

recognizing a pattern in the tracked portion of the image that corresponds to a selected gesture, including;

generating at least one histogram from the tracked portion of the at least one image from the camera,

comparing the at least one histogram with histograms created based on known patterns, and

selecting among the known patterns based on the comparisons; and

controlling the display based on the recognized pattern.

26. The method of claim 25, wherein comparing the at least one histogram comprises comparing the at least one histogram using a Bayesian probability routine.

27. The method of claim 25, wherein comparing the at least one histogram comprises comparing the at least one histogram using a chi-square routine to determine the most likely pattern, and a minimum intersection routine to determine if the pattern exceeds a threshold.

28. The method of claim 25, wherein comparing the at least one histogram comprises comparing the at least one histogram using a correlation routine.

29. The method of claim 25, wherein generating at least one histogram comprises generating a gradient orientation histogram.

30. The method of claim 25, and further comprising training on a number of known patterns prior to recognizing a pattern.