System and method for gesture based control system

• US 9,606,630 B2
• Filed: 09/09/2014
• Issued: 03/28/2017
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Original Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Inventors: Parent, Kevin T., Underkoffler, John S.
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US14/481,662
• Publication Number:
  US 20150205364A1
• Time in Patent Office: 931 Days
• Field of Search: 345156-168, 715856-865
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

The system provides a gestural interface to various visually presented elements, presented on a display screen or screens. A gestural vocabulary includes ‘instantaneous’ commands, in which forming one or both hands into the appropriate ‘pose’ results in an immediate, one-time action; and ‘spatial’ commands, in which the operator either refers directly to elements on the screen by way of literal ‘pointing’ gestures or performs navigational maneuvers by way of relative or “offset” gestures. The system contemplates the ability to identify the users hands in the form of a glove or gloves with certain indicia provided thereon, or any suitable means for providing recognizable indicia on a user'"'"'s hands or body parts. A system of cameras can detect the position, orientation, and movement of the user'"'"'s hands and translate that information into executable commands.

Claims

1. A system comprising:

a detector positioned to sense a body;

a processor coupled to the detector and configured for,

detecting a physical control gesture made by a user by dynamically detecting a position of at least one marker on the user, the detecting comprising using gesture data that is absolute three-space location data of an instantaneous state of the user at a point in time and space, and identifying the physical control gesture using only the gesture data;

translating the control gesture to an executable command;

updating a computer display in response to the executable command.

2. A system comprising:

a detector positioned to sense a body;

a processor coupled to the detector and configured for,

automatically detecting a gesture of a body from gesture data received via a detector, the detecting comprising detecting a position of at least one marker on the body using the gesture data that is absolute three-space location data of an instantaneous state of the body at a point in time and space, the detecting comprising identifying the gesture using only the gesture data;

translating the gesture to a gesture signal; and

controlling a component coupled to a computer in response to the gesture signal.

3. A system comprising:

a detector positioned to sense a body;

a processor coupled to the detector and configured for,

detecting poses and motion of an object from gesture data received via a detector, the detecting comprising dynamically detecting a position of at least one marker on the object using the gesture data that is absolute three-space location data of an instantaneous state of the poses and motion at a point in time and space, the detecting comprising identifying the poses and motion using only the gesture data;

translating the poses and motion into a control signal using a gesture notation; and

controlling a computer application using the control signal.

4. The system of claim 3, wherein the controlling includes controlling a function of an application hosted on the computer.

5. The system of claim 3, wherein the controlling includes controlling a component displayed on the computer.

6. The system of claim 3, wherein the detecting includes detecting a location of the body.

7. The system of claim 3, wherein the detecting includes detecting an orientation of the body.

8. The system of claim 3, wherein the detecting includes detecting motion of the body.

9. The system of claim 3, wherein the detecting comprises identifying the gesture, wherein the identifying includes identifying a pose and an orientation of a portion of the body.

10. The system of claim 3, wherein the detecting includes detecting at least one of a first set of appendages and a second set of appendages of the body.

11. The system of claim 3, wherein the detecting includes optically detecting motion of the body.

12. The system of claim 3, wherein the at least one marker includes a set of markers.

13. The system of claim 3, wherein the translating comprises translating information of the gesture to a gesture notation.

14. The system of claim 13, wherein the gesture notation represents a gesture vocabulary, and the gesture signal comprises communications of the gesture vocabulary.

15. The system of claim 14, wherein the gesture vocabulary represents in textual form instantaneous pose states of kinematic linkages of the body.

16. The system of claim 14, wherein the gesture vocabulary represents in textual form an orientation of kinematic linkages of the body.

17. The system of claim 14, wherein the gesture vocabulary represents in textual form a combination of orientations of kinematic linkages of the body.

18. The system of claim 14, wherein the gesture vocabulary includes a string of characters that represent a state of kinematic linkages of the body.

19. The system of claim 18, wherein the kinematic linkage is at least one first appendage of the body.

20. The system of claim 19, comprising assigning each position in the string to a second appendage, the second appendage connected to the first appendage.

21. The system of claim 20, comprising assigning characters of a plurality of characters to each of a plurality of positions of the second appendage.

22. The system of claim 21, wherein the plurality of positions is established relative to a coordinate origin.

23. The system of claim 22, comprising establishing the coordinate origin using a position selected from a group consisting of an absolute position and orientation in space, a fixed position and orientation relative to the body irrespective of an overall position and heading of the body, and interactively in response to an action of the body.

24. The system of claim 21, comprising assigning characters of the plurality of characters to each of a plurality of orientations of the first appendage.