Method and apparatus for detecting gesture in user-based spatial coordinate system

• US 10,846,864 B2
• Filed: 12/08/2017
• Issued: 11/24/2020
• Est. Priority Date: 06/10/2015
• Status: Active Grant

• Current Assignee: Vtouch Co.
• Original Assignee: Vtouch Co Ltd
• Inventors: Kim, Seokjoong, Kim, Chunghoon, Lee, Beomho
• Primary Examiner(s): Jansen, II, Michael J
• Application Number:
  US15/835,809
• Publication Number:
  US 20180173318A1
• Time in Patent Office: 1,082 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Disclosed are a method and an apparatus for accurately detecting a gesture from a user'"'"'s motion using a user-based spatial coordinate system. A method for detecting a gesture in a user-based spatial coordinate system comprises the steps of: setting a user-based spatial coordinate system using a first body coordinate corresponding to a first body part of the user as a starting point; analyzing the motion vector state of a second body coordinate corresponding to a second body part of the user in the user-based spatial coordinate system over time; and detecting the user'"'"'s gesture on the basis of a change in the motion vector state.

Claims

1. A method for detecting a gesture in a user-based spatial coordinate system, comprising the steps of - the user-based spatial coordinate system comprises a three- dimensional body coordinates reception unit, a user-based spatial coordinate system definition unit, a motion vector analysis unit, and a gesture detection unit:

receiving, by the three-dimensional body coordinates reception unit, a plurality of body coordinate points corresponding to a plurality of body parts including a first body part and a second body part of a user;

defining, by the user-based spatial coordinate system definition unit, a user- based spatial coordinate system whose origin is located at a first body coordinate point corresponding to the first body part of a user;

generate, by the motion vector analysis unit, a motion vector overtime of a second body coordinate point corresponding to the second body part of the user in the user-based spatial coordinate system;

analyzing, by the motion vector analysis unit, a state of the motion vector; and

detecting, by the gesture detection unit, a gesture of the user based on a change in the state of the motion vector,

wherein the step of defining the user-based spatial coordinate system further comprises the steps of;

creating, by the user-based spatial coordinate system definition unit, a front/rear axis of the user-based spatial coordinate system corresponding to a straight line connecting the first body coordinate point and the second body coordinate point, and

creating, by the user-based spatial coordinate system definition unit, a left/right axis and an up/down axis of the user-based spatial coordinate system to perpendicularly cross the created front/rear axis,

wherein the left/right axis of the user-based spatial coordinate system is perpendicular to the up/down axis of the user-based spatial coordinate system,

wherein the first body coordinate point corresponds to one of two eyes of the user and the second body coordinate point corresponds to a finger of the user,

the left/right axis of the user-based spatial coordinate system is defined as coinciding with the left and right directions recognized by the user, and

wherein the left/right axis is created to pass through the first body coordinate point and to be parallel to a ground surface, wherein the up/down axis is created to pass through the first body coordinate point.

2. The method of claim 1, wherein the up/down axis is created to pass through the first body coordinate point and to perpendicularly cross a straight line connecting the first body coordinate point and a third body coordinate point corresponding to a third body part at the first body coordinate point,

wherein the left/right axis is created to pass through the first body coordinate point,

wherein the first body part and the third body part are different from each other.

3. The method of claim 1, wherein the step of analyzing the state of the motion vector comprises the step of:

creating, by the motion vector analysis unit, the motion vector V to have spherical coordinates, with respect to the second body coordinate point (x₁, y₁, z₁) of the second body part at a first time point and the second body coordinate point (x₂, y₂, z₂) at a second time point, wherein;

4. The method of claim 3, wherein the step of analyzing the state of the motion vector further comprises the step of:

specifying, by the motion vector analysis unit, the state of the motion vector as any one of a stationary state, a forward movement state, a backward movement state, and a surface movement state, based on the r, θ and

φ

of the motion vector.

5. The method of claim 4, wherein the step of detecting the gesture of the user comprises the step of:

determining, by the gesture detection unit, that a touch gesture is made, when the state of the motion vector is changed from the forward movement state to the stationary state.

6. The method of claim 4, wherein the step of detecting the gesture of the user comprises the step of:

determining, by the gesture detection unit, that a release gesture is made, when the state of the motion vector is changed to the backward movement state.

7. The method of claim 4, wherein the step of detecting the gesture of the user comprises the step of:

determining, by the gesture detection unit, that a click gesture is made, when the state of the motion vector is changed from the forward movement state through the stationary state to the backward movement state, or from the forward movement state to the backward movement state.

8. The method of claim 4, wherein the step of detecting the gesture of the user comprises the step of:

determining, by the gesture detection unit, that a drag gesture is made on a surface of a virtual sphere, when the state of the motion vector is changed to the surface movement state.

9. The method of claim 4, wherein the step of detecting the gesture of the user comprises the step of:

determining, by the gesture detection unit, that a hold gesture is made, when the state of the motion vector remains in the stationary state.

10. The method of claim 4, wherein the step of detecting the gesture of the user comprises the step of:

determining, by the gesture detection unit, that a deep touch gesture is made, when the state of the motion vector is changed in an order of the forward movement state, the stationary state, the forward movement state, and the stationary state.

11. An apparatus for detecting a gesture in a user-based spatial coordinate system, comprising:

one or more processors;

machine readable and executable instructions that, when executed by the one or more processors, cause the apparatus to;

receive a plurality of body coordinate points corresponding to a plurality of body parts including a first body part and a second body part of a user;

create a user-based spatial coordinate system whose origin is located at a first body coordinate point corresponding to the first body part, based on the plurality of body coordinate points;

generate a motion vector over time of a second body coordinate point, and to determine a state of the motion vector; and

detect a gesture of the user based on a change in the state of the motion vector,

wherein the machine readable and executable instructions, when executed by the one or more processors, cause the apparatus to;

create a front/rear axis of the user-based spatial coordinate system corresponding to a straight line connecting the first body coordinate point and the second body coordinate point,

create a left/right axis of the user-based spatial coordinate system to perpendicularly cross the created front/rear axis, and

create a up/down axis of the user-based spatial coordinate system to perpendicularly cross the created front/rear axis,

wherein the left/right axis of the user-based spatial coordinate system is perpendicular to the up/down axis of the user-based spatial coordinate system, and

wherein the first body coordinate point corresponds to one of two eyes of the user and the second body coordinate point corresponds to a finger of the user,

the left/right axis of the user-based spatial coordinate system is defined as coinciding with the left and right directions recognized by the user, and

wherein the machine readable and executable instructions, when executed by the one or more processors, cause the apparatus to;

create the left/right axis to pass through the first body coordinate point and to the parallel to a ground surface; and

create the up/down axis to pass through the first body coordinate point.

12. The apparatus of claim 11, wherein the machine readable and executable instructions, when executed by the one or more processors, cause the apparatus to:

create the up/down axis to pass through the first body coordinate point and to perpendicularly cross a straight line connecting the first body coordinate point and a third body coordinate point corresponding to a third body part at the first body coordinate point,

wherein the left/right axis definition unit creates the left/right axis to pass through the first body coordinate point,

wherein the first body part and the third body part are different from each other.

13. The apparatus of claim 11, wherein the machine readable and executable instructions, when executed by the one or more processors, cause the apparatus to:

create the motion vector V to have spherical coordinates, with respect to the second body coordinate point (x₁, y₁, z₁) of the second body part at a first time point and the second body coordinate point (x₂, y₂, z₂) at a second time point, as below;

14. The apparatus of claim 13, wherein the machine readable and executable instructions, when executed by the one or more processors, cause the apparatus to:

specify the state of the motion vector as any one of a stationary state, a forward movement state, a backward movement state, and a surface movement state, based on the r, θ and

φ

of the motion vector.

15. The apparatus of claim 14, wherein machine readable and executable instructions, when executed by the one or more processors, cause the apparatus to:

monitor a change in the state of the motion vector;

store a type of gesture corresponding to each type of change in the state of the motion vector; and

determine the type of gesture of the user corresponding to the change in the state of the motion vector, based on the gesture storage unit.