Interactive input system and method

• US 10,203,765 B2
• Filed: 03/21/2016
• Issued: 02/12/2019
• Est. Priority Date: 04/12/2013
• Status: Active Grant

• Current Assignee: Shanghai YVR Technology Company Limited
• Original Assignee: uSens Inc.
• Inventors: He, Anli, Fei, Yue
• Primary Examiner(s): Nguyen, Jennifer
• Application Number:
  US15/075,714
• Publication Number:
  US 20160202844A1
• Time in Patent Office: 1,058 Days
• Field of Search: 345156, 345173, 715863
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method for human-machine interaction includes monitoring a movement of an object by a sensor that detects positions of the object over time, generating a time-dependent velocity of the object based on the movement of the object, detecting a tapping event of the object tapping on a surface by detecting a sudden change of the time-dependent velocity, and determining a position of the object at a time when the tapping event occurs as a tapping position of the object.

Claims

1. A method for human-machine interaction, comprising:

monitoring a movement of an object by a sensor that detects positions of the object over time;

generating a time-dependent velocity of the object based on the movement of the object;

detecting a tapping event of the object tapping on a surface by detecting a sudden change of the time-dependent velocity; and

determining a position of the object at a time when the tapping event occurs as a tapping position of the object;

wherein detecting the tapping event comprises;

determining whether the time-dependent velocity in a certain direction drops from a first speed value to a second speed value within a first time threshold, the first speed value being larger than the second speed value, and the second speed value being close to zero, and

determining whether the time-dependent velocity in the certain direction keeps at about the second speed value for longer than a second time threshold; and

determining the tapping event has occurred if the time-dependent velocity in the certain direction drops from the first speed value to the second speed value within the first period of time and the time-dependent velocity in the certain direction keeps at about the second speed value for longer than the second time threshold.

2. The method of claim 1, wherein monitoring the movement of the object includes:

recording a time-dependent position value of the object based on the detected positions over time; and

converting the time-dependent position value to the time-dependent velocity by differentiation.

3. The method of claim 2, wherein:

detecting the positions of the object includes recording the positions of the object in a first three-dimensional (3D) coordinate system, and

determining whether the time-dependent velocity in the certain direction drops from the first speed value to the second speed value within the first time threshold includes;

mapping the positions of the object from the first 3D coordinate system to a second 3D coordinate system using a dimension reduction method, a minimum axis of the second 3D coordinate system being along the certain direction; and

determining whether the time-dependent velocity in the certain direction drops from the first speed value to the second speed value based on the positions of the object in the second 3D coordinate system.

4. The method of claim 3, wherein:

mapping the positions of the object from the first 3D coordinate system to the second 3D coordinate system includes mapping the positions using a principal component analysis method, and

the minimum axis has a smallest eigenvalue among axes of the second 3D coordinate system, an eigenvalue of an axis of the second 3D coordinate system relating to an amount of variation of the positions along the axis in the second 3D coordinate system.

5. The method of claim 1, further comprising:

determining whether the tapping position is on an existing touch interactive surface of a human-machine interaction system;

if the tapping position is on the existing touch interactive surface, using the tapping position to increase a resolution of the existing touch interactive surface;

if the tapping position is slightly off the existing touch interactive surface, updating the existing touch interactive surface using the tapping position or deleting the existing touch interactive surface; and

if the tapping position is not associated with the existing touch interactive surface, initiating a process for creating a new touch interactive surface.

6. The method of claim 1, further comprising:

determining a plurality of tapping positions of the object; and

creating a touch interactive surface based on the tapping positions.

7. The method of claim 6, wherein creating the touch interactive surface based on the tapping positions includes:

plugging coordinate values of each of the tapping positions into a polynomial surface fitting equation containing a plurality of parameters; and

determining best values for the parameters of the polynomial surface fitting equation using a regression method.

8. The method of claim 6, wherein creating the touch interactive surface based on the tapping positions includes creating the touch interactive surface using a point cloud method.

9. The method of claim 6, further comprising:

after the touch interactive surface is determined, continuing to detect positions of the object by the sensor; and

determining a distance from the object to the touch interactive surface based on a position of the object and the touch interactive surface.

10. The method of claim 1, further comprising:

determining a plurality of tapping positions of the object;

calculating a covariance matrix using the tapping positions;

after the covariance matrix is calculated, continuing to monitor the position of the object; and

determining a distance from the object to a touch interactive surface of a human-machine interaction system represented by the tapping positions using a Gaussian regression method.

11. An apparatus for human-machine interaction, comprising:

a sensing device configured to monitor a movement of an object by detecting positions of the object over time; and

a processor configured to;

generate a time-dependent velocity of the object based on the movement of the object;

detect a tapping event of the object tapping on a surface by detecting a sudden change of the time-dependent velocity; and

determine a position of the object at a time when the tapping event occurs as a tapping position of the object;

wherein the processor is further configured to;

determine whether the time-dependent velocity in a certain direction drops from a first speed value to a second speed value within a first time threshold, the first speed value being larger than the second speed value, and the second speed value being close to zero, and

determine whether the time-dependent velocity in the certain direction keeps at about the second speed value for longer than a second time threshold; and

determine the tapping event has occurred if the time-dependent velocity in the certain direction drops from the first speed value to the second speed value within the first period of time and the time-dependent velocity in the certain direction keeps at about the second speed value for longer than the second time threshold.

12. The apparatus of claim 11, wherein the processor is further configured to:

record a time-dependent position value of the object based on the detected positions over time; and

convert the time-dependent position value to the time-dependent velocity by differentiation.

13. The apparatus of claim 12, wherein the processor is further configured to:

record the positions of the object in a first three-dimensional (3D) coordinate system;

map the positions of the object from the first 3D coordinate system to a second 3D coordinate system using a dimension reduction method, a minimum axis of the second 3D coordinate system being along the certain direction; and

determine whether the time-dependent velocity in the certain direction drops from the first speed value to the second speed value based on the positions of the object in the second 3D coordinate system.

14. The apparatus of claim 11, wherein the processor is further configured to:

determine whether the tapping position is on an existing touch interactive surface of a human-machine interaction system;

if the tapping position is on the existing touch interactive surface, use the tapping position to increase a resolution of the existing touch interactive surface;

if the tapping position is slightly off the existing touch interactive surface, update the existing touch interactive surface using the tapping position or delete the existing touch interactive surface; and

if the tapping position is not associated with the existing touch interactive surface, initiate a process for creating a new touch interactive surface.

15. The apparatus of claim 11, wherein the processor is further configured to:

determine a plurality of tapping positions of the object; and

create a touch interactive surface based on the tapping positions.

16. The apparatus of claim 11, wherein the processor is further configured to:

determine a plurality of tapping positions of the object;

calculate a covariance matrix using the tapping positions;

after the covariance matrix is calculated, continue to monitor the position of the object; and

determine a distance from the object to a touch interactive surface of a human-machine interaction system represented by the tapping positions using a Gaussian regression method.