Pointing detection

• US 10,025,492 B2
• Filed: 02/08/2016
• Issued: 07/17/2018
• Est. Priority Date: 02/08/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Pellikka, Matti
• Primary Examiner(s): Cheng, Joe H
• Application Number:
  US15/018,848
• Publication Number:
  US 20170228124A1
• Time in Patent Office: 890 Days
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A pointing detecting device is described. In an embodiment, a device comprises: a display; a processor; a storage containing instructions; wherein the instructions cause the processor to: select a set of points from a point cloud representing a pointing object in proximity or in contact of the display; determine the selected set of points as control points of a computational smooth surface representing the pointing object; determine a point nearest to the display on the computational smooth surface; based on the point nearest to the display, determine a point in a direction of a gradient of a curvature of the computational smooth surface; and report the point as a location of a desired pointing or touch action.

Claims

1. A device, comprising:

a display;

a processor;

a storage containing instructions;

wherein the instructions cause the processor to;

develop a point cloud representing a pointing object in proximity to or in contact with the display;

select a set of points from the point cloud;

determine the selected set of points as control points of a computational smooth surface representing the pointing object;

determine a point nearest to the display on the computational smooth surface;

based on the point nearest to the display, determine a point in a direction of a gradient of a curvature of the computational smooth surface; and

report the point as a location of a desired pointing or touch action.

2. The device of claim 1, wherein the computational smooth surface comprises a non-uniform rational B-spline, NURBS, surface.

3. The device of claim 2, wherein the NURBS surface comprises a 4^th order NURBS surface.

4. The device of claim 1 wherein the location is reported as a tip of the pointing object to at least one of an application or an operating system.

5. The device of claim 1 wherein the storage contains instructions which further cause the processor to:

calculate a matrix of second derivatives of the z-component of the computational smooth surface at the point nearest to the display;

calculate an eigenvector which corresponds to the largest eigenvalue of the matrix;

calculate an angle from the eigenvector and dot product of the eigenvector with the gradient of curvature; and

report the angle as azimuthal angle of the pointing object.

6. The device of claim 5, wherein the angle is reported as an orientation to at least one of an application or an operating system.

7. The device of claim 5, wherein the storage contains instructions which further cause the processor to:

calculate a ratio of eigenvalues of the matrix;

calculate an angle of elevation of the pointing object based on the ratio; and

report the angle of elevation.

8. The device of claim 7, wherein the angle of elevation is reported to at least one of an application or an operating system.

9. The device of claim 1, wherein the display comprises at least one of a capacitive touchscreen, a resistive touchscreen, an optical imaging based touchscreen, inductive touch screen and an ultrasound based touch screen.

10. The device of claim 1, wherein the computational smooth surface comprises a convex surface.

11. The device of claim 1, wherein the pointing object comprises a touching or pointing apparatus.

12. The device of claim 1, wherein determination of the point closest to the display comprises using Newton'"'"'s iterative method.

13. The device of claim 1, wherein the determining a gradient of curvature of the computational smooth surface comprises determining a gradient of Gaussian curvature of a NURBS surface.

14. The device of claim 1, wherein the storage contains instructions which cause the processor to perform or make the device perform an action in response to a report containing the location.

15. The device of claim 1, wherein the instructions is executed in real-time and cause the processor to perform the instruction in the real-time.

16. A device, comprising:

a touch sensitive element;

a processor;

a storage comprising a set of instructions; and

wherein the set of instructions is executed in and causes the processor to;

develop a point cloud representing a pointing object in proximity to or in contact with the touch sensitive element;

select a set of points from the point cloud;

use the selected set of points as control points of a non-uniform rational B-spline, NURBS, surface representing the pointing object;

determine a point nearest to the touch sensitive element on the NURBS surface;

based on the point nearest to the touch sensitive element, determine a point in a direction of a gradient of a curvature of the NURBS surface;

calculate a matrix of second derivatives in the z-direction at the point nearest to the touch sensitive element;

calculate an azimuthal angle from an eigenvector and dot product of the eigenvector with the gradient of curvature;

calculate a ratio of eigenvalues of the matrix;

calculate an angle of elevation of the pointing object based on the ratio;

report the point in the direction of a gradient of a curvature of the NURBS surface as a location of a pointing or a touch action, the azimuthal angle, angle of elevation; and

perform an action in response to the report, the action being based on at least one of location of the pointing or touch action, orientation of the pointing object, and the angle of the elevation.

17. The device of claim 16, wherein the pointing object comprises one of a human appendage, an active stylus, or passive stylus.

18. A method, comprising:

developing a point cloud representing a pointing object in proximity to or in contact with a screen;

selecting a set of points from the point cloud;

using the selected points as control points of a computational smooth surface representing the pointing object;

determining a point nearest to the screen on the computational smooth surface;

determining a point in a direction of a gradient of a curvature of the computational smooth surface from the point nearest to the screen; and

reporting the point as a location of a hover or a touch action.

19. The method of claim 18, further comprising:

calculating a matrix of second derivatives of the z-component of the computational smooth surface at the point nearest to the screen;

calculating an eigenvector which corresponds to the larger eigenvalue of the matrix;

calculating an angle from the eigenvector and dot product of the eigenvector with the gradient of curvature; and

reporting the angle as azimuthal angle of the pointing object.

20. The method of claim 19, further comprising:

calculating a ratio of eigenvalues of the matrix;

calculating an angle of elevation of the pointing object based on the ratio; and

reporting the angle of elevation.