State machine based tracking system for screen pointing control

• US 10,222,869 B2
• Filed: 08/03/2015
• Issued: 03/05/2019
• Est. Priority Date: 08/03/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Moran, Amit
• Primary Examiner(s): Danielsen, Nathan
• Application Number:
  US14/816,953
• Publication Number:
  US 20170038857A1
• Time in Patent Office: 1,310 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0393 Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

Generally, this disclosure provides systems, devices, methods and computer readable media for state machine based pointing control. A method may include receiving a position estimate of a first location associated with a first portion of a pointing device and a position estimate of a second location associated with a second portion of the pointing device; calculating a vector from the estimated position of the first location to the estimated position of the second location; and resolving the vector into a first distance component (Dx) and a second distance component (Dy), the Dy component orthogonal to the Dx component. The method may further include tracking temporal changes of the Dx and Dy components; updating an interaction state based on a rate of change of the Dx and Dy components; and moving a cursor position on a display element screen based on the temporal change and the interaction state.

Claims

1. A system for pointing control, said system comprising:

vector calculation circuitry to;

receive a position estimate of a first location associated with a first portion of a pointing device and a position estimate of a second location associated with a second portion of said pointing device;

calculate a vector from said estimated position of said first location to said estimated position of said second location; and

resolve said vector into a first distance component (Dx) and a second distance component (Dy), said Dy component orthogonal to said Dx component, wherein said first distance component (Dx) and said second distance component (Dy) represent a distance between the first and second locations in an x plane and a y plane, respectively, and a length of said vector in said x and y plane, wherein the pointing device is configured such that the length of said vector can change in response to movement of said pointing device;

tracking circuitry to track temporal change of said Dx and Dy components and to calculate a rate of change of said Dx and Dy components;

state machine circuitry to maintain and update an interaction state based on said rate of change of said Dx and Dy components; and

screen coordinate update circuitry to move a cursor position on a display element screen to a new screen coordinate based on said temporal change and said interaction state;

wherein;

said interaction state comprises a coarse pointing state, a fine pointing state and a freeze pointing state;

cursor position movement associated with said coarse pointing state is greater than cursor position movement associated with said fine pointing state; and

cursor position movement is disabled in association with said freeze pointing state; and

the system further comprises stabilization circuitry to perform different degrees of temporal low pass filtering of said cursor position movement based on whether said interaction state is said coarse pointing state, said fine pointing state, or said freeze pointing state.

2. The system of claim 1, wherein said pointing device is a hand and said first location is associated with the center of mass of the palm of said hand and said second location is associated with the tip of the index finger of said hand.

3. The system of claim 1, further comprising a 3-dimensional (3-D) visual tracking system to provide said estimated position of said first location and said estimated position of said second location.

4. The system of claim 1, wherein movement of said pointing device causes said first location to move relative to said second location, thereby causing a change in the length of said vector.

5. The system of claim 1, further comprising calibration circuitry to determine a maximum value of said Dx and Dy components based on the length of said pointing device;

and to normalize said Dx and Dy components based on said maximum value.

6. The system of claim 5, further comprising screen coordinate projection circuitry to scale said normalized Dx and Dy components based on resolution of said display element screen, wherein said new screen coordinate is further based on said scaled Dx and Dy components.

7. The system of claim 1, further comprising user initiation detection circuitry to detect user initiation of said pointing control system based on detection of proximity of said user to said display element screen crossing a threshold distance, or based on detection of a gesture of said user.

8. A method for pointing control, said method comprising:

receiving a position estimate of a first location associated with a first portion of a pointing device and a position estimate of a second location associated with a second portion of said pointing device;

calculating a vector from said estimated position of said first location to said estimated position of said second location;

resolving said vector into a first distance component (Dx) and a second distance component (Dy), said Dy component orthogonal to said Dx component, wherein said first distance component (Dx) and said second distance component (Dy) represent a distance between the first and second locations in an x plane and a y plane, respectively, and a length of said vector in said x and y plane, wherein the length of said vector can change in response to movement of said pointing device;

tracking temporal change of said Dx and Dy components;

updating an interaction state based on a rate of change of said Dx and Dy components; and

moving a cursor position on a display element screen based on said temporal change and said interaction state,

wherein;

said interaction state comprises a coarse pointing state, a fine pointing state and a freeze pointing state;

cursor position movement associated with said coarse pointing state is greater than cursor position movement associated with said fine pointing state; and

cursor position movement is disabled in association with said freeze pointing state; and

the method further comprises performing different degrees of temporal low pass filtering of said cursor position movement based on whether said interaction state is said coarse pointing state, said fine pointing state, or said freeze pointing state.

9. The method of claim 8, wherein said pointing device is a hand and said first location is associated with the center of mass of the palm of said hand and said second location is associated with the tip of the index finger of said hand.

10. The method of claim 8, further comprising receiving said position estimates from a 3-dimensional (3-D) visual tracking system.

11. The method of claim 8, wherein movement of said pointing device causes said first location to move relative to said second location, thereby causing a change in the length of said vector.

12. The method of claim 8, further comprising performing a calibration to determine a maximum value of said Dx and Dy components based on the length of said pointing device;

and normalizing said Dx and Dy components based on said maximum value.

13. The method of claim 8, further comprising detecting user initiation of said pointing control based on detection of proximity of said user to said display element screen crossing a threshold distance, or based on detection of a gesture of said user.

14. A non-transitory computer-readable storage medium having instructions stored thereon which when executed by a processor result in the following operations for pointing control, said operations comprising:

receiving a position estimate of a first location associated with a first portion of a pointing device and a position estimate of a second location associated with a second portion of said pointing device;

calculating a vector from said estimated position of said first location to said estimated position of said second location;

resolving said vector into a first distance component (Dx) and a second distance component (Dy), said Dy component orthogonal to said Dx component, wherein said first distance component (Dx) and said second distance component (Dy) represent a distance between the first and second locations in an x plane and a y plane, respectively, and a length of said vector in said x and y plane, wherein the length of said vector can change in response to movement of said pointing device;

tracking temporal change of said Dx and Dy components;

updating an interaction state based on a rate of change of said Dx and Dy components; and

moving a cursor position on a display element screen based on said temporal change and said interaction state;

wherein;

said interaction state comprises a coarse pointing state, a fine pointing state and a freeze pointing state;

cursor position movement associated with said coarse pointing state is greater than cursor position movement associated with said fine pointing state; and

cursor position movement is disabled in association with said freeze pointing state; and

said instructions when executed by said processor further result in performance of the following operations comprising;

performing different degrees of temporal low pass filtering of said cursor position movement based on whether said interaction state is said coarse pointing state, said fine pointing state, or said freeze pointing state.

15. The non-transitory computer-readable storage medium of claim 14, wherein said pointing device is a hand and said first location is associated with the center of mass of the palm of said hand and said second location is associated with the tip of the index finger of said hand.

16. The non-transitory computer-readable storage medium of claim 14, further comprising receiving said position estimates from a 3-dimensional (3-D) visual tracking system.

17. The non-transitory computer-readable storage medium of claim 14, wherein movement of said pointing device causes said first location to move relative to said second location, thereby causing a change in the length of said vector.

18. The non-transitory computer-readable storage medium of claim 14, further comprising performing a calibration to determine a maximum value of said Dx and Dy components based on the length of said pointing device;

and normalizing said Dx and Dy components based on said maximum value.

19. The non-transitory computer-readable storage medium of claim 14, further comprising detecting user initiation of said pointing control based on detection of proximity of said user to said display element screen crossing a threshold distance, or based on detection of a gesture of said user.