Autonomous haptic stylus

• US 10,671,186 B2
• Filed: 06/15/2016
• Issued: 06/02/2020
• Est. Priority Date: 06/15/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ribeiro, Flavio Protasio, Churikov, Anatoly Yuryevich
• Primary Examiner(s): Danielsen, Nathan
• Application Number:
  US15/182,946
• Publication Number:
  US 20170364167A1
• Time in Patent Office: 1,448 Days
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/04162 for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
• CPC Class Codes: G06F 3/0441 using active external devices, e.g. active pens, for receiving changes in electrical potential transmitted by the digitiser, e.g. tablet driving signals
• CPC Class Codes: G06F 3/0442 using active external devices, e.g. active pens, for transmitting changes in electrical potential to be received by the digitiser
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes

Abstract

An autonomous haptic stylus provides tactile feedback to a user writing or drawing on a smooth, uniform, touch-sensitive, glass display screen of a tablet, laptop computer, credit card point of sale device, or other device allowing a user to write or draw thereon. The stylus has an electrostatic detection sensor or a force/pressure sensor, which allows the stylus to detect certain characteristics of the display screen and to autonomously determine the position and/or velocity of the stylus on the display screen based on those characteristics. The position and/or velocity can be used to tailor the tactile feedback to the user. A display screen digitizer that provides driving signals for the row and column electrodes of a touch-sensitive display screen can be modified to transmit row and column information over the electrodes to allow a stylus to determine its position.

Claims

1. A stylus for use with a display screen which is touch-sensitive, the stylus comprising:

a sensor positioned at an end of the stylus and configured to provide a plurality of sensor signals in response to the stylus contacting the display screen or being positioned within a threshold distance of the display screen,

wherein the sensor comprises a force sensor responsive to changes in a physical surface texture of the display screen to provide one or more of the plurality of sensor signals responsive to the stylus being moved on the display screen,

wherein the physical surface texture includes bumps arranged in a pattern on a surface of the display screen, and

wherein the pattern is at least one of a random pattern, a pseudo random pattern, or a uniformly spaced pattern;

an actuator, within the stylus, configured to vibrate the stylus in response to a control signal;

a memory, within the stylus, containing operating instructions;

a processor, within the stylus and connected to the memory, to execute the operating instructions, and configured to;

determine a velocity of the stylus based on the one or more of the plurality of sensor signals responsive to the changes in the physical surface texture of the display screen; and

provide the control signal to the actuator based upon the velocity; and

a power supply, within the stylus, configured to provide operating power to at least the processor.

2. The stylus of claim 1, wherein the processor is further configured to:

process one or more of the plurality of sensor signals by detecting changes in the one or more of the plurality of sensor signals; and

determine the velocity of the stylus further based at least in part upon the changes.

3. The stylus of claim 1, wherein the memory comprises data defining a vibration pattern, and the processor is further configured to:

retrieve the vibration pattern from the memory; and

generate the control signal to correspond to the vibration pattern, wherein the control signal causes the actuator to vibrate in accordance with the vibration pattern.

4. The stylus of claim 1, wherein the memory comprises statistical information regarding vibrations as a pencil is drawn across paper, and the processor is further configured to:

retrieve the statistical information from the memory; and

generate the control signal to correspond to the statistical information, wherein the control signal causes the actuator to vibrate according to the statistical information.

5. The stylus of claim 1, wherein the sensor comprises an electrostatic sensor to detect electrostatic signals provided by the display screen to provide the plurality of sensor signals.

6. The stylus of claim 1, wherein:

the sensor comprises an electrostatic sensor responsive to a first reference signal provided by the display screen to provide a first sensor signal, and responsive to a second reference signal provided by the display screen to provide a second sensor signal; and

the processor is further configured to;

process the first sensor signal by determining a first position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a first time;

determine a second position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a second time; and

determine the velocity further based at least in part upon a difference between the first position and the second position.

7. The stylus of claim 1, wherein:

the sensor comprises an electrostatic sensor responsive to a first reference signal provided by the display screen to provide a first sensor signal, and responsive to a second reference signal provided by the display screen to provide a second sensor signal; and

the processor is further configured to;

process the first sensor signal by determining a first position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a first time;

determine a second position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a second time; and

if both the first position and the second position are within a predetermined area on the display screen, determine the velocity further based at least in part upon a difference between the first position and the second position.

8. The stylus of claim 1, wherein:

the processor is further configured to;

process the one or more of the plurality of sensor signals to determine a characteristic of the one or more of the plurality of sensor signals; and

determine the velocity further based at least in part upon the characteristic of the one or more of the plurality of sensor signals.

9. The stylus of claim 1, wherein:

the processor is further configured to;

determine the velocity based at least in part upon a time difference between a first occurrence of the one or more of the plurality of sensor signals having a predetermined characteristic corresponding to the physical surface texture and a subsequent occurrence of the one or more of the plurality of sensor signals having the predetermined characteristic.

10. The stylus of claim 1, wherein the processor is further configured to:

process the one or more of the plurality of sensor signals to determine whether the stylus is within a predetermined area on the display screen; and

determine the velocity further based at least in part upon the stylus being within the predetermined area.

11. The stylus of claim 1, wherein the processor is further configured to:

process the one or more of the plurality of sensor signals to determine a change in position of the stylus; and

determine the velocity further based at least in part upon the change.

12. The stylus of claim 1, wherein the sensor further comprises an electrostatic sensor, wherein the plurality of sensor signals comprise electrode identification information of one or more electrodes of the display screen and a measured signal strength between the sensor and each of the one or more electrodes, wherein the electrode identification information includes one or more of a time, a frequency, a code, or an address to permit identification of a corresponding electrode of the one or more electrodes, and

wherein the processor is further configured to:

receive a first sensor signal of the plurality of sensor signals;

process the first sensor signal to determine a first position of the stylus relative to each of the one or more electrodes based on the electrode identification information and the measured signal strength of the first sensor signal;

receive a second sensor signal of the plurality of sensor signals;

process the second sensor signal to determine a second position of the stylus relative to each of the one or more electrodes based on the electrode identification information and the measured signal strength of the second sensor signal; and

determine the velocity of the stylus further based on the first position, the second position, and a frequency of measuring the plurality of sensor signals.

13. A memory having operating instructions stored thereupon that, when executed by a processor in a stylus, cause the processor to:

receive a plurality of sensor signals from a sensor in the stylus, the sensor being at an end of the stylus which is adapted to be placed into contact with a display screen which is touch-sensitive or placed within a threshold distance of the display screen,

wherein the sensor comprises a force sensor responsive to changes in a physical surface texture of the display screen to provide one or more of the plurality of sensor signals responsive to the stylus being moved on the display screen,

wherein the physical surface texture includes bumps arranged in a pattern on a surface of the display screen, and

wherein the pattern is at least one of a random pattern, a pseudo random pattern, or a uniformly spaced pattern;

determine a velocity of the stylus based on the one or more of the plurality of sensor signals responsive to the changes in the physical surface texture of the display screen; and

provide a control signal, based upon the velocity, to an actuator in the stylus to cause the actuator to vibrate the stylus.

14. The memory of claim 13, wherein the operating instructions in the memory further cause the processor to:

process one or more of the plurality of sensor signals by detecting changes in the one or more of the plurality of sensor signals; and

determine the velocity of the stylus further based upon at least in part upon the changes.

15. The memory of claim 13, wherein the memory comprises data defining a vibration pattern, and the operating instructions in the memory further cause the processor to retrieve the vibration pattern from the memory and generate the control signal to correspond to the vibration pattern to cause the actuator to vibrate in accordance with the vibration pattern.

16. The memory of claim 13, wherein the memory comprises statistical information regarding vibrations as a pencil is drawn across paper, and the operating instructions in the memory further cause the processor to retrieve the statistical information from the memory and generate the control signal to correspond to the statistical information to cause the actuator to vibrate according to the statistical information.

17. The memory of claim 13, wherein the sensor comprises an electrostatic sensor to detect a first reference signal provided by a display screen to provide a first sensor signal, and a second reference signal provided by the display screen to provide a second sensor signal, and

wherein the operating instructions in the memory further cause the processor to:

process the first sensor signal by determining a first position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a first time;

determine a second position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a second time; and

determine the velocity of the stylus further based at least in part upon a difference between the first position and the second position.

18. The memory of claim 13, wherein the sensor comprises an electrostatic sensor to detect a first reference signal provided by a display screen to provide a first sensor signal and a second reference signal provided by the display screen to provide a second sensor signal;

and wherein;

the operating instructions in the memory further cause the processor to;

process the first sensor signal by determining a first position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a first time; and

process the second sensor signal by determining a second position of the stylus on the display screen based upon the first sensor signal and the second sensor signal at a second time; and

if both the first position and the second position are within a predetermined area on the display screen, to determine the velocity of the stylus further based at least in part upon a difference between the first position and the second position.

19. The memory of claim 13, wherein:

the operating instructions in the memory further cause the processor to;

determine the velocity by determining a time difference between a first occurrence of the one or more of the plurality of sensor signals having a predetermined characteristic corresponding to the physical surface texture and a subsequent occurrence of the one or more of the plurality of sensor signals having the predetermined characteristic.

20. The memory of claim 13, wherein the operating instructions in the memory further cause the processor to:

process the one or more of the plurality of sensor signals to determine whether the stylus is within a predetermined area on the display screen; and

determine the velocity further based at least in part upon the stylus being within the predetermined area.

21. The memory of claim 13, wherein the operating instructions in the memory further cause the processor to:

process the one or more of the plurality of sensor signals to determine a change in position of the stylus; and

determine the velocity further based at least in part upon the change.

22. The memory of claim 13, wherein the sensor further comprises an electrostatic sensor, wherein the plurality of sensor signals comprise electrode identification information of one or more electrodes of the display screen and a measured signal strength between the sensor and each of the one or more electrodes, wherein the electrode identification information includes one or more of a time, a frequency, a code, or an address to permit identification of a corresponding electrode of the one or more electrodes, and

wherein the operating instructions in the memory further cause the processor to:

process a first sensor signal of the plurality of sensor signals to determine a first position of the stylus relative to each of the one or more electrodes based on the electrode identification information and the measured signal strength of the first sensor signal;

process a second sensor signal of the plurality of sensor signals to determine a second position of the stylus relative to each of the one or more electrodes based on the electrode identification information and the measured signal strength of the second sensor signal; and

determine a velocity of the stylus based on the first position, the second position, and a frequency of measuring the plurality of sensor signals.