Intelligent stylus

• US 9,921,684 B2
• Filed: 05/02/2016
• Issued: 03/20/2018
• Est. Priority Date: 06/22/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Falkenburg, David R., Simon, David I., Harley, Jonah A., Mucignat, Andrea, Land, Brian R., Mullens, Christopher Tenzin, Hotelling, Steven P.
• Primary Examiner(s): Lee, Jr., Kenneth B
• Application Number:
  US15/144,615
• Publication Number:
  US 20160357343A1
• Time in Patent Office: 687 Days
• Field of Search: None
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 21/83 input devices, e.g. keyboards, mice or controllers thereof
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77

Abstract

An intelligent stylus is disclosed. The stylus can provide a stylus condition in addition to a touch input. The stylus architecture can include multiple sensors to sense information indicative of the stylus condition, a microcontroller to determine the stylus condition based on the sensed information, and a transmitter to transmit the determined condition to a corresponding touch sensitive device so as to cause some action based on the condition.

Claims

1. An input device comprising:

a camera configured to capture one or more images;

a motion sensor;

an orientation sensor; and

a controller configured to;

receive the one or more captured images, the motion data, and the orientation data;

detect that the input device is not proximate to a touch screen of an associated host device; and

in response to detecting that the input device is not proximate to the touch screen of the associated host device, transmit information associated with one or more of the captured one or more images, the motion data, and the orientation data so as to cause a writing action at the associated host device.

2. The input device of claim 1, further comprising an identification sensor configured to detect a code associated with the input device, wherein the transmitter is further configured to transmit the code so as to cause an authentication of the input device.

3. The input device of claim 1, further comprising a light emitter configured to emit light, and transmit to the controller an indication the light emitter emits light.

4. The input device of claim 1, wherein the one or more captured images comprise one of video footage and one or more still images.

13. The input device of claim 1, wherein detecting that the input device is not proximate to the touch screen of the associated host device comprises detecting that a tip of the input device is proximate to piece of paper.

14. The input device of claim 1, wherein detecting that the input device is not proximate to the touch screen of the associated host device comprises extracting one or more of the input device'"'"'s location, the input device'"'"'s environment, the input device'"'"'s motion, the input device'"'"'s orientation, the host device'"'"'s displayed information, the host device'"'"'s position, the host device'"'"'s motion, and the host device'"'"'s orientation from the one or more captured images.

5. A method of operating an input device, the method comprising:

capturing, with a camera of the input device, one or more images;

capturing, with a motion sensor of the input device, motion data;

capturing, with an orientation sensor of the input device, orientation data;

receiving the one or more captured images, the motion data, and the orientation data at a controller;

detecting that the input device is not proximate to a touch screen of an associated host device; and

in response to detecting that the input device is not proximate to the touch screen of the associated host device, transmitting information associated with one or more of the captured one or more images, the motion data, and the orientation data so as to cause a writing action at the associated host device.

6. The method of claim 5, wherein the input device further comprises an identification sensor configured to detect a code associated with the input device, and the transmitter is further configured to transmit the code so as to cause an authentication of the input device.

7. The method of claim 5, further comprising:

emitting light with a light emitter; and

transmitting to the controller an indication that the light emitter emits light.

8. The method of claim 5, wherein the one or more captured images comprise one of video footage and one or more still images.

15. The method of claim 5, wherein detecting that the input device is not proximate to the touch screen of the associated host device comprises detecting that a tip of the input device is proximate to piece of paper.

16. The method of claim 5, wherein detecting that the input device is not proximate to the touch screen of the associated host device comprises extracting one or more of the input device'"'"'s location, the input device'"'"'s environment, the input device'"'"'s motion, the input device'"'"'s orientation, the host device'"'"'s displayed information, the host device'"'"'s position, the host device'"'"'s motion, and the host device'"'"'s orientation from the one or more captured images.

9. A non-transitory computer readable storage medium storing executable program instructions, which, when executed by an input device with a controller, cause the input device to:

capture, with a camera of the input device, one or more images;

capture, with a motion sensor of the input device, motion data;

capture, with an orientation sensor of the input device, orientation data;

receive the one or more captured images, the motion data, and the orientation data at the controller;

detect that the input device is not proximate to a touch screen of an associated host device; and

in response to detecting that the input device is not proximate to the touch screen of the associated host device, transmit information associated with one or more of the captured one or more images, the motion data, and the orientation data so as to cause a writing cation at the associated host device.

10. The non-transitory computer readable storage medium of claim 9, wherein the input device further comprises an identification sensor configured to detect a code associated with the input device, and the transmitter is further configured to transmit the code so as to cause an authentication of the input device.

11. The non-transitory computer readable storage medium of claim 9, wherein the program instructions further cause the input device to:

emit light with a light emitter; and

transmit to the controller an indication that the light emitter emits light.

12. The non-transitory computer readable storage medium of claim 9, wherein the one or more captured images comprise one of video footage and one or more still images.

17. The non-transitory computer readable storage medium of claim 9, wherein detecting that the input device is not proximate to the touch screen of the associated host device comprises detecting that a tip of the input device is proximate to piece of paper.

18. The non-transitory computer readable storage medium of claim 9, wherein detecting that the input device is not proximate to the touch screen of the associated host device comprises detecting that a tip of the input device is proximate to piece of paper.