Texture capture stylus and method

• US 9,817,489 B2
• Filed: 01/27/2014
• Issued: 11/14/2017
• Est. Priority Date: 01/27/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lor, Jason, Carroll, Patrick A., Rhodes, Glen A., Verhoeve, Dustin J.
• Primary Examiner(s): Patel, Kumar
• Assistant Examiner(s): Ahn, Sejoon
• Application Number:
  US14/165,501
• Publication Number:
  US 20150212602A1
• Time in Patent Office: 1,387 Days
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]

Abstract

Embodiments disclosed herein relate generally to a stylus for texture capture. The stylus includes an image sensing and capture device to permit a surface to be scanned using the stylus and an image may be stored and displayed on an electronic device to represent the texture of the scanned surface.

Claims

1. A stylus comprising:

a housing;

an image sensing device in the housing;

a light transmissive tip on the housing that contacts a surface adjacent to the light transmissive tip and transmits light from the surface to the image sensing device so that the image sensing device captures an image of the surface based on the transmitted light, wherein the surface includes distance variations between the surface and the light transmissive tip, and wherein the image captured by the image sensing device includes the distance variations;

a haptic output device that generates a haptic output based on the image captured by the image sensing device in the housing of the stylus; and

a light source that illuminates the surface to provide additional light to the image sensing device at least while the image sensing device captures the image, and wherein the additional light provided by the light source is adjusted based on the captured image.

2. The stylus of claim 1, wherein the surface has a texture,

wherein the captured image includes the texture of the surface, and wherein the haptic output is based on the texture.

3. The stylus of claim 1, wherein the image sensing device includes a camera.

4. The stylus of claim 1, wherein the light transmissive tip comprises a lens that transmits the light from the surface to the image sensing device.

5. The stylus of claim 1, wherein the image sensing device is operative to capture a texture of the surface and a color of the surface, and wherein the captured image includes the texture of the surface and the color of the surface.

6. The stylus of claim 1 further comprising a power source that provides power to the stylus.

7. The stylus of claim 1 further comprising an inertial measurement unit that generates position data indicative of a position of the stylus relative to the surface.

8. A system comprising:

an electronic device that includes a display screen; and

a stylus in electrical communication with the electronic device, the stylus comprising;

a tip at one end of the stylus;

an image sensing device that generates image data in response to light from a surface of an object other than the electronic device, wherein the tip is light transmissive and transmits light from the surface to the image sensing device, wherein the surface includes distance variations between the surface and the transmissive tip, and wherein the image data captured by the image sensing device includes the distance variations; and

a haptic output device coupled to the image sensing device that produces a haptic output based on the image data, wherein the haptic output is produced based on the distance variations in the image data.

9. The system of claim 8, wherein the haptic output device produces the haptic output based on the distance variations as the image sensing device generates the image data.

10. The system of claim 8, wherein the image sensing device includes a camera.

11. The system of claim 8, wherein the distance variations form a three-dimensional texture of the surface, and wherein the haptic output device produces the haptic output as the stylus moves over the three-dimensional texture of the surface.

12. The system of claim 8 further comprising an inertial measurement unit that generates position data indicative of a position of the stylus relative to the surface.

16. The system of claim 8, wherein the image data is used to generate an image of the surface, and wherein the image of the surface is displayed on the display screen.

13. A method for operating an electronic device having a display screen, the method comprising:

scanning a stylus over a plurality of different surfaces,

wherein the plurality of different surfaces each include different distance variations between the plurality of different surfaces and the stylus;

capturing images of the plurality of different surfaces with the stylus;

storing the captured images in memory in the stylus;

selecting one of the captured images for reproduction on the display screen of the electronic device, wherein the selected one of the captured images includes the distance variations of one of the plurality of different surfaces;

transmitting the selected one of the captured images to the electronic device; and

as the stylus is moved relative to the display screen, displaying the selected one of the captured images on the display screen of the electronic device in response to the movement of the stylus relative to the display screen.

14. The method of claim 13, wherein each of the plurality of different surfaces has a different texture, the method further comprising extracting the different texture from the captured images.

15. The method of claim 13, wherein one of the plurality of different surfaces does not form part of the electronic device, and wherein capturing the images comprises capturing an image of the surface that does not form part of the electronic device.

17. A system comprising:

an electronic device having a display screen; and

a stylus in wireless communication with the electronic device, the stylus comprising;

a tip;

an image sensing device that generates image data in response to light from a surface that has a three-dimensional texture, wherein the surface is an object other than the electronic device, and wherein the three-dimensional texture of the surface includes variations between the surface and the tip;

a light source that illuminates the surface to provide additional light to the image sensing device at least while the image sensing device generates the image data, wherein the additional light provided by the light source is adjusted based on the image data; and

a wireless transmitter that transmits the image data to the electronic device, wherein the image data is displayed on the display screen as a two-dimensional representation of the distance variations of the three-dimensional texture.

18. The system of claim 17, the stylus further comprising:

a haptic output device that generates a haptic output based on the image data.

19. The system of claim 17, wherein the tip is transparent and comprises a lens that directs the light from the surface to the image sensing device.