Systems and methods for haptic and gesture-driven paper simulation

• US 9,547,366 B2
• Filed: 03/14/2013
• Issued: 01/17/2017
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Ullrich, Christopher J., Birnbaum, David M., Bothsa, Marcus Aurelius
• Primary Examiner(s): Shih, Haoshian
• Application Number:
  US13/827,951
• Publication Number:
  US 20140281954A1
• Time in Patent Office: 1,405 Days
• Field of Search: 715/702, 715/776, 345/156, 345/173, 345/179
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 15/0291 for reading, e.g. e-books constructional details of portable computers G06F1/1613
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Systems and methods for haptic and gesture-driven paper simulation are disclosed. For example, one disclosed method includes the steps of receiving an electronic document; receiving metadata associated with the electronic document, the metadata comprising a characteristic indicating a type of paper; generating and transmitting a display signal configured to cause display of the at least a portion of the document; and generating and transmitting a haptic signal based on the type of paper, the haptic signal configured to cause a haptic output device to generate a haptic effect.

Claims

1. A method comprising:

receiving an electronic document comprising a plurality of pages;

receiving metadata indicating a type of paper associated with the electronic document;

associating the metadata with the electronic document based on a second characteristic associated with the electronic document;

generating and transmitting a display signal configured to cause display of the at least a portion of the electronic document;

in response to a multi-page turn gesture on a touch sensitive input device, performing a multi-page turn by causing multiple pages of the plurality of pages to turn in succession; and

generating and transmitting a haptic signal in response to a particular page of the plurality of pages being passed during the multi-page turn, the haptic signal configured to cause a haptic output device to output a haptic effect.

2. The method of claim 1, further comprising:

receiving contact data based on an interaction with the touch sensitive input device, the contact data associated with the portion of the electronic document;

determining a gesture based on the contact data;

determining a deformation of the portion of the document based on the gesture;

generating and transmitting a second display signal configured to cause display of the portion of the document comprising the deformation; and

generating and transmitting a second haptic signal based on the type of paper and the deformation, the second haptic signal configured to cause the haptic output device to generate a second haptic effect, the second haptic effect configured to indicate the deformation.

3. The method of claim 2, wherein the gesture comprises a page-turn gesture or the multi-page turn gesture, the deformation is based on the gesture and the metadata, and the second haptic effect comprises texture or a change in friction.

4. The method of claim 3, wherein the contact data comprises pressure data, and the change in friction is based on the pressure data.

5. The method of claim 3, further comprising generating and transmitting an audio signal based on the gesture and the metadata.

6. The method of claim 3, wherein the multi-page turn gesture comprises a scrub gesture, and transmitting the second haptic signal comprises transmitting the second haptic signal for each page of the electronic document accessed by the scrub gesture comprising the deformation.

7. The method of claim 3, wherein determining the gesture is further based on the metadata.

8. The method of claim 2, wherein the gesture comprises a folding gesture, the deformation comprises a folding of the portion of the document, and the second haptic effect comprises a texture or a change in friction.

9. The method of claim 8, wherein the folding deformation is configured to simulate the size and shape of the fold based on the gesture and the metadata.

10. The method of claim 2, wherein the gesture comprises a cut gesture, the deformation comprises a cutting of the portion of the document, and the second haptic effect comprises a vibration.

11. The method of claim 1, wherein the metadata further comprises at least one of a roughness characteristic, a color characteristic, weight characteristic, an age characteristic, a brittleness characteristic, or a compliance characteristic.

25. The method of claim 1, further comprising:

transmitting the metadata to, or downloading the metadata from, a remote device.

26. The method of claim 1, wherein the metadata comprises a visual characteristic associated with the electronic document, the visual characteristic comprising a blemish, a reflectiveness, or a color change parameter.

27. The method of claim 1, further comprising:

responsive to a passage of time, generating updated metadata configured to simulate virtual wear and tear of the electronic document over the period of time.

28. The method of claim 1, wherein:

receiving the metadata comprises receiving the metadata from a remote device; and

associating the metadata with the electronic document based on the second characteristic comprises associating the received metadata with the electronic document based on the type of paper, a filename of the electronic document, or information within the electronic document.

29. The method of claim 1, wherein the particular page is a predetermined page of the electronic document, and further comprising:

determining the particular page based at least in part on the metadata prior to transmitting the haptic signal.

12. A non-transitory computer-readable medium comprising program code configured to be executed by a processor, the program code configured to:

receive an electronic document comprising a plurality of pages;

receive metadata associated with the electronic document;

associate the metadata with the electronic document;

generate and transmit a display signal configured to cause display of the at least a portion of the electronic document;

in response to a multi-page turn gesture on a touch sensitive input device, perform a multi-page turn by causing multiple pages of the plurality of pages to turn in succession; and

generate and transmit a haptic signal in response to a particular page of the plurality of pages being passed during the multi-page turn, the haptic signal configured to cause a haptic output device to output a haptic effect.

13. The non-transitory computer-readable medium of claim 12, wherein the program code is further configured to:

receive contact data based on an interaction with the touch sensitive input device, the contact data associated with the portion of the document;

determine a gesture based on the contact data;

determine a deformation of the portion of the document based on the gesture;

generate and transmit a second display signal configured to cause display of the portion of the document comprising the deformation; and

generate and transmit a second haptic signal based on a type of paper associated with the electronic document and the deformation, the second haptic signal configured to cause the haptic output device to generate a second haptic effect, the second haptic effect configured to indicate the deformation.

14. The non-transitory computer-readable medium of claim 13, wherein the gesture comprises a page-turn gesture or the multi-page turn gesture, the deformation is based on the gesture and the metadata, and the second haptic effect comprises texture or a change in friction.

15. The non-transitory computer-readable medium of claim 14, wherein the contact data comprises pressure data, and the change in friction is based on the pressure data.

16. The non-transitory computer-readable medium of claim 14, wherein the program code is further configured to generate and transmit an audio signal based on the gesture and the metadata.

17. The non-transitory computer-readable medium of claim 14, wherein the multi-page turn gesture comprises a scrub gesture, and wherein the program code is further configured to:

transmit the second haptic signal for each page of the electronic document accessed by the scrub gesture comprising the deformation.

18. The non-transitory computer-readable medium of claim 14, wherein the program code is further configured to determine the gesture based at least in part on the metadata.

19. The non-transitory computer-readable medium of claim 13, wherein the gesture comprises a folding gesture, the deformation comprises a folding of the portion of the document, and the second haptic effect comprises a texture or a change in friction.

20. The non-transitory computer-readable medium of claim 19, wherein the folding deformation is configured to simulate the size and shape of the fold based on the gesture and the metadata.

21. The non-transitory computer-readable medium of claim 13, wherein the gesture comprises a cut gesture, the deformation comprises a cutting of the portion of the document, and the second haptic effect comprises a vibration.

22. The non-transitory computer-readable medium of claim 12, wherein the metadata comprises at least one of a roughness characteristic, a color characteristic, weight characteristic, an age characteristic, a brittleness characteristic, or a compliance characteristic associated with the electronic document.

23. A system comprising:

a computer-readable medium;

a touch-sensitive input device;

a display; and

a processor in communication with the computer-readable medium, the touch-sensitive input device, and the display, the processor configured to;

receive an electronic document comprising a plurality of pages;

generate and transmit a display signal configured to cause display of the at least a portion of the electronic document;

in response to a multi-page turn gesture on the touch-sensitive input device, perform a multi-page turn by causing multiple pages of the plurality of pages to turn in succession; and

generate and transmit a haptic signal in response to a particular page of the plurality of pages being passed during the multi-page turn, the haptic signal configured to cause a haptic output device to output a haptic effect.

24. The system of claim 23, wherein the processor is further configured to:

receive contact data based on an interaction with the touch-sensitive input device, the contact data associated with the portion of the document;

determine a gesture based on the contact data;

determine a deformation of the portion of the document based on the gesture;

generate and transmit a second display signal configured to cause display of the portion of the document comprising the deformation; and

generate and transmit a second haptic signal based on a type of paper associated with the electronic device and the deformation, the second haptic signal configured to cause the haptic output device to generate a second haptic effect, the second haptic effect configured to indicate the deformation.