Using haptic technologies to provide enhanced media experiences

• US 10,013,857 B2
• Filed: 08/24/2012
• Issued: 07/03/2018
• Est. Priority Date: 12/21/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Sridhara, Vinay, Das, Saumitra Mohan, Sheynblat, Leonid
• Primary Examiner(s): Theriault, Steven B
• Application Number:
  US13/594,502
• Publication Number:
  US 20130227410A1
• Time in Patent Office: 2,139 Days
• Field of Search: 715702
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

Methods, apparatuses, systems, and computer-readable media for using haptic technologies to provide enhanced media experiences are presented. According to one or more aspects of the disclosure, a computing device, such as a smart phone, tablet computer, or portable media player, may establish a connection with a local content receiver. Subsequently, the computing device may receive, from the local content receiver, a sensation data signal that specifies one or more haptic effects to be provided at a particular time relative to playback of a media content item received by the local content receiver. Thereafter, the computing device may provide the one or more haptic effects.

Claims

1. A method comprising:

establishing, by a computing device, a connection with a local content receiver providing a content stream for display;

receiving, by the computing device, from the local content receiver a sensation data signal associated with the content stream being provided for display, the sensation data signal including sensation data information that specifies one or more haptic effects to be provided while the content stream is displayed, wherein the sensation data information includes one or more markers within the content stream associating the one or more haptic effects with particular points of the content stream provided by the local content receiver;

detecting, by the computing device, that the content stream is being provided for display;

detecting, by the computing device, a particular portion of the content stream being provided for display while the content stream is being provided for display by the local content receiver, the particular portion being associated with the one or more haptic effects according to the sensation data information; and

providing, by the computing device, the one or more haptic effects in synchronization with the particular portion of the content stream.

2. The method of claim 1, wherein establishing the connection with the local content receiver further includes:

detecting the local content receiver on a wireless area network; and

sending registration information to the local content receiver.

3. The method of claim 2, wherein the registration information includes device capability information.

4. The method of claim 1, further comprising:

notifying, by the computing device, the local content receiver of haptic feedback capabilities of the computing device; and

wherein detecting that the content stream is being provided for display occurs prior to receiving the sensation data signal and includes capturing audio data or video data corresponding to a playback of the content stream.

5. The method of claim 1, further comprising:

prior to providing the one or more haptic effects, determining whether activity is detected.

6. The method of claim 5, wherein determining whether activity is detected includes determining whether the computing device is in motion or at rest.

7. The method of claim 5, wherein determining whether activity is detected includes determining whether the computing device is being held by a user.

8. The method of claim 1, further comprising:

detecting a new content stream associated with a second media content item;

and

providing one or more additional haptic effects based on data received from the local content receiver.

9. The method of claim 8, further comprising:

prior to providing the one or more additional haptic effects, waiting for a predetermined amount of time.

10. The method of claim 1, wherein the one or more haptic effects have a logical relationship to associated images or associated audio of the content stream and are synchronized to occur during a display of the associated images of the content stream or a projection of the associated audio of the content stream.

11. At least one non-transitory computer-readable medium having computer-executable instructions stored thereon that, when executed, cause a computing device to:

establish, by a computing device, a connection with a local content receiver providing a content stream for display;

receive, by the computing device, from the local content receiver a sensation data signal associated with the content stream being provided for display, the sensation data signal including sensation data information that specifies one or more haptic effects to be provided while the content stream is displayed, wherein the sensation data information includes one or more markers within the content stream associating the one or more haptic effects with particular points of the content stream provided by the local content receiver;

detect, by the computing device, that the content stream is being provided for display;

detect, by the computing device, a particular portion of the content stream being provided for display while the content stream is being provided for display by the local content receiver, the particular portion being associated with the one or more haptic effects according to the sensation data information; and

provide, by the computing device, the one or more haptic effects in synchronization with the particular portion of the content stream.

12. The at least one non-transitory computer-readable medium of claim 11, wherein establishing the connection with the local content receiver further includes:

detecting the local content receiver on a wireless area network; and

sending registration information to the local content receiver.

13. The at least one non-transitory computer-readable medium of claim 12, wherein the registration information includes device capability information.

14. The at least one non-transitory computer-readable medium of claim 11, further comprising:

notify, by the computing device, the local content receiver of haptic feedback capabilities of the computing device; and

wherein detecting that the content stream is being provided for display occurs prior to receiving the sensation data signal and includes capturing audio data or video data corresponding to a playback of the content stream.

15. The at least one non-transitory computer-readable medium of claim 11, having additional computer-executable instructions stored thereon that, when executed, further cause the computing device to:

prior to providing the one or more haptic effects, determine whether activity is detected.

16. The at least one non-transitory computer-readable medium of claim 15, wherein determining whether activity is detected includes determining whether the computing device is in motion or at rest.

17. The at least one non-transitory computer-readable medium of claim 15, wherein determining whether activity is detected includes determining whether the computing device is being held by a user.

18. The at least one non-transitory computer-readable medium of claim 11, having additional computer-executable instructions stored thereon that, when executed, further cause the computing device to:

detect a new content stream associated with a second media content item;

and

provide one or more additional haptic effects based on data received from the local content receiver.

19. The at least one non-transitory computer-readable medium of claim 18, having additional computer-executable instructions stored thereon that, when executed, further cause the computing device to:

prior to providing the one or more additional haptic effects, wait for a predetermined amount of time.

20. The at least one non-transitory computer-readable medium of claim 11, wherein the one or more haptic effects have a logical relationship to associated images or associated audio of the content stream and are synchronized to occur during a display of the associated images of the content stream or a projection of the associated audio of the content stream.

21. An apparatus comprising:

at least one processor; and

memory storing computer-readable instructions that, when executed by the at least one processor, cause the apparatus to;

establish, by a computing device, a connection with a local content receiver providing a content stream for display;

receive, by the computing device, from the local content receiver a sensation data signal associated with the content stream being provided for display, the sensation data signal including sensation data information that specifies one or more haptic effects to be provided while the content stream is displayed, wherein the sensation data information includes one or more markers within the content stream associating the one or more haptic effects with particular points of the content stream provided by the local content receiver;

detect, by the computing device, that the content stream is being provided for display;

detect, by the computing device, a particular portion of the content stream being provided for display while the content stream is being provided for display by the local content receiver, the particular portion being associated with the one or more haptic effects according to the sensation data information; and

provide, by the computing device, the one or more haptic effects in synchronization with the particular portion of the content stream.

22. The apparatus of claim 21, wherein establishing the connection with the local content receiver further includes:

detecting the local content receiver on a wireless area network; and

sending registration information to the local content receiver.

23. The apparatus of claim 22, wherein the registration information includes device capability information.

24. The apparatus of claim 21, further comprising:

notify, by the computing device, the local content receiver of haptic feedback capabilities of the computing device; and

wherein detecting that the content stream is being provided for display occurs prior to receiving the sensation data signal and includes capturing audio data or video data corresponding to a playback of the content stream.

25. The apparatus of claim 21, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, further cause the apparatus to:

prior to providing the one or more haptic effects, determine whether activity is detected.

26. The apparatus of claim 25, wherein determining whether activity is detected includes determining whether the apparatus is in motion or at rest.

27. The apparatus of claim 25, wherein determining whether activity is detected includes determining whether the apparatus is being held by a user.

28. The apparatus of claim 21, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, further cause the apparatus to:

detect a new content stream associated with a second media content item;

and

provide one or more additional haptic effects based on data received from the local content receiver.

29. The apparatus of claim 28, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, further cause the apparatus to:

prior to providing the one or more additional haptic effects, wait for a predetermined amount of time.

30. The apparatus of claim 21, wherein the one or more haptic effects have a logical relationship to associated images or associated audio of the content stream and are synchronized to occur during a display of the associated images of the content stream or a projection of the associated audio of the content stream.

31. A system comprising:

means for establishing, by a computing device, a connection with a local content receiver providing a content stream for display;

means for receiving, by the computing device, from the local content receiver a sensation data signal associated with the content stream being provided for display, the sensation data signal including sensation data information that specifies one or more haptic effects to be provided while the content stream is displayed, wherein the sensation data information includes one or more markers within the content stream associating the one or more haptic effects with particular points of the content stream provided by the local content receiver;

means for detecting, by the computing device, that the content stream is being provided for display;

means for detecting, by the computing device, a particular portion of the content stream being provided for display while the content stream is being provided for display by the local content receiver, the particular portion being associated with the one or more haptic effects according to the sensation data information; and

means for providing, by the computing device, the one or more haptic effects in synchronization with the particular portion of the content stream.

32. The system of claim 31, wherein the means for establishing the connection with the local content receiver further includes:

means for detecting the local content receiver on a wireless area network; and

means for sending registration information to the local content receiver.

33. The system of claim 32, wherein the registration information includes device capability information.

34. The system of claim 31, further comprising:

means for notifying, by the computing device, the local content receiver of haptic feedback capabilities of the computing device;

wherein the means for detecting that the content stream is being provided for display occurs prior to receiving the sensation data signal and includes capturing audio data or video data corresponding to a playback of the content stream.

35. The system of claim 31, further comprising:

means for determining whether activity is detected prior to providing the one or more haptic effects.

36. The system of claim 35, wherein the means for determining whether activity is detected includes means for determining whether the system is in motion or at rest.

37. The system of claim 35, wherein the means for determining whether activity is detected includes means for determining whether the system is being held by a user.

38. The system of claim 31, further comprising:

means for detecting a new content stream associated with a second media content item;

and

means for providing one or more additional haptic effects based on data received from the local content receiver.

39. The system of claim 38, further comprising:

means for waiting for a predetermined amount of time prior to providing the one or more additional haptic effects.

40. The system of claim 31, wherein the one or more haptic effects have a logical relationship to associated images or associated audio of the content stream and are synchronized to occur during a display of the associated images of the content stream or a projection of the associated audio of the content stream.