User-experience customization

• US 8,994,613 B1
• Filed: 01/06/2012
• Issued: 03/31/2015
• Est. Priority Date: 01/06/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Weaver, Joshua, Starner, Thad, Johnson, Michael
• Primary Examiner(s): FARAGALLA, MICHAEL A
• Application Number:
  US13/345,493
• Time in Patent Office: 1,180 Days
• Field of Search: 345/8, 345/173, 345/178
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09G 5/37 Details of the operation on graphic patterns G09G5/38 takes precedence

Abstract

Methods and systems for customizing user-experience of media content and providing content of interest to a user are provided. A computing device, such as a wearable user device having a head-mounted display (HMD) may receive media content and physiological data associated with a user experiencing the media content. The received physiological data may include one of a pupillary response, a galvanic skin response, an electrocardiographic response, an electroencephalographic response, a body temperature response, or a blood pressure response. The computing device may determine a portion of the media content which corresponds to the physiological data, and accordingly search for content of interest to the user. The content of interest may then be provided to the user, thereby customizing the user-experience of the user. The content of interest to the user may be displayed on a display of the wearable user device.

Claims

1. A method comprising:

capturing, with a camera coupled to a computing device, first media content that is indicative of a user-experience associated with the computing device, wherein the user-experience comprises second media content from an external media device;

receiving physiological data indicating a physiological state, wherein the physiological data is received from one or more physiological sensors;

in response to receiving the physiological data indicating the physiological state, analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state;

using the portion of the second media content that corresponds to the physiological state as a basis for initiating a search for content of interest; and

displaying the content of interest at the computing device.

2. The method of claim 1, wherein the computing device comprises a wearable computing device having a head-mounted display (HMD).

3. The method of claim 2, wherein the HMD comprises a substantially transparent display.

4. The method of claim 1, wherein analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state comprises analyzing one or more associations between previously received physiological data and previously received media content.

5. The method of claim 1, wherein analyzing the first media content to determining a portion of the second media content that corresponds to the physiological state comprises synchronizing the first media content and the physiological data.

6. The method of claim 1, further comprising:

further analyzing the first media content to detect a portion of the second media content during which the content of interest should be displayed; and

displaying the content of interest at the computing device in response to detecting the portion of the second media content during which the content of interest should be displayed.

7. The method of claim 6, wherein the second media content comprises a television program, and wherein the portion of the second media content during which the content of interest should be displayed comprises a next commercial break during the television program.

8. The method of claim 1, further comprising:

determining that the portion of the second media content that corresponds to the physiological state comprises a subject of interest; and

wherein initiating the search for content of interest comprises initiating a search for content related to the subject of interest.

9. The method of claim 8, wherein the computing device comprises a wearable computing device having a head-mounted display (HMD), the method further comprising:

using eye-tracking data associated with the HMD as a basis for determining a region of interest within the portion of the second media content that corresponds to the physiological state; and

identifying the subject of interest by determining that the subject of interest is located at or near the region of interest.

10. The method of claim 8, wherein the computing device comprises a wearable computing device having a head-mounted display (HMD), wherein the HMD comprises a substantially transparent display, the method further comprising:

determining a location of the subject of interest within the second media content; and

based on the location of the subject of interest, determining a display location in the substantially transparent display, such that displaying the content of interest at the display location augments a view of the subject of interest through the substantially transparent display.

11. The method of claim 1, wherein the physiological data is indicative of one or more of the following:

(a) galvanic skin response, (b) pupillary response, (c) electrocardiographic response, (d) electroencephalographic response, (e) body temperature, (f) blood pressure, and (g) hemoglobin oxygenation response.

12. A method comprising:

receiving, at a server system, first media content that is indicative of a user-experience associated with a computing device, wherein the user-experience comprises second media content from an external media device and wherein the first media content was captured by a camera coupled to a head-mounted computing device;

receiving, at the server system, physiological data associated with the computing device, wherein, wherein the physiological data is indicative a physiological state;

in response to receiving the physiological data indicating the physiological state, analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state;

using the portion of the second media content that corresponds to the physiological state as a basis for initiating a search for content of interest; and

sending the content of interest to the computing device.

13. The method of claim 12, wherein analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state comprises analyzing one or more associations between previously received physiological data and previously received media content.

14. The method of claim 12, wherein analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state comprises synchronizing the first media content and the physiological data.

15. The method of claim 12, further comprising:

determining that the portion of the second media content that corresponds to the physiological state comprises a subject of interest; and

wherein initiating the search for content of interest comprises initiating a search for content related to the subject of interest.

16. An article of manufacture including a tangible computer-readable media having computer-readable instructions encoded thereon, the instructions comprising instructions for performing the methods of:

capturing, with a camera coupled to a computing device, first media content that is indicative of a user-experience associated with the computing device, wherein the user-experience comprises second media content from an external media device;

receiving physiological data indicating a physiological state, wherein the physiological data is received from one or more physiological sensors;

in response to receiving the physiological data indicating the physiological state, analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state;

using the portion of the second media content that corresponds to the physiological state as a basis for initiating a search for content of interest; and

displaying the content of interest at the computing device.

17. An article of manufacture including a tangible computer-readable media having computer-readable instructions encoded thereon, the instructions comprising instructions for performing the methods of:

receiving, at a server system, first media content that is indicative of a user-experience associated with a computing device, wherein the user-experience comprises second media content from an external media device and wherein the first media content was captured by a camera coupled to a head-mounted computing device;

receiving, at the server system, physiological data associated with the computing device, wherein, wherein the physiological data is indicative of a physiological state;

in response to receiving the physiological data indicating the physiological state, analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state;

using the portion of the second media content that corresponds to the physiological state as a basis for initiating a search for content of interest; and

sending the content of interest to the computing device.

18. A device comprising:

a processor; and

memory including instructions stored therein executable by the processor to perform functions comprising;

capturing, with a camera coupled to a computing device, first media content that is indicative of a user-experience associated with the computing device, wherein the user-experience comprises second media content from an external media device;

receiving physiological data indicating a physiological state, wherein the physiological data is received from one or more physiological sensors;

in response to receiving the physiological data indicating the physiological state, analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state;

using the portion of the second media content that corresponds to the physiological state as a basis for initiating a search for content of interest; and

displaying the content of interest at the computing device.

19. A device comprising:

a processor; and

memory including instructions stored therein executable by the processor to perform functions comprising;

receiving, at a server system, first media content that is indicative of a user-experience associated with a computing device, wherein the user-experience comprises second media content from an external media device and wherein the first media content was captured by a camera coupled to a head-mounted computing device;

receiving, at the server system, physiological data associated with the computing device, wherein, wherein the physiological data is indicative of a physiological state;

in response to receiving the physiological data indicating the physiological state, analyzing the first media content to determine a portion of the second media content that corresponds to the physiological state;

using the portion of the second media content that corresponds to the physiological state as a basis for initiating a search for content of interest; and

sending the content of interest to the computing device.