Experience sharing for training

• US 8,914,472 B1
• Filed: 04/04/2012
• Issued: 12/16/2014
• Est. Priority Date: 07/20/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lee, Steven John, Mendis, Indika Charles, Braun, Max Benjamin
• Primary Examiner(s): Gee, Jason K.
• Assistant Examiner(s): LWIN, MAUNG T
• Application Number:
  US13/439,645
• Time in Patent Office: 986 Days
• Field of Search: None
• US Class Current: 709/219
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 30/04 Billing or invoicing
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0605 Supply or demand aggregation
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 Processing of requisition or of purchase orders
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/214 using selective forwarding
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06

Abstract

Some disclosed examples relate to facilitating an experience-sharing session in real-time between a first head-mountable display (HMD) and a second HMD. The experience-sharing session can include a demonstration phase and an observation phase. During the demonstration phase, a server system can receive first media content from the first HMD and send the first media content in real-time to the second HMD. The first media content can include a point-of-view video recorded at the first HMD. During the observation phase, the server system can receive second media content from the second HMD and send the second media content in real-time to the first HMD. The second media content can include a point-of-view video recorded at the second HMD.

Claims

1. A computer-implemented method comprising:

a server system facilitating an experience-sharing session in real-time between a first head-mountable display (HMD) and two or more second HMDs, wherein the experience-sharing session includes a demonstration phase and an observation phase;

during the demonstration phase, the server system receiving first media content from the first HMD and sending the first media content in real-time to at least one of the second HMDs, wherein the first media content includes first point-of-view video recorded at the first HMD, wherein the first point-of-view video comprises a first performance of one or more tasks;

determining a desired characteristic of the one or more tasks; and

during the observation phase;

receiving, by the server system, multiple point-of-view videos that were recorded at multiple corresponding HMDs, wherein receiving the multiple point-of-view videos comprises the server system receiving second media content from each of two or more of the second HMDs, wherein the second media content received from each second HMD includes second point-of-view video recorded at the second HMD, wherein the second point-of-view video comprises a second performance of the one or more tasks, and wherein the first performance is different from the second performance;

determining a ranking of the multiple point-of-view videos based on the desired characteristic; and

sending the second media content received from at least one second HMD to the first HMD in real-time.

2. The method of claim 1, further comprising, during the demonstration phase, the server system:

receiving information indicative of a perspective of a given one the second HMDs, wherein the information is generated based on one or more sensors of the given second HMD;

identifying a portion of the first media content that corresponds to the perspective; and

sending the portion in real-time to the given second HMD.

3. The method of claim 1, further comprising, during the observation phase, the server system:

receiving information based on data captured at one or more sensors a given one of the second HMDs; and

sending the information in real-time to the first HMD.

4. The method of claim 1, further comprising, during the observation phase, the server system:

receiving a comment corresponding to the second point-of-view video recorded at a given one of the second HMDs; and

sending the comment in real-time to the given second HMD.

5. The method of claim 1, further comprising the server system:

determining the desired characteristic of the one or more tasks based on the first performance of the one or more tasks; and

during the observation phase, comparing a corresponding characteristic of each second performance of the one or more tasks with the desired characteristic in real-time.

6. The method of claim 5, further comprising, during the observation phase, the server system:

generating an indicator in response to the comparison of the desired characteristic of the one or more tasks to the corresponding characteristic of a given one of the second performances of the one or more tasks; and

sending the indicator in real-time to at least one of the first HMD and the corresponding second HMD.

7. The method of claim 1, further comprising, during the observation phase, the server system:

processing the multiple point-of-view videos that were recorded at the multiple corresponding HMDs to identify a feature that is common to the multiple point-of-view videos; and

sending information indicative of the feature to the first HMD in real-time.

8. The method of claim 1, further comprising sending a predetermined number of highest-ranking videos to the first HMD.

9. The method of claim 1, further comprising sending a predetermined number of lowest-ranking videos to the first HMD.

10. The method of claim 1, wherein the first media content and the second media content correspond to a same activity.

11. The method of claim 1, wherein the demonstration phase overlaps the observation phase.

12. The method of claim 1, wherein the demonstration phase and the observation phase occur simultaneously.

13. A system comprising:

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to;

facilitate an experience-sharing session in real-time between a first head-mountable display (HMD) and two or more second HMDs, wherein the experience-sharing session includes a demonstration phase and an observation phase;

during the demonstration phase, receive first media content from the first HMD and send the first media content in real-time to at least one of the second HMDs, wherein the first media content includes first point-of-view video recorded at the first HMD, wherein the first point-of-view video comprises a first performance of one or more tasks;

determine a desired characteristic of the one or more tasks; and

during the observation phase;

receive multiple point-of-view videos that were recorded at multiple corresponding HMDs, wherein receiving the multiple point-of-view videos comprises receipt of second media content from each of two or more of the second HMDs, wherein the second media content received from a given second HMD includes second point-of-view video recorded at the second HMD, wherein the second point-of-view video comprises a second performance of the one or more tasks, and wherein the first performance is different from the second performance;

determine a ranking of the multiple point-of-view videos based on the desired characteristic; and

send the second media content received from at least one second HMD to the first HMD in real-time.

14. A computer-implemented method comprising:

a first wearable device initiating in an experience-sharing session in real-time with two or more second wearable devices, wherein the experience-sharing session includes a first phase and a second phase;

during the first phase, the first wearable device generating first media content and sending the first media content in real-time to two or more of the second wearable devices, wherein the first media content includes first point-of-view video recorded at the first wearable device, wherein the first point-of-view video comprises a first performance of one or more tasks;

determining a desired characteristic of the one or more tasks; and

during the second phase;

receiving, by the first wearable device, multiple point-of-view videos that were recorded at multiple second wearable devices, wherein receiving the multiple point-of-view videos comprises the first wearable device receiving second media content from each of two or more of the second wearable devices, wherein the second media content received from each of the two or more of the second wearable devices includes second point-of-view video recorded at the second wearable device, wherein the second point-of-view video comprises a second performance of the one or more tasks, and wherein the first performance is different from the second performance;

determining a ranking of the multiple point-of-view videos based on the desired characteristic; and

displaying, in real-time, the second media content received from at least one second wearable device to the first wearable device.

15. The method of claim 14, wherein sending the first media content in real-time to the second wearable device comprises sending the first media content directly to the second wearable device.

16. The method of claim 14, wherein sending the first media content in real-time to the second wearable device comprises sending the first media content to the second wearable device via a server system.

17. A wearable-device system comprising:

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to;

cause a first wearable device to initiate in an experience-sharing session with two or more second wearable devices, wherein the experience-sharing session includes a first phase and a second phase;

during the first phase, generate first media content and send the first media content in real-time to two or more of the second wearable devices, wherein the first media content includes first point-of-view video recorded at the first wearable device, wherein the first point-of-view video comprises a first performance of one or more tasks;

determine a desired characteristic of the one or more tasks; and

during the second phase;

receive multiple point-of-view videos that were recorded at multiple second wearable devices, wherein receipt of the multiple point-of-view videos comprises receipt of second media content from each of two or more of the second wearable devices, wherein the second media content received from each of the two or more of the second wearable devices includes second point-of-view video recorded at the second wearable device, wherein the second point-of-view video comprises a second performance of the one or more tasks, and wherein the first performance is different from the second performance;

determine a ranking of the multiple point-of-view videos based on the desired characteristic; and

display, in real-time, the second media content received from at least one second wearable device to the first wearable device.

18. The wearable-device system of claim 17, wherein at least one of the first wearable device and the second wearable device includes a head-mountable display.

19. The wearable-device system of claim 17, wherein the wearable-device system is disposed at the first wearable device or the second wearable device.