Group video session

• US 10,148,910 B2
• Filed: 12/30/2016
• Issued: 12/04/2018
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Mercredi, Dwayne B., Mravec, Pavol, Ho, Chi Wang, Li, Naizhi
• Primary Examiner(s): Woo, Stella L.
• Application Number:
  US15/395,172
• Publication Number:
  US 20180192000A1
• Time in Patent Office: 704 Days
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 50/00 Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism healthcare informatics G16H
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04L 65/1089 by adding media; by removing media
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831

Abstract

In one embodiment, a computer server machine receives, from one or more client devices, one or more video streams associated with a video session associated with one or more users. The one or more users are respectively associated with each client device. The computer server machine analyzes the one or more video streams to identify an interesting event and generates a marker for the one or more video streams based on a time associated with the identified interesting event. The computer server machine sends information regarding the marker to the one or more client devices.

Claims

1. A method comprising:

by a computer server machine, receiving, from one or more client devices with each client device respectively associated with a user, one or more video streams associated with a video session, wherein the one or more video streams are associated with the one or more users, and wherein the one or more users are communicating with one another within the video session;

by the computer server machine, analyzing the one or more video streams to identify an interesting event candidate;

by the computer server machine, generating a marker for the one or more of the video streams based on a time associated with the identified interesting event candidate;

by the computing server machine, determining, for each of the one or more users, whether the identified interesting event candidate is an interesting event for the user based at least on information related to a user profile associated with the user; and

by the computer server machine, sending, for each of the one or more users for whom the identified interesting event candidate is an interesting event, information regarding the marker to the client device associated with the user.

2. The method of claim 1, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises:

by the computer server machine, recognizing an expression in one or more of the video streams; and

by the computer server machine, identifying a change in the expression in one or more of the video streams.

3. The method of claim 2, further comprising:

by the computer server machine, determining an emotion with the change in the expression; and

by the computer server machine, associating information regarding the emotion with the information regarding the marker.

4. The method of claim 2, further comprising providing, by the computer server machine, an overlay to one or more of the video streams, wherein the overlay comprises an audio or visual indication corresponding to the change in the expression.

5. The method of claim 1, wherein the each of the one or more video streams comprises a video component and an audio component.

6. The method of claim 1, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in the audio content captured in one or more of the video streams by the computer server machine.

7. The method of claim 1, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in a scene captured in one or more of the video streams by the computer server machine.

8. The method of claim 1, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in quality in one or more of the video streams by the computer server machine.

9. The method of claim 1, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in an attribute for one or more of the video streams by the computer server machine.

10. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, from one or more client devices with each client device respectively associated with a user, one or more video streams associated with a video session, wherein the one or more video streams are associated with the one or more users, and wherein the one or more users are communicating with one another within the video session;

analyze the one or more video streams to identify an interesting event candidate;

generate a marker for the one or more of the video streams based on a time associated with the identified interesting event candidate;

determine, for each of the one or more users, whether the identified interesting event candidate is an interesting event for the user based at least on information related to a user profile associated with the user; and

send, for each of the one or more users for whom the identified interesting event candidate is an interesting event, information regarding the marker to the client device associated with the user.

11. The media of claim 10, wherein the software is further operable when executed to:

recognize an expression in one or more of the video streams; and

identify a change in the expression in one or more of the video streams.

12. The media of claim 11, wherein the software is further operable when executed to:

determine an emotion with the change in the expression; and

associate information regarding the emotion with the information regarding the marker.

13. The media of claim 11, wherein the software is further operable when executed to provide an overlay to one or more of the video streams, wherein the overlay comprises an audio or visual indication corresponding to the change in the expression.

14. The media of claim 10, wherein the each of the one or more video streams comprises a video component and an audio component.

15. The media of claim 10, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in the audio content captured in one or more of the video streams.

16. The media of claim 10, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in a scene captured in one or more of the video streams.

17. The media of claim 10, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in quality in one or more of the video streams.

18. The media of claim 10, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in an attribute for one or more of the video streams.

19. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

receive, from one or more client devices with each client device respectively associated with a user, one or more video streams associated with a video session, wherein the one or more video streams are associated with the one or more users, and wherein the one or more users are communicating with one another within the video session;

analyze the one or more video streams to identify an interesting event candidate;

generate a marker for the one or more of the video streams based on a time associated with the identified interesting event candidate;

determine, for each of the one or more users, whether the identified interesting event candidate is an interesting event for the user based at least on information related to a user profile associated with the user; and

send, for each of the one or more users for whom the identified interesting event candidate is an interesting event, information regarding the marker to the client device associated with the user.

20. The system of claim 19, wherein the processors are further operable when executing the instructions to:

recognize an expression in one or more of the video streams; and

identify a change in the expression in one or more of the video streams.

21. The system of claim 20, wherein the processors are further operable when executing the instructions to:

determine an emotion with the change in the expression; and

associate information regarding the emotion with the information regarding the marker.

22. The system of claim 20, wherein the software is further operable when executed to provide an overlay to one or more of the video streams, wherein the overlay comprises an audio or visual indication corresponding to the change in the expression.

23. The system of claim 19, wherein the each of the one or more video streams comprises a video component and an audio component.

24. The system of claim 19, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in the audio content captured in one or more of the video streams.

25. The system of claim 19, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in a scene captured in one or more of the video streams.

26. The system of claim 19, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in quality in one or more of the video streams.

27. The system of claim 19, wherein analyzing the one or more video streams to identify an interesting event candidate further comprises detecting a change in an attribute for one or more of the video streams.