User interface for multi-user communication session

• US 10,866,703 B2
• Filed: 02/13/2020
• Issued: 12/15/2020
• Est. Priority Date: 09/29/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Anzures, Freddy Allen, King, Nicholas V., Lemay, Stephen O., Pham, Hoan, Yerkes, Giancarlo
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US16/790,619
• Publication Number:
  US 20200183548A1
• Time in Patent Office: 306 Days
• Field of Search: 348 1401- 1416
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04M 1/27457 Management thereof, e.g. manual editing of data
• CPC Class Codes: H04M 1/27475 using interactive graphical means or pictorial representations
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72433 for voice messaging, e.g. dictaphones for answering incoming calls H04M1/64
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04M 2250/62 user interface aspects of conference calls
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

The present disclosure generally relates to user interfaces for multi-user communication sessions. In some examples, a device initiates a live stream in a communication session. In some examples, a device transitions between streaming live audio and live video. In some examples, a device enables synchronizing media playback during a live stream.

Claims

1. An electronic device, comprising:

a display;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

while transmitting an outgoing live media stream of a first type to one or more participants of a communication session;

outputting, at the electronic device, at least one or more incoming live media streams;

receiving user input to transition from transmitting the outgoing live media stream of the first type to transmitting an outgoing live media stream of a second type, wherein the first type is different from the second type; and

in response to receiving the user input to transition, transitioning from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type, including;

in accordance with the first type of live media stream including live audio and not including live video, transmitting live video; and

in accordance with the first type of live media stream including live audio and live video;

 

ceasing to transmit live video; and

 

replacing, on the display, display of live video captured by one or more cameras of the electronic device with a predefined static image while continuing to output live audio.

2. The electronic device of claim 1, wherein receiving the user input to transition includes:

detecting a tap user input on an affordance, wherein the affordance is displayed as part of an avatar of a user associated with the electronic device.

3. The electronic device of claim 1, wherein the one or more programs further include instructions for:

subsequent to transitioning from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type;

receiving user input to transition from transmitting the outgoing live media stream of the second type to transmitting outgoing live media stream of the first type; and

in response to receiving the user input to transition, transitioning from transmitting the outgoing live media stream of the second type to transmitting the outgoing live media stream of the first type, including;

in accordance with the second type of live media stream including live audio and not including live video, transmitting live video; and

in accordance with the second type of live media stream including live audio and live video, ceasing to transmit live video.

4. The electronic device of claim 1, wherein:

the electronic device includes one or more camera sensors and one or more microphones; and

transmitting the outgoing live media stream of the second type includes;

transmitting images detected using the one or more camera sensors; and

transmitting audio detected using the one or more microphones.

5. The electronic device of claim 1, wherein the one or more programs further include instructions for:

while transmitting live video, displaying, on the display, live video.

6. The electronic device of claim 1, wherein:

the electronic device includes a first camera and a second camera; and

the one or more programs further include instructions for;

while detecting, using the first camera and without using the second camera, first images and while transmitting the first images to one or more participants of the communication session;

displaying an affordance for switching cameras;

detecting activation of the affordance for switching cameras; and

in response to detecting activation of the affordance for switching cameras;

detecting, using the second camera and without using the first camera, second images; and

transmitting the second images to one or more participants of the communication session.

7. The electronic device of claim 1, wherein the predefined static image is

a static avatar image.

8. The electronic device of claim 1, wherein

replacing, on the display, display of live video captured by one or more cameras of the electronic device with the predefined static image includes displaying a text display region.

9. The electronic device of claim 1, wherein the one or more programs further include instructions for:

in response to receiving the user input to transition and further in accordance with the first type of live media stream including live audio and live video;

transitioning display of one or more avatars of one or more participants of the communication session by concurrently;

reducing sizes of the displayed one or more avatars of the one or more participants;

changing shapes of the displayed one or more avatars of the one or more participants; and

changing locations of the displayed one or more avatars of the one or more participants.

10. The electronic device of claim 1, wherein the one or more programs further include instructions for:

detecting user input requesting to end transmitting an outgoing live media stream; and

in response to detecting user input request to end transmitting the outgoing live media stream;

ceasing to transmit the outgoing live media stream; and

ceasing to receive live media streams from one or more participants of the communication session.

11. The electronic device of claim 1, wherein the one or more programs further include instructions for:

before receiving user input to transition from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type, concurrently displaying;

a representation of the outgoing live media stream of the first type that is being transmitted to one or more participants of the communication session at a first location on the display; and

a plurality representations of one or more incoming live media streams from the one or more participants of the communication session at one or more respective locations on the display, wherein;

in response to receiving the user input to transition and in accordance with the first type of live media stream including live audio and live video;

replacing, on the display, display of live video captured by one or more cameras of the electronic device with the predefined static image while continuing to output live audio includes;

 

ceasing to display, at the first location, the first representation of the outgoing live media stream and displaying, at the first location, a second representation of the outgoing live media stream, wherein the first representation of the outgoing live media stream is a representation of live video and the second representation of the outgoing live media stream is the predefined static image; and

 

continuing to display the plurality of representations of the one or more incoming live media streams from the one or more participants of the communication session at the one or more respective locations on the display while displaying the second representation of the outgoing live media stream at the first location on the display.

12. The electronic device of claim 1, wherein the predefined static image identifies a user of the electronic device.

13. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display, the one or more programs including instructions for:

while transmitting an outgoing live media stream of a first type to one or more participants of a communication session;

outputting, at the electronic device, at least one or more incoming live media streams;

receiving user input to transition from transmitting the outgoing live media stream of the first type to transmitting an outgoing live media stream of a second type, wherein the first type is different from the second type; and

in response to receiving the user input to transition, transitioning from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type, including;

in accordance with the first type of live media stream including live audio and not including live video, transmitting live video; and

in accordance with the first type of live media stream including live audio and live video;

ceasing to transmit live video; and

replacing, on the display, display of live video captured by one or more cameras of the electronic device with a predefined static image while continuing to output live audio.

14. The non-transitory computer-readable storage medium of claim 13, wherein receiving the user input to transition includes:

detecting a tap user input on an affordance, wherein the affordance is displayed as part of an avatar of a user associated with the electronic device.

15. The non-transitory computer-readable storage medium of claim 13, wherein the one or more programs further include instructions for:

subsequent to transitioning from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type;

receiving user input to transition from transmitting the outgoing live media stream of the second type to transmitting outgoing live media stream of the first type; and

in response to receiving the user input to transition, transitioning from transmitting the outgoing live media stream of the second type to transmitting the outgoing live media stream of the first type, including;

in accordance with the second type of live media stream including live audio and not including live video, transmitting live video; and

in accordance with the second type of live media stream including live audio and live video, ceasing to transmit live video.

16. The non-transitory computer-readable storage medium of claim 13, wherein:

the electronic device includes one or more camera sensors and one or more microphones; and

transmitting the outgoing live media stream of the second type includes;

transmitting images detected using the one or more camera sensors; and

transmitting audio detected using the one or more microphones.

17. The non-transitory computer-readable storage medium of claim 13, wherein the one or more programs further include instructions for:

while transmitting live video, displaying, on the display, live video.

18. The non-transitory computer-readable storage medium of claim 13, wherein:

the electronic device includes a first camera and a second camera; and

the one or more programs further include instructions for;

while detecting, using the first camera and without using the second camera, first images and while transmitting the first images to one or more participants of the communication session;

displaying an affordance for switching cameras;

detecting activation of the affordance for switching cameras; and

in response to detecting activation of the affordance for switching cameras;

detecting, using the second camera and without using the first camera, second images; and

transmitting the second images to one or more participants of the communication session.

19. The non-transitory computer-readable storage medium of claim 13, wherein the predefined static image is a static avatar image.

20. The non-transitory computer-readable storage medium of claim 13, wherein replacing, on the display, display of live video captured by one or more cameras of the electronic device with the predefined static image includes displaying a text display region.

21. The non-transitory computer-readable storage medium of claim 13, wherein the one or more programs further include instructions for:

in response to receiving the user input to transition and further in accordance with the first type of live media stream including live audio and live video;

transitioning display of one or more avatars of one or more participants of the communication session by concurrently;

reducing sizes of the displayed one or more avatars of the one or more participants;

changing shapes of the displayed one or more avatars of the one or more participants; and

changing locations of the displayed one or more avatars of the one or more participants.

22. The non-transitory computer-readable storage medium of claim 13, wherein the one or more programs further include instructions for:

detecting user input requesting to end transmitting an outgoing live media stream; and

in response to detecting user input request to end transmitting the outgoing live media stream;

ceasing to transmit the outgoing live media stream; and

ceasing to receive live media streams from one or more participants of the communication session.

23. The non-transitory computer-readable storage medium of claim 13, wherein the one or more programs further include instructions for:

before receiving user input to transition from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type, concurrently displaying;

a representation of the outgoing live media stream of the first type that is being transmitted to one or more participants of the communication session at a first location on the display; and

one or more representations of one or more incoming live media streams from the one or more participants of the communication session at one or more respective locations on the display, wherein;

in response to receiving the user input to transition and in accordance with the first type of live media stream including live audio and live video;

replacing, on the display, display of live video captured by one or more cameras of the electronic device with the predefined static image while continuing to output live audio includes;

 

ceasing to display, at the first location, the first representation of the outgoing live media stream and displaying, at the first location, a second representation of the outgoing live media stream, wherein the first representation of the outgoing live media stream is a representation of live video and the second representation of the outgoing live media stream is the predefined static image; and

 

continuing to display the one or more representations of the one or more incoming live media streams from the one or more participants of the communication session at the one or more respective locations on the display while displaying the second representation of the outgoing live media stream at the first location on the display.

24. The non-transitory computer-readable storage medium of claim 13, wherein the predefined static image identifies a user of the electronic device.

25. A method, comprising:

at an electronic device with a display;

while transmitting an outgoing live media stream of a first type to one or more participants of a communication session;

outputting, at the electronic device, at least one or more incoming live media streams;

receiving user input to transition from transmitting the outgoing live media stream of the first type to transmitting an outgoing live media stream of a second type, wherein the first type is different from the second type; and

in response to receiving the user input to transition, transitioning from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type, including;

in accordance with the first type of live media stream including live audio and not including live video, transmitting live video; and

in accordance with the first type of live media stream including live audio and live video;

 

ceasing to transmit live video; and

 

replacing, on the display, display of live video captured by one or more cameras of the electronic device with a predefined static image while continuing to output live audio.

26. The method of claim 25, wherein receiving the user input to transition includes:

detecting a tap user input on an affordance, wherein the affordance is displayed as part of an avatar of a user associated with the electronic device.

27. The method of claim 25, further comprising:

subsequent to transitioning from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type;

receiving user input to transition from transmitting the outgoing live media stream of the second type to transmitting outgoing live media stream of the first type; and

in response to receiving the user input to transition, transitioning from transmitting the outgoing live media stream of the second type to transmitting the outgoing live media stream of the first type, including;

in accordance with the second type of live media stream including live audio and not including live video, transmitting live video; and

in accordance with the second type of live media stream including live audio and live video, ceasing to transmit live video.

28. The method of claim 25, wherein:

the electronic device includes one or more camera sensors and one or more microphones; and

transmitting the outgoing live media stream of the second type includes;

transmitting images detected using the one or more camera sensors; and

transmitting audio detected using the one or more microphones.

29. The method of claim 25, further comprising:

while transmitting live video, displaying, on the display, live video.

30. The method of claim 25, wherein:

the electronic device includes a first camera and a second camera; and

the method further comprising;

while detecting, using the first camera and without using the second camera, first images and while transmitting the first images to one or more participants of the communication session;

displaying an affordance for switching cameras;

detecting activation of the affordance for switching cameras; and

in response to detecting activation of the affordance for switching cameras;

detecting, using the second camera and without using the first camera, second images; and

transmitting the second images to one or more participants of the communication session.

31. The method of claim 25, wherein the predefined static image is a static avatar image.

32. The method of claim 25, wherein replacing, on the display, display of live video captured by one or more cameras of the electronic device with the predefined static image includes displaying a text display region.

33. The method of claim 25, further comprising:

in response to receiving the user input to transition and further in accordance with the first type of live media stream including live audio and live video;

transitioning display of one or more avatars of one or more participants of the communication session by concurrently;

reducing sizes of the displayed one or more avatars of the one or more participants;

changing shapes of the displayed one or more avatars of the one or more participants; and

changing locations of the displayed one or more avatars of the one or more participants.

34. The method of claim 25, further comprising:

detecting user input requesting to end transmitting an outgoing live media stream; and

in response to detecting user input request to end transmitting the outgoing live media stream;

ceasing to transmit the outgoing live media stream; and

ceasing to receive live media streams from one or more participants of the communication session.

35. The method of claim 25, further comprising:

before receiving user input to transition from transmitting the outgoing live media stream of the first type to transmitting the outgoing live media stream of the second type, concurrently displaying;

a representation of the outgoing live media stream of the first type that is being transmitted to one or more participants of the communication session at a first location on the display; and

one or more representations of one or more incoming live media streams from the one or more participants of the communication session at one or more respective locations on the display, wherein;

in response to receiving the user input to transition and in accordance with the first type of live media stream including live audio and live video;

replacing, on the display, display of live video captured by one or more cameras of the electronic device with the predefined static image while continuing to output live audio includes;

 

ceasing to display, at the first location, the first representation of the outgoing live media stream and displaying, at the first location, a second representation of the outgoing live media stream, wherein the first representation of the outgoing live media stream is a representation of live video and the second representation of the outgoing live media stream is the predefined static image; and

 

continuing to display the one or more representations of the one or more incoming live media streams from the one or more participants of the communication session at the one or more respective locations on the display while displaying the second representation of the outgoing live media stream at the first location on the display.

36. The method of claim 25, wherein the predefined static image identifies a user of the electronic device.