Interactive video conferencing

• US 9,794,515 B2
• Filed: 06/13/2016
• Issued: 10/17/2017
• Est. Priority Date: 01/06/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Intel IP Corporation (Intel Corporation)
• Inventors: Oyman, Ozgur
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US15/180,510
• Publication Number:
  US 20160295166A1
• Time in Patent Office: 491 Days
• Field of Search: 345 55, 348 1402, 348 1407, 348 1408, 348 1409, 348 1405, 37524008, 37524016, 37524024, 382195, 725 25, 709219, 714 25
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/632 using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices broadcast-related systems characterised by transmission among terminal devices H04H60/80; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

Technology for a multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MTSI) receiver operable to support region of interest (ROI) signaling with a MTSI sender is disclosed. The MTSI receiver can receive a requested region of interest (ROI). The MTSI receiver can signal the requested ROI for transmission to the MTSI sender via a real-time transport control protocol (RTCP) feedback message. The MTSI receiver can decode encoded video received from the MTSI sender. The encoded video can correspond to the requested ROI.

Claims

1. An apparatus of a multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MT SI) receiver operable to support region of interest (ROI) signaling with a MTSI sender, the apparatus including one or more processors and memory configured to:

receive, at the MTSI receiver, a requested region of interest (ROI);

signal the requested ROI for transmission to the MTSI sender via a real-time transport control protocol (RTCP) feedback message; and

decode, at the MTSI receiver, encoded video received from the MTSI sender, wherein the encoded video corresponds to the requested ROI.

2. The apparatus of claim 1, further comprising a transceiver configured to:

transmit the requested ROI to the MTSI sender via the RTCP feedback message; and

receive the encoded video from the MTSI sender.

3. The apparatus of claim 1, wherein the one or more processors and memory are further configured to decode a sent ROI that corresponds to the encoded video received from the MTSI sender, wherein the sent ROI is received from the MTSI sender via a real-time transport protocol (RTP) header extension.

4. The apparatus of claim 1, wherein the one or more processors and memory are further configured to decode a session description protocol (SDP) offer message received from the MTSI sender, wherein the SDP offer message indicates that the MTSI sender supports an ROI feature.

5. The apparatus of claim 1, wherein the one or more processors and memory are further configured to signal a session description protocol (SDP) answer message to the transceiver for transmission to the MTSI sender, wherein the SDP answer message acknowledges that the MTSI sender supports an ROI feature.

6. The apparatus of claim 4, wherein the SDP offer message includes:

an RTCP feedback (FB) attribute to signal the requested ROI;

or

an extension map attribute to signal the requested ROI.

7. The apparatus of claim 5, wherein the SDP answer message includes:

an RTCP feedback (FB) attribute to signal the requested ROI;

or

an extension map attribute to signal the requested ROI.

8. The apparatus of claim 1, wherein the RTCP feedback message includes a position and a size that define the ROI.

9. The apparatus of claim 2, wherein the transceiver is configured to transmit the ROI to the MTSI sender, and the MTSI sender is configured to encode only video that corresponds to the requested ROI.

10. The apparatus of claim 3, wherein the RTP header extension includes a position and a size that define the sent ROI corresponding to the encoded video received from the MTSI sender.

11. The apparatus of claim 1, wherein the apparatus includes an antenna, a touch sensitive display screen, a speaker, a microphone, a graphics processor, an application processor, an internal memory, or a non-volatile memory port.

12. At least one non-transitory machine readable storage medium having instructions embodied thereon for supporting video region of interest (ROI) signaling between a first multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MTSI) client and a second MTSI client, the instructions when executed perform the following:

defining, using one or more processors of the first MTSI client, a region of interest (ROI) within a field of view of the second MTSI client;

formatting, using the one or more processors of the first MTSI client, the ROI for transmission to the second MTSI client via a real-time transport control protocol (RTCP) feedback message that directs the second MTSI client to capture video that corresponds to the ROI and encode the video that corresponds to the ROI;

decoding, using the one or more processors of the first MTSI client, encoded video that corresponds to the ROI received from the second MTSI client, the encoded video including regions that correspond to the ROI and excluding regions that do not correspond to the ROI, the encoded video including the regions that correspond to the ROI at an increased zoom level while substantially maintaining a defined quality level;

decoding, using the one or more processors of the first MTSI client, an actual ROI received from the second MTSI client that corresponds to the encoded video from the second MTSI client, wherein the actual ROI is received via a real-time transport protocol (RTP) header extension; and

providing, using the one or more processors of the first MTSI client, the encoded video that corresponds to the ROI for rendering and display at the first MTSI client.

13. The at least one non-transitory machine readable storage medium of claim 12, further comprising instructions when executed perform the following:

decoding a session description protocol (SDP) offer message received from the second MTSI client, wherein the SDP offer message indicates that the second MTSI client supports a ROI feature.

14. The at least one non-transitory machine readable storage medium of claim 12, further comprising instructions when executed perform the following:

processing a session description protocol (SDP) answer message for transmission to the second MTSI client, wherein the SDP answer message indicates that the second MTSI client supports a ROI feature.

15. The at least one non-transitory machine readable storage medium of claim 12, wherein the RTCP feedback message includes a position and a size that define the ROI.

16. The at least one non-transitory machine readable storage medium of claim 12, wherein the RTP header extension includes a position and a size that define the actual ROI corresponding to the encoded video received from the second MTSI client.

17. An apparatus of a multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MTSI) sender operable to support video region of interest (ROI) signaling with a MTSI receiver, the apparatus including one or more processors and memory configured to:

decode, at the MTSI sender, a region of interest (ROI) received from the MTSI receiver, wherein the ROI is received via a real-time transport control protocol (RTCP) feedback message

generate, at the MTSI sender, encoded video that corresponds to the ROI, the encoded video including regions that correspond to the ROI and excluding regions that do not correspond to the ROI, the encoded video including the regions that correspond to the ROI at an increased zoom level while substantially maintaining a defined quality level; and

signal the encoded video to a transceiver of the apparatus for transmission to the MTSI receiver to enable the MTSI receiver to render and display the encoded video that corresponds to the ROI.

18. The apparatus of claim 17, wherein the one or more processors and memory are further configured to:

signal an actual ROI to the transceiver of the apparatus for transmission to the MTSI receiver, wherein the actual ROI corresponds to the encoded video, wherein the actual ROI is transmitted via a real-time transport protocol (RTP) header extension.

19. The apparatus of claim 17, wherein the one or more processors and memory are further configured to signal a session description protocol (SDP) offer message to the transceiver of the apparatus for transmission to the MTSI receiver, wherein the SDP offer message indicates that the MTSI sender supports an ROI feature.

20. The apparatus of claim 17, wherein the one or more processors and memory are further configured to decode a session description protocol (SDP) answer message received from the MTSI receiver, wherein the SDP answer message acknowledges support of an ROI feature at the MTSI sender.

21. The apparatus of claim 17, wherein the RTCP feedback message includes a position and a size that define the ROI.