MTSI based UE configurable for video region-of-interest (ROI) signaling

• US 10,225,817 B2
• Filed: 08/21/2017
• Issued: 03/05/2019
• Est. Priority Date: 04/26/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Intel IP Corporation (Intel Corporation)
• Inventors: Oyman, Ozgur
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US15/682,168
• Publication Number:
  US 20170374647A1
• Time in Patent Office: 561 Days
• Field of Search: 348 1402, 348 1407, 348 1408, 348 1409, 348142, 348 1416, 348143, 348 77, 348135, 37524001, 37524008, 37524012, 37524013, 37524026, 37524016, 381 92, 382254, 382103, 382239, 382282, 382232, 455458
• CPC Class Codes: H04J 11/00 Orthogonal multiplex systems, e.g. using WALSH codesH04J13/00 takes precedence
• CPC Class Codes: H04J 11/0023 Interference mitigation or co-ordination direct sequence spread spectrum [DSSS] systems H04B1/7097; frequency hopping H04B1/713; allocation criteria for ingress interference avoidance H04L5/0062; frequency allocation criteria for requirements on out-of-channel emissions H04L5/0066; arrangements for removing intersymbol interference or baseband equalisers H04L25/03006; peak power aspects in multicarrier modulation H04L27/2614; power management H04W52/00; traffic scheduling H04W72/54, H04W72/541
• CPC Class Codes: H04J 11/005 of intercell interference
• CPC Class Codes: H04L 1/1671 the supervisory signal being transmitted together with control information
• CPC Class Codes: H04L 1/1812 Hybrid protocols; Hybrid automatic repeat request [HARQ]
• CPC Class Codes: H04L 1/1861 Physical mapping arrangements for ACK signaling see also H04L5/0053
• CPC Class Codes: H04L 12/1407 Policy-and-charging control [PCC] architecture
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 5/0007 the frequencies being orthogonal, e.g. OFDM(A), DMT
• CPC Class Codes: H04L 5/0032 Distributed allocation, i.e. involving a plurality of allocating devices, each making partial allocation
• CPC Class Codes: H04L 5/0033 each allocating device acting autonomously, i.e. without negotiation with other allocating devices
• CPC Class Codes: H04L 5/0044 allocation of payload
• CPC Class Codes: H04L 5/005 of common pilots, i.e. pilots destined for multiple users or terminals
• CPC Class Codes: H04L 5/0055 Physical resource allocation for ACK/NACK for physical mapping arrangements in ARQ protocols H04L1/1861
• CPC Class Codes: H04L 5/0073 Allocation arrangements that take into account other cell interferences for intercell interference mitigation or co-ordination in orthogonal multiplex systems H04J11/005
• CPC Class Codes: H04L 65/1016 IP multimedia subsystem [IMS]
• CPC Class Codes: H04L 65/1045 Proxies, e.g. for session initiation protocol [SIP]
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1073 : Registration or de-registration
• CPC Class Codes: H04L 65/1104 : Session initiation protocol [SIP]
• CPC Class Codes: H04L 65/403 : Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/65 : Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 : at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 : Responding to QoS
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 69/08 : Protocols for interworking; Protocol conversion
• CPC Class Codes: H04N 21/4728 : for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6437 : Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 7/141 : between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04W 24/06 : Testing, supervising or monitoring using simulated traffic
• CPC Class Codes: H04W 28/0236 : radio quality, e.g. interference, losses or delay
• CPC Class Codes: H04W 36/0066 : of control information between different types of networks in order to establish a new radio link in the target network
• CPC Class Codes: H04W 4/70 : Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 48/06 : based on traffic conditions
• CPC Class Codes: H04W 48/18 : Selecting a network or a communication service
• CPC Class Codes: H04W 52/0212 : managed by the network, e.g. network or access point is master and terminal is slave
• CPC Class Codes: H04W 52/0216 : using a pre-established activity schedule, e.g. traffic indication frame
• CPC Class Codes: H04W 52/08 : Closed loop power control
• CPC Class Codes: H04W 52/146 : Uplink power control
• CPC Class Codes: H04W 52/242 : taking into account path loss
• CPC Class Codes: H04W 52/32 : TPC of broadcast or control channels
• CPC Class Codes: H04W 52/325 : Power control of control or pilot channels
• CPC Class Codes: H04W 60/00 : Affiliation to network, e.g. registration; Terminating affiliation with the network, e.g. de-registration
• CPC Class Codes: H04W 68/00 : User notification, e.g. alerting and paging, for incoming communication, change of service or the like
• CPC Class Codes: H04W 68/02 : Arrangements for increasing efficiency of notification or paging channel
• CPC Class Codes: H04W 72/04 : Wireless resource allocation
• CPC Class Codes: H04W 72/044 : based on the type of the allocated resource
• CPC Class Codes: H04W 72/0446 : Resources in time domain, e.g. slots or frames
• CPC Class Codes: H04W 72/0453 : Resources in frequency domain, e.g. a carrier in FDMA
• CPC Class Codes: H04W 72/20 : Control channels or signalling for resource management
• CPC Class Codes: H04W 72/21 : in the uplink direction of a wireless link, i.e. towards the network
• CPC Class Codes: H04W 72/23 : in the downlink direction of a wireless link, i.e. towards a terminal
• CPC Class Codes: H04W 72/52 : based on load
• CPC Class Codes: H04W 72/542 : using measured or perceived quality
• CPC Class Codes: H04W 76/14 : Direct-mode setup
• CPC Class Codes: H04W 76/18 : Management of setup rejection or failure
• CPC Class Codes: H04W 76/27 : Transitions between radio resource control [RRC] states
• CPC Class Codes: H04W 76/28 : Discontinuous transmission [DTX]; Discontinuous reception [DRX]
• CPC Class Codes: H04W 76/38 : triggered by timers
• CPC Class Codes: H04W 8/005 : Discovery of network devices, e.g. terminals
• CPC Class Codes: H04W 8/22 : Processing or transfer of terminal data, e.g. status or physical capabilities
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: H04W 88/06 : adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

In a video conferencing application, a user viewing a scene can performing zooming by selecting an area in the scene, then dynamically switching a video feed of the scene to the selected area of the scene. The hardware and software of the video conferencing application can limit the video transmitted to the user to include only the selected area of the scene. Transmitting only the selected area of the scene, and excluding the non-selected area of the scene from transmission, can more efficiently use the available bandwidth of the video conferencing application.

Claims

1. An apparatus of a multimedia telephony services over an IP multimedia subsystem (MTSI) based user equipment (UE) configurable for video Region-of-Interest (ROI) signaling and operation as an MTSI sender, the apparatus comprising:

memory; and

processing circuitry configured to;

perform a Session Description Protocol (SDP) capability negotiation with a MTSI receiver;

provide a set of predefined ROIs to the MTSI receiver during the SDP capability negotiation;

receive signaling for one of the predefined ROIs in RTCP feedback reports from the MTSI receiver when the MTSI receiver is operating in a predefined ROI mode; and

encode a region corresponding to the one of the predefined ROIs of a captured frame, the encoded video comprises the encoded region of the captured frame and refrain from encoding video outside the predefined ROI of the captured frame.

2. The apparatus of claim 1, wherein the processing circuitry is configured to:

signal, to the MTSI receiver, video Region-of-Interest (ROI) information for a first ROI of the MTSI receiver in real-time protocol (RTP) packets, the RTP packets to include at least a zoom command to capture the first ROI, the first ROI being a requested ROI of the MTSI receiver; and

decode received RTP payload packets from the MTSI receiver, the RTP payload packets comprising video corresponding to the first ROI.

3. The apparatus of claim 2, wherein the processing circuitry is configured to:

receive signaling in RTCP feedback reports from the MTSI receiver requesting a second ROI, the second ROI provided during a second SDP capability negotiation, the second ROI being a predefined ROI of the MTSI sender; and

encode a region corresponding to the second ROI of a captured frame with the negotiated resolution, the encoded video comprises the encoded region of the captured frame.

4. The apparatus of claim 1, the SDP capability negotiation comprising a resolution of the one of the predefined ROIs.

5. The apparatus of claim 4, the encoded region having the negotiated resolution.

6. The apparatus of claim 5, wherein only the encoded region is encoded with the negotiated resolution.

7. The apparatus of claim 1, wherein the processing circuitry is configured to encode a region of a captured frame corresponding to the one of the predefined ROIs and refrain from encoding outside the one of the predefined ROIs of the captured frame, the encoded video comprising the encoded region of the captured frame.

8. The apparatus of claim 2, further comprising a display to display the decoded received RTP payload packets received from the MTSI sender.

9. The apparatus of claim 8, further comprising two or more antennas to receive the RTP packets from an enhanced node B (eNB).

10. A non-transitory computer-readable storage medium that stores instructions for execution by processing circuitry of a multimedia telephony services over an IP multimedia subsystem (MTSI) based user equipment (UE) configurable for video Region-of-Interest (ROI) signaling and operation as an MTSI sender, the instructions to configure the UE to:

perform a Session Description Protocol (SDP) capability negotiation with a MTSI receiver;

provide a set of predefined ROIs to the MTSI receiver during the SDP capability negotiation;

receive signaling for one of the predefined ROIs in RTCP feedback reports from the MTSI receiver when the MTSI receiver is operating in a predefined ROI mode; and

encode a region corresponding to the one of the predefined ROIs of a captured frame, the encoded video comprises the encoded region of the captured frame and refrain from encoding video outside the predefined ROI of the captured frame.

11. The non-transitory computer-readable storage medium of claim 10, wherein the instructions further configure the UE to:

signal, to the MTSI receiver, video Region-of-Interest (ROI) information for a first ROI of the MTSI receiver in real-time protocol (RTP) packets, the RTP packets to include at least a zoom command to capture the first ROI, the first ROI being a requested ROI of the MTSI receiver; and

decode received RIP payload packets from the MTSI receiver, the RIP payload packets comprising video corresponding to the first ROI.

12. The non-transitory computer-readable storage medium of claim 11, wherein the instructions further configure the UE to:

receive signaling in RTCP feedback reports from the MTSI receiver requesting a second ROI, the second ROI provided during a second SDP capability negotiation, the second ROI being a predefined ROI of the MTSI sender; and

encode a region corresponding to the second ROI of a captured frame with the negotiated resolution, the encoded video comprises the encoded region of the captured frame.

13. The non-transitory computer-readable storage medium of claim 10, the SDP capability negotiation comprising a resolution of the one of the predefined ROIs.

14. The non-transitory computer-readable storage medium of claim 13, the encoded region having the negotiated resolution.

15. The non-transitory computer-readable storage medium of claim 14, wherein only the encoded region is encoded with the negotiated resolution.

16. The non-transitory computer-readable storage medium of claim 10, wherein the instructions further configure the UE to encode a region of a captured frame corresponding to the one of the predefined ROIs and refrain from encoding outside the one of the predefined ROIs of the captured frame, the encoded video comprising the encoded region of the captured frame.

17. The non-transitory computer-readable storage medium of claim 10, wherein the instructions further configure the UE to display the decoded received RTP payload packets received from the MTSI sender.

18. An apparatus of a multimedia telephony services over an IP multimedia subsystem (MTSI) based user equipment (UE) configurable for video Region-of-Interest (ROI) signaling and operation as an MTSI receiver, the apparatus comprising:

memory; and

processing circuitry configured to;

perform a Session Description Protocol (SDP) capability negotiation with a MTSI sender;

obtain a set of predefined ROIs from the MTSI sender during the SDP capability negotiation;

operate in a predefined ROI mode;

encode information to signal one of the predefined ROIs in RTCP feedback reports; and

decode a region corresponding to the one of the predefined ROIs of a captured frame, the encoded video comprises the encoded region of the captured frame, the video outside the predefined ROI of the captured frame is refrained from encoding.

19. The apparatus of claim 18, wherein the processing circuitry is configured to:

receive, from the MTSI sender, video Region-of-Interest (ROI) information for a first ROI of the MTSI receiver in real-time protocol (RTP) packets, the RTP packets to include at least a zoom command to capture the first ROI, the first ROI being a requested ROI of the MTSI receiver; and

encode RTP payload packets for the MTSI sender, the RTP payload packets comprising video corresponding to the first ROI.

20. The apparatus of claim 19, wherein the processing circuitry is configured to:

encode signaling in RTCP feedback reports to the MTSI sender requesting a second ROI, the second ROI provided during a second SDP capability negotiation, the second ROI being a predefined ROI of the MTSI sender; and

decode a region corresponding to the second ROI of a captured frame with the negotiated resolution, the encoded video comprises the region of the captured frame.

21. The apparatus of claim 18, the SDP capability negotiation comprising a resolution of the one of the predefined ROIs.

22. The apparatus of claim 21, the encoded region having the negotiated resolution.

23. The apparatus of claim 22, wherein only the encoded region is encoded with the negotiated resolution.

24. The apparatus of claim 18, wherein the processing circuitry is configured to decode a region of a captured frame corresponding to the one of the predefined ROIs and refrain from encoding outside the one of the predefined ROIs of the captured frame, the decoded video comprising the region of the captured frame.

25. The apparatus of claim 19, further comprising two or more antennas to transmit the RTP packets from an enhanced node B (eNB).