MTSI based UE configurable for video region-of-interest (ROI) signaling

• US 9,743,380 B2
• Filed: 03/29/2016
• Issued: 08/22/2017
• Est. Priority Date: 04/26/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Intel IP Corporation (Intel Corporation)
• Inventors: Oyman, Ozgur
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US15/083,897
• Publication Number:
  US 20160227166A1
• Time in Patent Office: 511 Days
• Field of Search: 348 1402, 348 1408, 348 1409, 348 1413, 348 1416, 348 43, 348 51, 348142, 348 1407, 37524008, 37524012, 37524026, 37524001, 382240, 382173, 709217, 725 62, 345 55
• CPC Class Codes: H04J 11/00 Orthogonal multiplex systems, e.g. using WALSH codesH04J13/00 takes precedence
• CPC Class Codes: H04J 11/0023 Interference mitigation or co-ordination direct sequence spread spectrum [DSSS] systems H04B1/7097; frequency hopping H04B1/713; allocation criteria for ingress interference avoidance H04L5/0062; frequency allocation criteria for requirements on out-of-channel emissions H04L5/0066; arrangements for removing intersymbol interference or baseband equalisers H04L25/03006; peak power aspects in multicarrier modulation H04L27/2614; power management H04W52/00; traffic scheduling H04W72/54, H04W72/541
• CPC Class Codes: H04J 11/005 of intercell interference
• CPC Class Codes: H04L 1/1671 the supervisory signal being transmitted together with control information
• CPC Class Codes: H04L 1/1812 Hybrid protocols; Hybrid automatic repeat request [HARQ]
• CPC Class Codes: H04L 1/1861 Physical mapping arrangements for ACK signaling see also H04L5/0053
• CPC Class Codes: H04L 12/1407 Policy-and-charging control [PCC] architecture
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 5/0007 the frequencies being orthogonal, e.g. OFDM(A), DMT
• CPC Class Codes: H04L 5/0032 Distributed allocation, i.e. involving a plurality of allocating devices, each making partial allocation
• CPC Class Codes: H04L 5/0033 each allocating device acting autonomously, i.e. without negotiation with other allocating devices
• CPC Class Codes: H04L 5/0044 allocation of payload
• CPC Class Codes: H04L 5/005 of common pilots, i.e. pilots destined for multiple users or terminals
• CPC Class Codes: H04L 5/0055 Physical resource allocation for ACK/NACK for physical mapping arrangements in ARQ protocols H04L1/1861
• CPC Class Codes: H04L 5/0073 Allocation arrangements that take into account other cell interferences for intercell interference mitigation or co-ordination in orthogonal multiplex systems H04J11/005
• CPC Class Codes: H04L 65/1016 IP multimedia subsystem [IMS]
• CPC Class Codes: H04L 65/1045 Proxies, e.g. for session initiation protocol [SIP]
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1073 : Registration or de-registration
• CPC Class Codes: H04L 65/1104 : Session initiation protocol [SIP]
• CPC Class Codes: H04L 65/403 : Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/65 : Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 : at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 : Responding to QoS
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 69/08 : Protocols for interworking; Protocol conversion
• CPC Class Codes: H04N 21/4728 : for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6437 : Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 7/141 : between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04W 24/06 : Testing, supervising or monitoring using simulated traffic
• CPC Class Codes: H04W 28/0236 : radio quality, e.g. interference, losses or delay
• CPC Class Codes: H04W 36/0066 : of control information between different types of networks in order to establish a new radio link in the target network
• CPC Class Codes: H04W 4/70 : Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 48/06 : based on traffic conditions
• CPC Class Codes: H04W 48/18 : Selecting a network or a communication service
• CPC Class Codes: H04W 52/0212 : managed by the network, e.g. network or access point is master and terminal is slave
• CPC Class Codes: H04W 52/0216 : using a pre-established activity schedule, e.g. traffic indication frame
• CPC Class Codes: H04W 52/08 : Closed loop power control
• CPC Class Codes: H04W 52/146 : Uplink power control
• CPC Class Codes: H04W 52/242 : taking into account path loss
• CPC Class Codes: H04W 52/32 : TPC of broadcast or control channels
• CPC Class Codes: H04W 52/325 : Power control of control or pilot channels
• CPC Class Codes: H04W 60/00 : Affiliation to network, e.g. registration; Terminating affiliation with the network, e.g. de-registration
• CPC Class Codes: H04W 68/00 : User notification, e.g. alerting and paging, for incoming communication, change of service or the like
• CPC Class Codes: H04W 68/02 : Arrangements for increasing efficiency of notification or paging channel
• CPC Class Codes: H04W 72/04 : Wireless resource allocation
• CPC Class Codes: H04W 72/044 : based on the type of the allocated resource
• CPC Class Codes: H04W 72/0446 : Resources in time domain, e.g. slots or frames
• CPC Class Codes: H04W 72/0453 : Resources in frequency domain, e.g. a carrier in FDMA
• CPC Class Codes: H04W 72/20 : Control channels or signalling for resource management
• CPC Class Codes: H04W 72/21 : in the uplink direction of a wireless link, i.e. towards the network
• CPC Class Codes: H04W 72/23 : in the downlink direction of a wireless link, i.e. towards a terminal
• CPC Class Codes: H04W 72/52 : based on load
• CPC Class Codes: H04W 72/542 : using measured or perceived quality
• CPC Class Codes: H04W 76/14 : Direct-mode setup
• CPC Class Codes: H04W 76/18 : Management of setup rejection or failure
• CPC Class Codes: H04W 76/27 : Transitions between radio resource control [RRC] states
• CPC Class Codes: H04W 76/28 : Discontinuous transmission [DTX]; Discontinuous reception [DRX]
• CPC Class Codes: H04W 76/38 : triggered by timers
• CPC Class Codes: H04W 8/005 : Discovery of network devices, e.g. terminals
• CPC Class Codes: H04W 8/22 : Processing or transfer of terminal data, e.g. status or physical capabilities
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: H04W 88/06 : adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

In a video conferencing application, a user viewing a scene can performing zooming by selecting an area in the scene, then dynamically switching a video feed of the scene to the selected area of the scene. The hardware and software of the video conferencing application can limit the video transmitted to the user to include only the selected area of the scene. Transmitting only the selected area of the scene, and excluding the non-selected area of the scene from transmission, can more efficiently use the available bandwidth of the video conferencing application.

Claims

1. An apparatus of a multimedia telephony services over an IP multimedia subsystem (MTSI) based user equipment (UE) configurable for video Region-of-Interest (ROI) signaling and operation as an MTSI sender, the apparatus comprising:

memory; and

processing circuitry configured to;

signal, to an MTSI receiver, video Region-of-Interest (ROI) information for a first ROI of the MTSI receiver in real-time protocol (RIP) packets, the RIP packets to include at least a zoom command to capture the first ROI, the first ROI being a requested ROI of the MTSI receiver;

decode received RTP payload packets from the MTSI receiver, the RIP payload packets comprising video corresponding to the first ROI;

receive signaling in RTCP feedback reports from the MTSI receiver requesting a second ROI, the second ROI provided during a Session Description Protocol (SDP) capability negotiation, the second ROI being a predefined ROI of the MTSI, sender; and

encode video corresponding to the second ROI in RTP payload packets for transmission to the MTSI receiver.

2. The apparatus of claim 1 wherein the processing circuitry is configured to encode a region of a captured frame corresponding to the second ROI and refrain from encoding outside the second ROI of the captured frame, the encoded video comprising the encoded region of the captured frame.

3. The apparatus of claim 1 wherein the processing circuitry is configured to signal the video ROI information for the first ROI of the MTSI receiver in RIP packets when operating in a remote camera control mode, the RIP packets to include at least the zoom command to capture the requested region of interest.

4. The apparatus of claim 1 wherein the processing circuitry is configured to signal the video ROI information for the first ROI of the MTSI receiver in RIP packets when operating in a remote camera control mode, the RTP packets to include at least the zoom command to capture the requested region of interest without changing a field-of-view (FOV) of the MTSI receiver.

5. The apparatus of claim 1 wherein the processing circuitry is configured to encode video corresponding to the second ROI in RIP payload packets for transmission to the MIST receiver without changing a field-of-view (FOV) of the MTSI sender.

6. The apparatus of claim 1 wherein e processing circuitry is further configured to:

perform the SDP capability negotiation with the MTSI receiver;

obtain a set of second ROIs from the MTSI receiver during the SDP capability negotiation; and

signal one of the second ROIs in RTCP feedback reports to the MTSI receiver when operating in a second ROI mode.

7. The apparatus of claim 1 wherein the processing circuitry is further configured to:

perform the SDP capability negotiation with the MTSI receiver;

provide a set of second ROIs to the MTSI receiver during the SDP capability negotiation, the set of second ROIs comprising a ROI region smaller than a size of a captured frame; and

receive signaling for one of the second ROIs in RTCP feedback reports from the MTSI receiver when the MTSI receiver is operating in a second ROT mode.

8. The apparatus of claim 7 wherein to signal the video ROI information to the MTSI receiver, the processing circuitry is configured to encode the video ROI information in RIP packets for transmission to the MTSI receiver.

9. The apparatus of claim 8 further comprising a camera to capture video corresponding to the second ROI.

10. The apparatus of claim 9 further comprising a display to display the decoded received RTP payload packets received from the MTSI receiver.

11. The apparatus of claim 1 wherein the processing circuitry is configured to:

perform the SDP capability negotiation with the MTSI receiver, the SDP capability negotiation comprising a resolution of the second ROT; and

encode a region corresponding to the second ROT of a captured frame with the negotiated resolution, the encoded video comprises the encoded region of the captured frame.

12. The apparatus of claim 11 wherein only the region corresponding to the second ROI of the captured frame is encoded with the negotiated resolution.

13. A non-transitory computer-readable storage medium that stores instructions for execution by processing circuitry of a multimedia telephony services over an IP multimedia subsystem (MTSI) based user equipment (UE) configurable for video Region-of-Interest (ROI) signaling and operation as an MTSI sender, the instructions to configure the UE to:

perform a Session Description Protocol (SDP) capability negotiation with an MTSI receiver;

encode video Region-of-Interest (ROI) information to signal a first ROI in real-time protocol (RTP) packets, the RTP packets to include at least a zoom command to capture the first ROI, the first ROI being a requested ROI of the MTSI receiver;

decode received RTP payload packets from the MTSI receiver, the RIP payload packets comprising video corresponding to the first ROI;

receive signaling in RTCP feedback reports from the MTSI receiver requesting a second ROI, the second ROI provided during the SDP capability negotiation, the second ROI being a predefined ROI of the MTSI sender; and

encode video corresponding to the second ROI in RTP payload packets for transmission to the MTSI receiver.

14. The non-transitory computer-readable storage medium of claim 13 wherein the processing circuitry is configured to encode a region of a captured frame corresponding to the second ROI and refrain from encoding outside the second ROI of the captured frame, the encoded video comprising the encoded region of the captured frame.

15. The non-transitory computer-readable storage medium of claim 13 wherein the processing circuitry is configured to signal the video ROI information for the first ROI of the MTSI receiver in RTP packets when operating in the remote camera control mode, the RTP packets to include at least the zoom command to capture the requested region of interest.

16. The non-transitory computer-readable storage medium of claim 13 wherein the processing circuitry is configured to signal the video ROI information for the first ROI of the MTSI receiver in RTP packets when operating in the remote camera control mode, the RIP packets to include at least the zoom command to capture the requested region of interest without changing a field-of-view (FOV) of the MTSI receiver.

17. The non-transitory computer-readable storage medium of claim 13 wherein the processing circuitry is configured to encode video corresponding to the second ROI in RTP payload packets for transmission to the MTSI receiver without changing a field-of-view (FOV) of the MTSI sender.

18. An apparatus of a multimedia telephony services over an IP multimedia subsystem (MTSI) based user equipment (UE) configurable for video Region-of-Interest (ROI) signaling and operation as an MTSI receiver, the apparatus comprising:

memory; and

processing circuitry configured to;

perform a Session Description Protocol (SDP) capability negotiation with an MTSI sender;

obtain a set of second ROIs from the MTSI sender during the SDP capability negotiation, the second ROIs being predefined ROIs of the MTSI sender, the set of second ROIs comprising a ROI region smaller than a size of a captured frame;

encode information to signal a first ROI in real-time protocol (RIP) packets;

the RTP packets to include at least a zoom command to capture the first ROI when operating in a remote camera control mode, the first ROI being a requested ROI of the MTSI receiver;

encode information to signal one of the second ROIs in RTCP feedback reports when operating in a predefined ROI mode;

encode video corresponding to the first ROI in RTP payload packets for transmission to the MTSI sender; and

decode received RTP payload packets from the MTSI sender, the RTP payload packets comprising video corresponding to the second ROI.

19. The apparatus of claim 18 further comprising:

a display to display the decoded received RIP payload packets received from the MTSI sender.

20. The apparatus of claim 19 further comprising two or more antennas to receive the RIP packets from an enhanced node B (eNB).