Interactive video conferencing

• US 9,832,369 B2
• Filed: 11/30/2016
• Issued: 11/28/2017
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Intel Corporation
• Inventors: Oyman, Ozgur, Giacalone, Jean-Pierre, Fox, Ivan
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US15/365,746
• Publication Number:
  US 20170085782A1
• Time in Patent Office: 363 Days
• Field of Search: 348 1401, 348 1403, 348 1405
• CPC Class Codes: H04L 65/1106 Call signalling protocols; H.323 and related
• CPC Class Codes: H04L 65/4046 with distributed floor control
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Technology for a local user equipment (UE) operable to perform video conferencing with a remote UE is disclosed. The local UE can define a region of interest (ROI) within a field of view of a camera of the remote UE. The local UE can map the ROI to one or more pan, tilt, zoom and focus (PTZF) commands. The local UE can send the one or more PTZF commands to the remote UE, wherein the remote UE is configured to identify the ROI based on the one or more PTZF commands. The local UE can receive encoded video within the ROI from the remote UE. The encoded video can include the regions within the ROI at an increased zoom level while substantially maintaining a defined quality level to enable the encoded video within the ROI to be rendered and displayed at the local UE.

Claims

1. An apparatus of a multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MTSI) receiver operable to support region of interest (ROI) signaling with a MTSI sender, the apparatus comprising one or more processors and memory configured to:

define, at the MTSI receiver, a requested region of interest (ROI)

map, at the MTSI receiver, the requested ROI to one or more pan, tilt, zoom and focus (PTZF) commands;

store, in the memory, the one or more PTZF commands that are mapped to the requested ROI;

signal the one or more PTZF commands representing the requested ROI for transmission to the MTSI sender via real-time transport protocol (RTP) packets, wherein the requested ROI is identified at the MTSI sender based on the one or more PTZF commands; and

decode, at the MTSI receiver, encoded video received from the MTSI sender, wherein the encoded video is within the requested ROI.

2. The apparatus of claim 1, further comprising a transceiver configured to:

transmit the one or more PTZF commands to the MTSI sender via the RTP packets; and

receive the encoded video from the MTSI sender.

3. The apparatus of claim 1, wherein the one or more PTZF commands are in accordance with an International Telecommunication Union (ITU) H.281 protocol that supports camera movements for pan, tilt, zoom and focus (PTZF).

4. The apparatus of claim 1, wherein:

a pan command is mapped to left or right movements or translations over an image plane when the MTSI sender includes a fixed non-moving camera;

or

a tilt command is mapped to up or down movements or translations over the image plane when the MTSI sender includes the fixed non-moving camera.

5. The apparatus of claim 1, wherein the one or more PTZF commands are signaled to the MTSI sender in accordance with a far end camera control (FECC) protocol.

6. The apparatus of claim 1, wherein the one or more PTZF commands are signaled to the MTSI sender for transmission via the RTP packets that carry H.224 frames using a stack internet protocol (IP)/user datagram protocol (UDP)/RTP/H.224/H.281.

7. The apparatus of claim 1, wherein the one or more PTZF commands enables the MTSI receiver to zoom into an arbitrary ROI.

8. At least one non-transitory machine readable storage medium having instructions embodied thereon for operating a video conferencing application at a local client that supports an interactive zooming feature, the instructions when executed by one or more processors cause the local client to perform the following:

defining a region of interest (ROI) within a field of view of a camera associated with a remote client;

mapping the ROI to one or more pan, tilt, zoom and focus (PTZF) commands;

processing the one or more PTZF commands for transmission from the local UE client to the remote client via real-time transport protocol (RTP) packets, wherein the ROI is identified at the remote client based on the one or more PTZF commands;

decoding encoded video within the ROI received from the remote client, the encoded video including regions within the ROI and excluding regions outside the ROI; and

providing the encoded video within the ROI for rendering and display at the local client.

9. The at least one non-transitory machine readable storage medium of claim 8, wherein the one or more PTZF commands are in accordance with an International Telecommunication Union (ITU) H.281/H.224 protocol.

10. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

processing the one or more PTZF commands for delivery to the remote client in a single transmission.

11. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

defining the ROI based on a received user input.

12. The at least one non-transitory machine readable storage medium of claim 8, wherein the one or more PTZF commands are sent to the remote client in accordance with a far end camera control (FECC) protocol.

13. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

defining the ROI based on a user selection provided to the local client.

14. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

operate a video conferencing application with the remote client that supports a ROI-based interactive zooming feature.

15. An apparatus of a remote user equipment (UE) operable to perform video conferencing with a local UE, the apparatus comprising one or more processors configured to:

process, at the remote UE, one or more pan, tilt, zoom and focus (PTZF) commands received from the local UE via real-time transport protocol (RTP) packets;

identify, at the remote UE, a region of interest (ROI) based on the one or more PTZF commands, the ROI being within a field of view of a camera of the remote UE;

encode video within the ROI, the encoded video including regions within the ROI and excluding regions outside the ROI; and

process, at the remote UE, the encoded video within the ROI for transmission to the local UE.

16. The apparatus of claim 15, further comprising memory configured to store the one or more PTZF commands and the ROI that corresponds to the one or more PTZF commands.

17. The apparatus of claim 15, wherein the one or more PTZF commands are in accordance with an International Telecommunication Union (ITU) H.281/H.224 protocol.

18. The apparatus of claim 15, wherein the one or more processors are configured to process the one or more PTZF commands received from the local UE in a single transmission.

19. The apparatus of claim 15, further comprising a fixed non-moving camera configured to capture the encoded video within the ROI, wherein a pan command is mapped to left or right movements or translations over an image plane when the remote UE includes the fixed non-moving camera, or a tilt command is mapped to up or down movements or translations over the image plane when the remote UE includes the fixed non-moving camera.

20. The apparatus of claim 15, wherein the one or more PTZF commands are received from the local UE in accordance with a far end camera control (FECC) protocol.