Interactive video conferencing

• US 10,148,868 B2
• Filed: 11/01/2017
• Issued: 12/04/2018
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Intel Corporation
• Inventors: Oyman, Ozgur, Giacalone, Jean-Pierre, Fox, Ivan
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US15/801,100
• Publication Number:
  US 20180054563A1
• Time in Patent Office: 398 Days
• CPC Class Codes: H04L 65/1106 Call signalling protocols; H.323 and related
• CPC Class Codes: H04L 65/4046 with distributed floor control
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MTSI) receiver operable to support region of interest (ROI) signaling with a MTSI sender is disclosed. The MTSI receiver can define a requested region of interest (ROI). The MTSI receiver can map the requested ROI to one or more pan, tilt, zoom and focus (PTZF) commands. The MTSI receiver can encode the one or more PTZF commands for transmission to the MTSI sender via real-time transport protocol (RTP) packets. The one or more PTZF commands can represent the requested ROI.

Claims

1. An apparatus of a multimedia telephony services over internet protocol (IP) multimedia subsystems (IMS) (MTSI) receiver operable to support region of interest (ROI) signaling with a MTSI sender, the apparatus comprising one or more processors and memory configured to:

define, at the MTSI receiver, a requested region of interest (ROI);

map, at the MTSI receiver, the requested ROI to one or more pan, tilt, zoom and focus (PTZF) commands;

encode the one or more PTZF commands for transmission to the MTSI sender via real-time transport protocol (RTP) packets, wherein the one or more PTZF commands represent the requested ROI;

decode, at the MTSI receiver, encoded video received from the MTSI sender, wherein the encoded video is within the requested ROI; and

store, in the memory, the one or more PTZF commands that are mapped to the requested ROI.

2. The apparatus of claim 1, further comprising a transceiver configured to transmit the one or more PTZF commands to the MTSI sender via the RTP packets.

3. The apparatus of claim 1, wherein the one or more PTZF commands are in accordance with an International Telecommunication Union (ITU) H.281 protocol that supports camera movements for pan, tilt, zoom and focus (PTZF).

4. The apparatus of claim 1, wherein:

a pan command is mapped to left or right movements or translations over an image plane when the MTSI sender includes a fixed non-moving camera;

or

a tilt command is mapped to up or down movements or translations over the image plane when the MTSI sender includes the fixed non-moving camera.

5. The apparatus of claim 1, wherein the one or more PTZF commands are signaled to the MTSI sender in accordance with a far end camera control (FECC) protocol.

6. The apparatus of claim 1, wherein the one or more PTZF commands are signaled to the MTSI sender for transmission via the RTP packets that carry H.224 frames using a stack internet protocol (IP)/user datagram protocol (UDP)/RTP/H.224/H.281.

7. The apparatus of claim 1, wherein the one or more PTZF commands enables the MTSI receiver to zoom into an arbitrary ROI.

8. At least one non-transitory machine readable storage medium having instructions embodied thereon for supporting an interactive zooming feature at a local client, the instructions when executed by one or more processors cause the local client to perform the following:

defining a region of interest (ROI) within a field of view of a camera associated with a remote client;

mapping the ROI to one or more pan, tilt, zoom and focus (PTZF) commands;

processing the one or more PTZF commands for transmission from the local UE to the remote client via real-time transport protocol (RTP) packets; and

decoding encoded video within the ROI received from the remote client, the encoded video including regions within the ROI and excluding regions outside the ROI; and

storing, in the memory, the one or more PTZF commands that are mapped to the requested ROI.

9. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

providing the encoded video within the ROI for rendering and display at the local client.

10. The at least one non-transitory machine readable storage medium of claim 8, wherein the one or more PTZF commands are in accordance with an International Telecommunication Union (ITU) H.281/H.224 protocol.

11. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

processing the one or more PTZF commands for delivery to the remote client in a single transmission.

12. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

defining the ROI based on a received user input.

13. The at least one non-transitory machine readable storage medium of claim 8, wherein the one or more PTZF commands are sent to the remote client in accordance with a far end camera control (FECC) protocol.

14. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

defining the ROI based on a user selection provided to the local client.

15. The at least one non-transitory machine readable storage medium of claim 8, further comprising instructions which when executed by the one or more processors cause the local client to perform the following:

operate a video conferencing application with the remote client that supports a ROI-based interactive zooming feature.

16. At least one non-transitory machine readable storage medium having instructions embodied thereon for performing video conferencing between a remote user equipment (UE) and a local UE, the instructions when executed by one or more processors cause the remote UE to perform the following:

processing, at the remote UE, one or more pan, tilt, zoom and focus (PTZF) commands received from the local UE via real-time transport protocol (RTP) packets;

identifying, at the remote UE, a region of interest (ROI) based on the one or more PTZF commands, the ROI being within a field of view of a camera of the remote UE;

encoding video within the ROI, the encoded video including regions within the ROI and excluding regions outside the ROI; and

processing, at the remote UE, the encoded video within the ROI for transmission to the local UE.

17. The at least one non-transitory machine readable storage medium of claim 16, further comprising instructions which when executed by the one or more processors cause the remote UE to perform the following:

storing, in a memory of the remote UE, the one or more PTZF commands and the ROI that corresponds to the one or more PTZF commands.

18. The at least one non-transitory machine readable storage medium of claim 16, further comprising instructions which when executed by the one or more processors cause the remote UE to perform the following:

processing the one or more PTZF commands received from the local UE in a single transmission.

19. The at least one non-transitory machine readable storage medium of claim 16, further comprising instructions which when executed by the one or more processors cause the remote UE to perform the following:

capturing the encoded video within the ROI via a fixed non-moving camera in the remote UE, wherein a pan command is mapped to left or right movements or translations over an image plane when the remote UE includes the fixed non-moving camera, or a tilt command is mapped to up or down movements or translations over the image plane when the remote UE includes the fixed non-moving camera.

20. The at least one non-transitory machine readable storage medium of claim 16, further comprising instructions which when executed by the one or more processors cause the remote UE to perform the following:

receiving the one or more PTZF commands from the local UE in accordance with a far end camera control (FECC) protocol.

21. The at least one non-transitory machine readable storage medium of claim 16, wherein the one or more PTZF commands are in accordance with an International Telecommunication Union (ITU) H.281/H.224 protocol.