Methods and systems for mobile telepresence

• US 9,185,343 B2
• Filed: 05/13/2013
• Issued: 11/10/2015
• Est. Priority Date: 08/07/2009
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Blackberry Limited
• Inventors: George, Richard John, Oliver, Brian Alexander
• Primary Examiner(s): Nguyen, Joseph J
• Application Number:
  US13/893,172
• Publication Number:
  US 20130242037A1
• Time in Patent Office: 911 Days
• Field of Search: 348/14.12
• US Class Current: 1/1
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1104 Session initiation protocol [SIP]
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/44245 Monitoring the upstream path of the transmission network, e.g. its availability, bandwidth
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 48/08 Access restriction or access information delivery, e.g. discovery data delivery signalling during connection H04W76/00
• CPC Class Codes: H04W 52/0261 managing power supply demand, e.g. depending on battery level
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

Exemplary methods and systems relate to telepresence at mobile devices. In one example the method can be performed on the mobile device. The mobile device can determine real-time availability of resources. The mobile device can further determine the composition of the telepresence session. The mobile device can further communicate the determined composition of telepresence data to a remote server. The mobile device can further receive telepresence data and perform a portion of the telepresence session data. Additionally, the present system can be configured to operate on a server or other electronic device.

Claims

1. A method for telepresence at a mobile device comprising:

determining, on the mobile device, a real-time availability of resources, including a current status of a battery powering the mobile device and a current bandwidth available over a wireless network;

determining, on the mobile device, a video composition and a display composition of telepresence session data based at least in part on the determined availability of resources including video capabilities and display capabilities of the mobile device, wherein determining the display composition of the telepresence session data comprises selecting a video stream composition requiring higher bandwidth upon detection of availability of a higher bandwidth wireless network connection;

transmitting, from the mobile device to a remote server, the video composition and the display composition of telepresence data;

receiving, on the mobile device, a first portion of the telepresence session data in accordance with the display composition;

transmitting, by the mobile device, a second portion of the telepresence session data in accordance with the video composition; and

,

performing, on the mobile device, the first portion of the telepresence session data.

2. The method of claim 1, further comprising receiving a selection of one or more of a plurality of high-definition video sources available on the mobile device, and implementing that selection in determining the video composition of telepresence session data.

3. The method of claim 1, wherein the display composition is based on one or more of a resolution of a display at the mobile device, size of the display, and availability of video and audio codecs at the mobile device.

4. The method of claim 1, wherein the performing the first portion of the telepresence session data includes outputting an audio portion over speakers of the mobile device.

5. The method of claim 1, wherein the performing the first portion of the telepresence session data includes rendering a video portion on a display of the mobile device.

6. The method of claim 1, wherein the telepresence session data includes at least one of audio or video data.

7. A server-based method for providing telepresence-like capabilities to one or more mobile devices, comprising:

establishing a connection from the server to one or more mobile devices each having one or more cameras and at least one camera configured for high-definition video;

receiving telepresence session configuration information updates from one of the one or more mobile devices for a telepresence session;

receiving a determination of real-time availability of resources, including a current status of a battery powering the one or more mobile devices and a current bandwidth available over a wireless network to the respective one or more mobile devices;

determining, at the server, a video composition and a display composition of telepresence session data based at least in part on the determined availability of resources including video capabilities and display capabilities of the mobile device, wherein determining the display composition of the telepresence session data comprises selecting a video stream composition requiring higher bandwidth upon detection of availability of a higher bandwidth wireless network connection; and

transmitting, from the server, the video composition and the display composition of telepresence data to the mobile device.

8. The method of claim 7, wherein the video composition comprises a video composition format from available formats comprising a roster view, an active speaker view, and a thumbnail matrix view.

9. The method of claim 7, the video composition comprises one or more of a selection from the received video streams, a frame rate, and a resolution.

10. The method of claim 7, wherein the determining of the display composition of the telepresence session data comprises selecting a video stream composition requiring lower bandwidth upon detection of unavailability of the higher bandwidth wireless network connection.

11. A mobile telepresence-enabled device, comprising:

a display;

a network interface;

a memory resource;

processing resources coupled with the memory resource, the network interface, the display, for operation under battery power, and comprising one or more modules configured to determine a real-time availability of resources, including a current status of a battery powering the mobile device and a current bandwidth available over a wireless network;

determine a video composition and a display composition of telepresence session data based at least in part on the determined availability of resources including video capabilities and display capabilities of the mobile device, wherein determining the display composition of the telepresence session data comprises selecting a video stream composition requiring higher bandwidth upon detection of availability of a higher bandwidth wireless network connection;

transmit the video composition and the display composition of telepresence data;

receive a first portion of the telepresence session data in accordance with the display composition;

transmit a second portion of the telepresence session data in accordance with the video composition; and

,

perform the first portion of the telepresence session data.

12. The mobile device of claim 11, wherein the display composition is based on one or more of a resolution of the display at the mobile device, size of the display, and availability of video and audio codecs at the mobile device.

13. The mobile device of claim 11, wherein the determining of real-time availability of resources further includes determining whether an alternative network is available to mobile device.

14. The mobile device of claim 11, wherein the performing portion of the telepresence session data includes transmitting data to speakers of the mobile device.

15. The mobile device of claim 11, wherein the performing portion of the telepresence session data includes transmitting data to the display of the mobile device.