Method of displaying multiple content streams on user device

• US 10,080,044 B2
• Filed: 05/08/2012
• Issued: 09/18/2018
• Est. Priority Date: 09/26/2009
• Status: Active Grant

• Current Assignee: Mimik Technology Canada Incorporated
• Original Assignee: Mimik Technology, Inc.
• Inventors: Sharif-Ahmadi, Seyed, Arjomandi, Fay
• Primary Examiner(s): Asanbayev, Oleg
• Assistant Examiner(s): Doshi, Akshay
• Application Number:
  US13/466,900
• Publication Number:
  US 20130125171A1
• Time in Patent Office: 2,324 Days
• Field of Search: 725 9- 16
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 43/0835 One way packet loss
• CPC Class Codes: H04L 43/0847 Transmission error
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04M 15/42 Dynamic individual rates per user
• CPC Class Codes: H04N 21/23113 involving housekeeping operations for stored content, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; snloading stored programs G06F9/445; housekeeping operations in file systems, e.g. deletion policies G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/25833 involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities allocation of resources considering hardware capabilities in multiprogramming arrangements G06F9/5044; allocation of resources considering software capabilities in multiprogramming arrangements G06F9/5055
• CPC Class Codes: H04N 21/25891 : being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2665 : Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/4183 : providing its own processing capabilities, e.g. external module for video decoding
• CPC Class Codes: H04N 21/4263 : involving specific tuning arrangements, e.g. two tuners
• CPC Class Codes: H04N 21/4334 : Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4335 : Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; unloading stored programs G06F9/445; storage management in file systems G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/43615 : Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/437 : Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server flow control in data networks H04L47/10; streaming protocols, e.g. RTP or RTCP, H04L65/65; scheduling or organising the servicing of application requests in data packet switching networks H04L67/60
• CPC Class Codes: H04N 21/4402 : involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/440245 : the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/44204 : Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/44213 : Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44226 : on social networks
• CPC Class Codes: H04N 21/4532 : involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/458 : Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/4667 : Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information

Abstract

A method and system for micro-cloud computing is described. An application serving node, which is a specifically configured piece of customer premises equipment such as a set-top box, is the server for a micro-cloud network. Various user devices that form part of the micro-cloud network, such as television sets and smart phones, communicate with the application serving node. The application serving node is configured with software that allows various heterogeneous types of data to be obtained and streamed to the user devices in multiple ways. The application serving node performs much of the processing necessary for such content streaming itself and thus represents an edge-based network architecture.

Claims

1. A method of displaying a first video stream from a first video source and a second video stream from a second video source on a user device, said user device registered to a serving node, in different areas of a display of said user device, the method comprising:

determining a type of said user device and display characteristics of said user device;

determining a network connection from said serving node to said user device;

streaming a transcoded and transrated first video stream from said first video source via said serving node to said display of said user device;

invoking said second video stream from said second video source on a menu available on said display;

obtaining, at said serving node, said second video stream from said second video source and transcoding and transrating said second video stream based on said network connection and on said display characteristics of said user device;

providing, by said serving node, coordinates for each of said first video stream and said second video stream to a user interface to fit both said first video stream and said second video stream on said display;

providing, by said serving node, said transcoded and transrated first video stream and said transcoded and transrated second video stream to said user device, wherein said user interface redirects said first video stream and said second video stream based on said coordinates;

receiving, by said serving node, a frame number and a location associated with one of said first video stream or said second video stream being displayed on said user device, the frame number and the location being determined in response to selecting by a user said first video stream or said second video stream;

retrieving, by said serving node, extractable information associated with the selected frame number and the location;

displaying, by said serving node, the extractable information;

inspecting, by said serving node, said first video stream to determine an identity of the user associated with said first video stream, a context associated with said first video stream, and a time said first video stream was displayed;

comparing, by said serving node, said first video stream to a history associated with the user stored at said serving node to determine whether said first video stream is typical or atypical in relation to the user;

comparing, by said serving node, the context of said first video stream with a history of other users to determine whether a social variable is associated with said first video stream; and

updating, by said serving node, the history associated with the user and user characterization associated with the user based on the context the time, said user device, whether said first video stream was typical and whether the social variable was present.

2. The method of claim 1, wherein said first video stream is television content and said second video stream is Internet content.

3. The method of claim 1, wherein said serving node is located at a premises of said user associated with said user device.

4. The method of claim 3, wherein said user device is a television.

6. The method of claim 1, wherein said user device is a mobile device.

7. The method of claim 1, wherein said second video stream is multiplexed by said serving node.

8. The method of claim 1, wherein said network connection is a wireless connection.

5. A system for sharing a display of a user device, the system comprising:

a serving node, wherein said user device is registered to said serving node, said serving node receiving a first video stream from a first source and a second video stream from a second source, said serving node configured to determine a type of said user device and display characteristics of said user device and a network connection to said user device;

said serving node having an interface to said user device, said interface determining coordinates for said first video stream and said second video stream;

said serving node configured to transcode and transrate said first video stream and said second video stream based on said network connection and on the display characteristics of said user device;

said serving node configured to communicate said first video stream and said second video stream to said display of said user device using said coordinates;

said display configured to display said first video stream and said second video stream;

said serving node configured to receive a frame number and a location associated with one of said first video stream or said second video stream displayed on said display, the frame number and the location being determined when selected by a user;

said serving node configured to retrieve extractable information associated with the selected frame number and the location and to send the extractable information to said user device for display;

said serving node configured to inspect said first video stream to determine an identity of the user associated with said first video stream, a context associated with said first video stream, and a time said first video stream was accessed by the user;

said serving node configured to compare said first video stream to a history associated with the user stored at said serving node to determine whether said first video stream is typical or atypical in relation to the user;

said serving node configured to compare the context of said first video stream with a history of other users to determine whether a social variable is associated with said first video stream; and

said serving node configured to update the history associated with the user and user characterization associated with the user based on the context, the time, said user device, whether said first video stream was typical, and whether the social variable was present.