PROCESS FOR INCREASING THE QUALITY OF EXPERIENCE FOR USERS THAT WATCH ON THEIR TERMINALS A HIGH DEFINITION VIDEO STREAM

• US 20160360267A1
• Filed: 12/05/2014
• Published: 12/08/2016
• Est. Priority Date: 01/14/2014
• Status: Abandoned Application

• Current Assignee: RPX Corporation
• Original Assignee: Alcatel-Lucent SA (Nokia Corporation)
• Inventors: Gonguet, Arnaud, Marilly, Emmanuel, Tytgat, Donny
• Application Number:
  US15/111,342
• Publication Number:
  US 20160360267A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/25825 involving client display capabilities, e.g. screen resolution of a mobile phone optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics

Abstract

Process for increasing the Quality of Experience for users that watch on their terminals (1) a high definition video stream (2, I, V) captured by at least one video capturing device (3) and provided by a server (4) to which said users are connected through their terminals (1) in a network, said process providing for: —collecting, for each user of a sample of the whole audience of said video stream, at least information about the position of the gaze of said user on said video stream; —aggregating all of said collected information and analysing said aggregated information to identify the main regions of interest (R1, R2, R3, R4) for said video stream according to the number of users'"'"' gazes positioned on said regions of interest; —selecting at least a region of interest (R1, R2, R3) of said video stream to be displayed on some terminals (1) of said users.

Claims

1. Process for increasing the Quality of Experience for users that watch on their terminals a high definition video stream captured by at least one video capturing device and provided by a server to which said users are connected through their terminals in a network, said process providing for:

collecting, for each user of a sample of the whole audience of said video stream, at least information about the position of the gaze of said user on said video stream;

aggregating all of said collected information and analysing said aggregated information to identify the main regions of interest for said video stream according to the number of users'"'"' gazes positioned on said regions of interest;

selecting at least a region of interest of said video stream to be displayed on some terminals of said users;

said process wherein the video stream comprises several synchronised video views, the main regions of interest of said video stream being identified from processing said video views, said process providing for creating for each video view a 2D saliency map localising the regions of interests of said video view, and thus for creating from all of said 2D saliency maps a global 3D saliency map so as to identify the main regions of interest of the video stream.

3. Process according to claim 1, wherein the video views are video inputs that have been each captured by a video capturing device.

4. Process according to claim 1, wherein the video views are virtual camera views that have been generated from a scene model built from video inputs that have been each captured by a video capturing device.

6. Process according to claim 1, wherein it provides for transforming each 2D saliency map so as to create back-projections of the regions of interest of said 2D saliency map and thus to create a 3D saliency estimate for said 2D saliency map from said back-projections, the global 3D saliency map being created upon combination of all 3D saliency estimates.

7. Process according to claim 1, wherein it provides for analysing the created 3D saliency map to optimize the creation of a further 3D saliency map.

16. Computer program adapted to perform a process according to claim 1.

2. (canceled)

5. (canceled)

8. Engine for increasing the Quality of Experience for users that watch on their terminals a high definition video stream captured by at least one video capturing device and provided by a server to which said users are connected through their terminals in a network, said engine comprising:

at least a collector module for collecting, for each user of at least a sample of the whole audience of said video stream, at least information about the position of the gaze of said user on said video stream;

at least an estimator module that comprises means for aggregating all of said collected information and means for analysing said aggregated information to identify the main regions of interest for said video stream according to the number of users'"'"' gazes positioned on said regions of interest;

at least a selector module adapted for selecting at least a region of interest and for interacting with said server so that said selected region of interest will be displayed on some terminals of said users;

said engine wherein the video stream comprises several synchronised video views and is provided from several video capturing devices, the collector module or the estimator module comprising means for creating for each video view a 2D saliency map localising the regions of interests of said video view, and the estimator module comprising means for creating from all of said 2D saliency maps a global 3D saliency map so as to identify the main regions of interest of the video stream.

9. Engine according to claim 8, wherein it further comprises a tracker module adapted to track the collected positions of the gazes of users for predicting the next positions of said gazes.

10. Engine according to claim 8, wherein it further comprises a trend module adapted to track the identified regions of interests to identify the evolution of the number of users'"'"' gazes positioned on said regions of interest.

11. Engine according to claim 8, wherein it further comprises an alert module adapted to track, for each identified region of interest, the number of users'"'"' gazes positioned on said region of interest, so as to send an alert for identifying new regions of interest when one of said number changes significantly.

12. Engine according to claim 8, wherein it further comprises a optimizer module adapted to interact with the selector module for optimizing the selection of regions of interests according to information about at least the number of users'"'"' gazed positioned on said regions of interest and/or technical capabilities of the network and/or the terminals of users.

13. Server for providing a high definition video stream captured by at least one video capturing device to users connected through their terminals to said server in a network, so that said users watch said video stream on their terminals, said server comprising means for interacting with an engine according to claim 8 to increase the Quality of Experience for said users, said means comprising:

a focus module comprising means for interacting with the selector module of said engine to build at least one ROI video stream comprising a region of interest selected by said selector module;

a streamer module comprising means for providing the ROI video stream to some of said users.

14. Server according to claim 13, wherein it comprises a Quality of Service analyser module comprising means for providing to the optimizer module of the engine information about technical capabilities of the network and/or the terminals through which users are connected to said server, so as to optimize the selection of regions of interest according at least to said information.

15. Architecture for a network for providing to users connected through their terminals a high definition video stream to be watched by said users on said terminals, said video stream being captured by at least one video capturing device, said architecture comprising:

an engine for increasing the Quality of Experience for users, comprising;

at least a collector module for collecting, for each user of at least a sample of the whole audience of said video stream, at least information about the position of the gaze of said user on said video stream;

at least an estimator module that comprises means for aggregating all of said collected information and means for analysing said aggregated information to identify the main regions of interest for said video stream according to the number of users'"'"' gazes positioned on said regions of interest;

a selector module adapted for selecting at least a region of interest to be displayed on some terminals of said users;

a server to which users are connected through their terminals, said server providing said high definition video stream to said users, said server further comprising;

a focus module comprising means for interacting with the selector module of said engine to build at least one ROI video stream comprising a region of interest selected by said selector module;

a streamer module comprising means for providing the ROI video stream to some of said users;

said architecture wherein the video stream comprises several synchronised video views and is provided from several video capturing devices, the collector module or the estimator module comprising means for creating for each video view a 2D saliency map localising the regions of interests of said video view, and the estimator module comprising means for creating from all of said 2D saliency maps a global 3D saliency map so as to identify the main regions of interest of the video stream.