Audio content delivery from multi-display device ecosystem

• US 9,866,916 B1
• Filed: 08/17/2016
• Issued: 01/09/2018
• Est. Priority Date: 08/17/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Boss, Gregory J., Hodges, Brent, Moore, Jr., John E., Rakshit, Sarbajit K.
• Primary Examiner(s): Kim, William J
• Application Number:
  US15/239,053
• Time in Patent Office: 510 Days
• Field of Search: None
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4852 for modifying audio parameters, e.g. switching between mono and stereo
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method for transmitting an audio stream based on a focus of attention of a user within a multi-screen venue is presented. The method may include connecting to a mobile device associated with the user. The method may also include tracking a user face associated with the user by using at least one camera. The method may then include determining the focus of attention based on the tracked user face. The method may further include determining a video feed corresponding with the determined focus of attention. The method may also include transmitting the audio stream corresponding with the determined video feed to the mobile device.

Claims

1. A method for transmitting an audio stream based on a focus of attention of a user within a multi-screen venue having a plurality of cameras, the method comprising:

detecting the user within the multi-screen venue;

connecting to a mobile device associated with the detected user;

retrieving a user profile associated with the detected user, wherein the user profile includes at least one user face image;

identifying a user face associated with the user within the multi-screen venue by using the plurality of cameras based on the at least one user face image;

tracking the identified user face within the multi-screen venue using the plurality of cameras;

determining the focus of attention based on the tracked user face;

determining a video feed corresponding with the determined focus of attention; and

transmitting the audio stream corresponding with the determined video feed to the mobile device.

2. The method of claim 1, further comprising:

determining that the focus of attention has changed;

determining a new video feed corresponding with the changed focus of attention; and

transmitting the audio stream corresponding with the determined new video feed to the mobile device.

3. The method of claim 1, further comprising:

determining that a trigger event has occurred; and

transmitting the audio stream corresponding with the determined trigger event to the mobile device.

4. The method of claim 3, further comprising:

generating an audio cue identifying a screen displaying a trigger event video feed; and

transmitting the generated audio cue to the mobile device.

5. The method of claim 3, further comprising:

collecting at least one user-defined trigger event;

wherein determining that a trigger event has occurred is based on the collected at least one user-defined trigger event.

6. The method of claim 4, wherein generating the audio cue identifying the screen displaying the trigger event video feed is biased to an audio channel to direct the user to the screen relative to the user.

7. A computer system for transmitting an audio stream based on a focus of attention of a user within a multi-screen venue having a plurality of cameras, comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising;

detecting the user within the multi-screen venue;

connecting to a mobile device associated with the detected user;

retrieving a user profile associated with the detected user, wherein the user profile includes at least one user face image;

identifying a user face associated with the user within the multi-screen venue by using the plurality of cameras based on the at least one user face image;

tracking the identified user face within the multi-screen venue using the plurality of cameras;

determining the focus of attention based on the tracked user face;

determining a video feed corresponding with the determined focus of attention; and

transmitting the audio stream corresponding with the determined video feed to the mobile device.

8. The computer system of claim 7, further comprising:

determining that the focus of attention has changed;

determining a new video feed corresponding with the changed focus of attention; and

transmitting the audio stream corresponding with the determined new video feed to the mobile device.

9. The computer system of claim 7, further comprising:

determining that a trigger event has occurred; and

transmitting the audio stream corresponding with the determined trigger event to the mobile device.

10. The computer system of claim 9, further comprising:

generating an audio cue identifying a screen displaying a trigger event video feed; and

transmitting the generated audio cue to the mobile device.

11. The computer system of claim 9, further comprising:

collecting at least one user-defined trigger event;

wherein determining that a trigger event has occurred is based on the collected at least one user-defined trigger event.

12. The computer system of claim 10, wherein generating the audio cue identifying the screen displaying the trigger event video feed is biased to an audio channel to direct the user to the screen relative to the user.

13. A computer program product for transmitting an audio stream based on a focus of attention of a user within a multi-screen venue having a plurality of cameras, comprising:

one or more computer-readable storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor, the program instructions comprising;

program instructions to detect the user within the multi-screen venue;

program instructions to connect to a mobile device associated with the detected user;

program instructions to retrieve a user profile associated with the detected user, wherein the user profile includes at least one user face image;

program instructions to identify a user face associated with the user within the multi-screen venue by using the plurality of cameras based on the at least one user face image;

program instructions to track the identified user face within the multi-screen venue using the plurality of cameras;

program instructions to determine the focus of attention based on the tracked user face;

program instructions to determine a video feed corresponding with the determined focus of attention; and

program instructions to transmit the audio stream corresponding with the determined video feed to the mobile device.

14. The computer program product of claim 13, further comprising:

program instructions to determine that the focus of attention has changed;

program instructions to determine a new video feed corresponding with the changed focus attention; and

program instructions to transmit the audio stream corresponding with the determined new video feed to the mobile device.

15. The computer program product of claim 13, further comprising:

program instructions to determine that a trigger event has occurred; and

program instructions to transmit the audio stream corresponding with the determined trigger event to the mobile device.

16. The computer program product of claim 13, further comprising:

program instructions to detect the user within the multi-screen venue.

17. The computer program product of claim 15, further comprising:

program instructions to generate an audio cue identifying a screen displaying a trigger event video feed; and

program instructions to transmit the generated audio cue to the mobile device.

18. The computer program product of claim 15, further comprising:

program instructions to collect at least one user-defined trigger event;

wherein determining that a trigger event has occurred is based on the collected at least one user-defined trigger event.