Systems and methods for providing a multi-perspective video display

• US 10,462,530 B2
• Filed: 06/15/2017
• Issued: 10/29/2019
• Est. Priority Date: 10/29/1999
• Status: Expired due to Term

• Current Assignee: OpenTV, Inc. (Kudelski SA)
• Original Assignee: OpenTV, Inc. (Kudelski SA)
• Inventors: Hensgen, Debra, Pierre, Ludovic
• Primary Examiner(s): Shang, Annan Q
• Application Number:
  US15/623,930
• Publication Number:
  US 20170289644A1
• Time in Patent Office: 866 Days
• Field of Search: 725 43- 61, 725 87-118, 725131-134, 725139-142, 725151-153, 709217-232
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4335 Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; unloading stored programs G06F9/445; storage management in file systems G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/4345 Extraction or processing of SI, e.g. extracting service information from an MPEG stream
• CPC Class Codes: H04N 21/4349 by extracting from data carousels, e.g. extraction of software modules from a DVB carousel
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4351 involving reassembling additional data, e.g. rebuilding an executable program from recovered modules
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8166 involving executable data, e.g. software arrangements for executing specific programs G06F9/44; broadcasting computer programmes in broadcast systems H04H20/91; involving the movement of software or configuration parameters H04L67/34
• CPC Class Codes: H04N 21/8545 for generating interactive applications
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 5/76 : Television signal recording
• CPC Class Codes: H04N 7/163 : by receiver means only
• CPC Class Codes: H04N 9/8227 : the additional signal being at least another television signal

Abstract

A method for providing video and audio perspectives of audio/video content is presented. In an example embodiment, a media content item is received that includes audio/video content having at least one video perspective and a plurality of audio perspectives for the at least one video perspective. The at least one video perspective is provided with a first one of the plurality of audio perspectives for presentation on a display device. During the providing of the at least one video perspective with the first one of the plurality of audio perspectives, a first command is received from a media application configured to allow a user to select among the plurality of audio perspectives. Responsive to the first command, the video perspective is provided along with a selected second one of the plurality of audio perspectives in substitution of the first one of the plurality of audio perspectives for presentation on the display device.

Claims

1. A computerized method comprising:

receiving, by a media device, via a network and from a provider system, first data and video content, the first data indicating control information that identifies a video perspective of the video content and each of a plurality of audio perspectives of audio content related to a frame of reference for presentation with the video perspective;

receiving, by the media device, via the network and from the provider system, second data including metadata that relates to one or more attributes of the frame of reference for presentation of each of the plurality of audio perspectives with the video perspective;

accessing, by the media device and from a storage location, using a first identifier for a first audio perspective identified in the control information, first audio data for the first audio perspective selected from among the plurality of audio perspectives of audio content, the first audio perspective related to a first frame of reference for presentation with the video perspective;

providing, by the media device, the video perspective with the first audio data for the first audio perspective for presentation of the first audio perspective with the video perspective on a display device;

receiving, by the media device, at a time during the presentation of the video perspective rendered with the first audio perspective at the display device, a command indicating a selection of a second audio perspective from among the plurality of audio perspectives, the second audio perspective related to a second frame of reference for presentation with the video perspective;

identifying, by the media device, using the metadata in the second data and based on the time of the command, a portion of second audio data for the second audio perspective related to the second frame of reference for presentation of the second audio perspective with the video perspective;

accessing, by the media device, using a second identifier in the control information, the identified portion of the second audio data for the second audio perspective; and

responsive to the command indicating the selection of the second audio perspective, updating, by the media device, using the second audio data and during the presentation of the video perspective rendered with the first audio perspective at the display device, the presentation to render the video perspective with the selected second audio perspective in substitution of the first audio perspective.

2. The computerized method of claim 1, wherein each of the plurality of audio perspectives is associated with a scene of the video content.

3. The computerized method of claim 1, wherein each of the plurality of audio perspectives is associated with an event depicted in the video content.

4. The computerized method of claim 3, wherein the event comprises a sporting event.

5. The computerized method of claim 3, wherein the event comprises a live event.

6. The computerized method of claim 1, wherein the control information further comprises descriptive information for each of the plurality of audio perspectives.

7. The computerized method of claim 6, further comprising:

providing the descriptive information for each of the plurality of audio perspectives for presentation on the display device prior to the receiving of the command.

8. The computerized method of claim 1, wherein the first audio perspective of the plurality of audio perspectives corresponds to a first spoken language, and wherein the second audio perspective of the plurality of audio perspectives corresponds to a second spoken language different from the first spoken language.

9. The computerized method of claim 1, wherein the video content comprises the video perspective, and wherein the computerized method further comprises receiving the first audio perspective of the plurality of audio perspectives and the second audio perspective of the plurality of audio perspectives concurrently with the first data and the video content.

10. The computerized method of claim 9, wherein the first audio perspective of the plurality of audio perspectives comprises a first plurality of data packets, the second audio perspective of the plurality of audio perspectives comprises a second plurality of data packets, and the receiving of the first data and the video content comprises receiving a transport stream in which the first plurality of data packets and the second plurality of data packets are multiplexed.

11. The computerized method of claim 10, wherein the providing of the video perspective with the first audio perspective of the plurality of audio perspectives comprises identifying the first plurality of data packets based on metadata associated with each of the first plurality of data packets, and the providing of the video perspective with the second audio perspective of the plurality of audio perspectives comprises identifying the second plurality of data packets based on metadata associated with each of the second plurality of data packets.

12. The computerized method of claim 10, wherein the first plurality of data packets comprise a first elementary stream of the transport stream for the first audio perspective of the plurality of audio perspectives, and the second plurality of data packets comprise a second elementary stream of the transport stream for the second audio perspective of the plurality of audio perspectives.

13. The computerized method of claim 1, wherein the provider system comprises an Internet communication system or a cellular communication system.

14. The computerized method of claim 1, wherein the receiving of the first data and the video content continues during the providing of the video perspective with the first audio perspective and the providing of the video perspective with the second audio perspective.

15. The computerized method of claim 1, further comprising:

receiving, from the provider system via the network, a media application; and

initiating execution, in response to receiving the media application, of the media application on the media device.

16. The computerized method of claim 15, wherein the receiving of the media application occurs in conjunction with the receiving of the first data and the video content.

17. The computerized method of claim 1, wherein the media device is communicatively coupled with the display device or the media device comprises the display device.

18. The computerized method of claim 1, wherein the first frame of reference of the video perspective is based on a first perception of the video perspective, and wherein the second frame of reference of the video perspective is based on a second perception of the video perspective.

19. The computerized method of claim 18, wherein:

the first perception of the video perspective is of a first person; and

the second perception of the video perspective is of a second person different from the first person.

20. The computerized method of claim 19, wherein:

the first person is an announcer for an event of the video perspective; and

the second person is a viewer at the event.

21. The computerized method of claim 1, wherein the first audio perspective is based on a voice of a person, and wherein the second audio perspective is based on audio from an object.

22. The computerized method of claim 1, wherein the metadata relates a time of the first audio perspective to an offset in audio data for the first audio perspective.

23. The computerized method of claim 1, wherein the metadata is received with the first data and the video content.

24. A media content system comprising:

one or more hardware processors; and

a memory storing instructions that, when executed by at least one of the one or more hardware processors, cause the media content system to perform operations comprising;

receiving, via a network and from a provider system, first data and video content, the first data indicating control information that identifies a video perspective of the video content and each of a plurality of audio perspectives of audio content related to a frame of reference for presentation with the video perspective;

receiving, via the network and from the provider system, second data including metadata that relates to one or more attributes of the frame of reference for presentation of each of the plurality of audio perspectives with the video perspective;

accessing, from a storage location, using a first identifier for a first audio perspective identified in the control information, first audio data for the first audio perspective selected from among the plurality of audio perspectives of audio content, the first audio perspective related to a first frame of reference for presentation with the video perspective;

providing the video perspective with the first audio data for the first audio perspective for presentation of the first audio perspective with the video perspective on a display device;

receiving, at a time during the presentation of the video perspective rendered with the first audio perspective at the display device, a command indicating a selection of a second audio perspective from among the plurality of audio perspectives, the second audio perspective related to a second frame of reference for presentation with the video perspective;

identifying, using the metadata in the second data and based on the time of the command, a portion of second audio data for the second audio perspective related to the second frame of reference for presentation of the second audio perspective with the video perspective;

accessing, using a second identifier in the control information, the identified portion of the second audio data for the second audio perspective; and

responsive to the command indicating the selection of the second audio perspective, updating, using the second audio data and during the presentation of the video perspective rendered with the first audio perspective at the display device, the presentation to render the video perspective with the selected second audio perspective in substitution of the first audio perspective.

25. A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more hardware processors of a machine, cause the machine to perform operations comprising:

receiving, by a media device, via a network and from a provider system, first data and video content, the first data indicating control information that identifies a video perspective of the video content and each of a plurality of audio perspectives of audio content related to a frame of reference for presentation with the video perspective;

receiving, by the media device, via the network and from the provider system, second data including metadata that relates to one or more attributes of the frame of reference for presentation of each of the plurality of audio perspectives with the video perspective;

accessing, by the media device and from a storage location, using a first identifier for a first audio perspective identified in the control information, first audio data for the first audio perspective selected from among the plurality of audio perspectives of audio content, the first audio perspective related to a first frame of reference for presentation with the video perspective;

providing, by the media device, the video perspective with the first audio data for the first audio perspective for presentation of the first audio perspective with the video perspective on a display device;

receiving, by the media device, at a time during the presentation of the video perspective rendered with the first audio perspective at the display device, a command indicating a selection of a second audio perspective from among the plurality of audio perspectives, the second audio perspective related to a second frame of reference for presentation with the video perspective;

identifying, by the media device, using the metadata in the second data and based on the time of the command, a portion of second audio data for the second audio perspective related to the second frame of reference for presentation of the second audio perspective with the video perspective;

accessing, by the media device, using a second identifier in the control information, the identified portion of the second audio data for the second audio perspective; and

responsive to the command indicating the selection of the second audio perspective, updating, by the media device, using the second audio data and during the presentation of the video perspective rendered with the first audio perspective at the display device, the presentation to render the video perspective with the selected second audio perspective in substitution of the first audio perspective.

26. A computerized method comprising:

receiving, by a media device, via a network and from a provider system, first data and video content, the first data including control information that identifies a video perspective of an event and each of a plurality of audio perspectives of audio content related to a frame of reference for presentation with the video perspective;

receiving, by the media device, via the network and from the provider system, second data including metadata that relates to one or more attributes of the frame of reference for presentation of each of the plurality of audio perspectives with the video perspective;

accessing, by the media device and from a storage location, using a first identifier for a first audio perspective identified in the control information, first audio data for the first audio perspective selected from among the plurality of audio perspectives of audio content, the first audio perspective related to a first frame of reference for presentation with the video perspective;

generating, by the media device, using the first audio data, a first view to present, at a display device, the first view including the video perspective with the first audio perspective of the plurality of audio perspectives;

causing, by the media device, a presentation of the first view at the display device;

causing, by the media device, a presentation of a second view at the display device, the second view enabling selection of a second audio perspective from among the plurality of audio perspectives, the second audio perspective relating to a second frame of reference of the video perspective;

receiving, by the media device, at a time during the presentation of the second view at the display device, a command indicating a selection of the second audio perspective from among the plurality of audio perspectives;

identifying, by the media device, using the metadata in the second data and based on the time of the command, a portion of second audio data for the second audio perspective related to the second frame of reference for presentation of the second audio perspective with the video perspective;

accessing, by the media device, using a second identifier in the control information, the identified portion of the second audio data for the second audio perspective;

responsive to the command, using the second audio data and during the presentation of the second view, modifying the first view to substitute the first audio perspective with the second audio perspective to be presented with the video perspective; and

based on modifying the first view, causing, by the media device, the presentation of the first view to be updated with the modified first view.