Information processing apparatus, method, and program

• US 10,484,728 B2
• Filed: 10/10/2014
• Issued: 11/19/2019
• Est. Priority Date: 10/21/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Inoue, Akira, Takahashi, Shusuke, Nishiguchi, Masayuki
• Primary Examiner(s): Alata, Yassin
• Application Number:
  US15/029,600
• Publication Number:
  US 20160269762A1
• Time in Patent Office: 1,866 Days
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/23602 Multiplexing isochronously with the video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4342 Demultiplexing isochronously with video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/60 for the sound signals

Abstract

An information processing apparatus for reproducing second content in synchronization with reproduction of first content by a second information processing apparatus different from the information processing apparatus, the first content comprising audio content. The information processing apparatus comprising circuitry configured to: extract a first feature from the audio content; obtain a second feature of the audio content, the second feature being together with the second content; compare the first feature with the second feature; and generate, based on results of the comparing, synchronization information used for reproducing the second content in synchronization with the first content.

Claims

1. An information receiving apparatus for reproducing a second media content in synchronization with reproduction of a first media content by a second information receiving apparatus different from the information receiving apparatus, the first media content comprising audio content, the information receiving apparatus comprising:

circuitry configured to;

extract an audio-derived feature from an audio signal generated via reproduction of the audio content by the second information receiving apparatus;

obtain, from an information providing apparatus different from the second information receiving apparatus, the second media content together with a synchronization feature, the synchronization feature being extracted from the audio content of the first media content prior to being transmitted by the information providing apparatus;

compare the audio-derived feature with the synchronization feature;

generate, based on results of the comparing, synchronization information; and

reproduce, using the generated synchronization information and the information receiving apparatus, the second media content in synchronization with reproduction of the first media content by the second information receiving apparatus.

2. The information receiving apparatus of claim 1, wherein the second media content and the synchronization feature are transmitted to the information processing apparatus at a time determined based on a difference between an arrival time of the first media content and an arrival time of the second media content and the synchronization feature.

3. The information receiving apparatus of claim 1, wherein the circuitry is configured to:

request transmission of the second media content and the synchronization feature; and

receive the second media content and the synchronization feature which are transmitted in response to the request.

4. The information receiving apparatus of claim 3, wherein the circuitry is configured to:

request transmission of the first media content; and

receive the first media content which is transmitted in response to the request for transmission of the first media content.

5. The information receiving apparatus of claim 1, wherein the circuitry is configured to:

extract a first plurality of features from the audio signal, the first plurality of features including the audio-derived feature;

obtain a second plurality of features together with the second media content, the second plurality of features including the synchronization feature;

compare the first plurality of features with the second plurality of features; and

generate, based on results of comparing the first plurality of features with the second plurality of features, the synchronization information used for reproducing the second media content in synchronization with the first media content.

6. The information receiving apparatus of claim 1, wherein the circuitry is configured to control reproduction of the second media content.

7. The information receiving apparatus of claim 6, wherein the circuitry is configured to correct a reproduction position of the second media content based on the synchronization information.

8. The information receiving apparatus of claim 1, wherein the circuitry is configured to:

acquire presentation time information associated with the first media content; and

obtain a range of time based on results of comparing the presentation time information associated with the first media content and presentation time information associated with the second media content,

wherein the circuitry is configured to generate the synchronization information at least in part by comparing the synchronization feature with one or more features extracted from one or more portions of the first media content appearing in the first media content in the range of time.

9. The information receiving apparatus of claim 1, wherein the circuitry is configured to compare the audio-derived feature and the synchronization feature after performing frame rate conversion on at least one of the audio-derived feature and the synchronization feature such that a frame rate of the audio-derived feature coincides with a frame rate of the synchronization feature.

10. The information receiving apparatus of claim 1, wherein the second media content includes the synchronization feature.

11. An information processing method for reproducing by an information receiving apparatus a second media content in synchronization with reproduction of a first media content, the first media content comprising audio content, the information processing method comprising:

extracting an audio-derived feature from an audio signal generated via reproduction of the audio content by a second information receiving apparatus;

obtaining, from an information providing apparatus different from the second information receiving apparatus, the second media content together with a synchronization feature, the synchronization feature being extracted from the audio content of the first media content prior to being transmitted by the information providing apparatus;

comparing the audio-derived feature with the synchronization feature;

generating, based on results of the comparing, synchronization information; and

reproducing, using the generated synchronization information and by the information receiving apparatus, the second media content in synchronization with reproduction of the first media content by the second information receiving apparatus.

12. At least one non-transitory computer readable storage medium storing processor-executable instructions that, when executed by at least one computer, causes the at least one computer to perform an information processing method for reproducing by an information receiving apparatus a second media content in synchronization with reproduction of a first media content, the first media content comprising audio content, the information processing method comprising:

extracting an audio-derived feature from an audio signal generated via reproduction of the audio content by a second information receiving apparatus;

obtaining, from an information providing apparatus different from the second information receiving apparatus, the second media content together with a synchronization feature, the synchronization feature being extracted from the audio content prior to the first media content being transmitted by the information providing apparatus;

comparing the audio-derived feature with the synchronization feature;

generating, based on results of the comparing, synchronization information; and

reproducing using the generated synchronization information and by the information receiving apparatus the second media content in synchronization with reproduction of the first media content by the second information receiving apparatus.

13. An information providing apparatus for transmitting a first media content and a second media content to one or more other devices, the first media content comprising audio content, the information providing apparatus comprising:

circuitry configured to;

extract a synchronization feature from the audio content of the first media content prior to transmission of the first media content to the one or more other devices;

associate, in synchronization with the first media content, the synchronization feature with the second media content that has a time-synchronized relationship with the first media content;

transmit the first media content to a first receiving device; and

transmit the second media content together with the synchronization feature to a second receiving device different from the first receiving device.

14. The information providing apparatus of claim 13, wherein the circuitry is configured to transmit the second media content and the synchronization feature at a time determined based on a difference between arrival time of the first media content and the second media content.

15. The information providing apparatus of claim 13, wherein the circuitry is configured to transmit the second media content and the synchronization feature in response to a request for their transmission.

16. The information providing apparatus of claim 15, wherein the circuitry is configured to transmit the first media content in response to a request for its transmission.

17. The information providing apparatus of claim 13, wherein the circuitry is configured to:

extract a plurality of features from a plurality of portions of the audio content of the first media content prior to transmission of the first media content to the one or more other devices, the plurality of features including the synchronization feature; and

associate and transmit the extracted features with the second media content.

18. The information providing apparatus of claim 13, wherein the circuitry is configured to:

downsample the synchronization feature to obtain a downsampled synchronization feature; and

transmit the second media content and the downsampled synchronization feature.

19. The information receiving apparatus of claim 13, wherein the second media content includes the audio-derived feature.

20. An information processing method for transmitting by an information providing apparatus a first media content and a second media content to one or more other devices, the first content comprising audio content, the method comprising:

extracting a synchronization feature from the audio content of the first media content prior to transmission of the first media content to the one or more other devices;

associating, in synchronization with the first media content, the synchronization feature with second media content that has a time-synchronized relationship with the first media content;

transmitting the first media content to a first receiving device; and

transmitting the second media content together with the synchronization feature to a second receiving device different from the first receiving device.

21. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer, causes the at least one computer to perform an information processing method for transmitting by an information providing apparatus a first media content and a second media content to one or more other devices, the first media content comprising audio content, the information processing method comprising:

extracting a synchronization feature from the audio content of the first media content prior to transmission of the first media content to the one or more other devices;

associating, in synchronization with the first media content, the synchronization feature with second media content that has a time-synchronized relationship with the first media content;

transmitting the first media content to a first receiving device; and

transmitting the second media content together with the synchronization feature to a second receiving device different from the first receiving device.