Systems and methods for automatically enabling subtitles based on detecting an accent

• US 10,182,266 B2
• Filed: 11/17/2017
• Issued: 01/15/2019
• Est. Priority Date: 01/30/2017
• Status: Active Grant

• Current Assignee: ROVI Product Corporation (Xperi Inc.)
• Original Assignee: Rovi Guides, Inc. (Adeia Inc.)
• Inventors: Panchaksharaiah, Vishwas Sharadanagar, Gupta, Vikram Makam
• Primary Examiner(s): Saltarelli, Dominic D
• Application Number:
  US15/816,602
• Publication Number:
  US 20180220195A1
• Time in Patent Office: 424 Days
• Field of Search: 725 34, 704200, 704231, 704236, 704246, 704250
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44226 on social networks
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

Systems and methods are described for automatically enabling subtitles based on a user profile when a language is spoken with an accent a user has difficulty understanding. For example, a media guidance application may detect a first plurality of user interactions of the user while the given language is being spoken with the accent. Based on the first plurality of interactions, the media guidance application may calculate a first value associated with a user specific level of difficulty indicating how difficult it is for the user to understand the language when spoken with the accent. If the first plurality of user interactions are not being performed again, the media guidance application may update the user specific difficulty with a second value that is lower than the first value. The media guidance application may automatically generate for display subtitles for a media asset based on the user specific level of difficulty.

Claims

1. A method for automatically displaying explanatory messages based on a user profile when a language is spoken with an accent in a media asset, the method comprising:

storing, in a user profile, a first data structure indicating a list of one or more languages that the user understands;

determining, at a first point in time, that a language is being spoken in the media asset with an accent by;

performing natural language processing to detect an audio signature of the language;

determining, based on the audio signature, that the language is being spoken with an accent;

identifying the language spoken in the media asset; and

determining that the identified language spoken in the media asset matches one of the languages in the list of one or more languages that the user understands;

receiving, from a remote source, information needed to populate the explanatory messages;

extracting from the user profile a user specific level of difficulty being indicative of a measure of difficulty the user has in understanding the language when spoken with an accent; and

in response to determining that the language is being spoken in the media asset with the accent, automatically generating for display explanatory messages for the media asset based on the user specific level of difficulty.

2. The method of claim 1, wherein the information needed to populate the explanatory messages is received based on searching a dictionary of dialects.

3. The method of claim 1, wherein the information needed to populate the explanatory messages is received based on searching the Internet for an indication of alternate words for, or definitions of, a word.

4. The method of claim 1, wherein the user specific level of difficulty is determined by detecting a plurality of user interactions of the user while the language is being spoken with the accent.

5. The method of claim 4, further comprising:

retrieving, from the remote source, an information table associating user interactions with levels of difficulty being indicative of a measure of difficulty a plurality of users have in understanding accents;

determining a plurality of values by associating each user interaction of the plurality of user interactions with a value and;

determining the user specific level of difficulty based on the plurality of values.

6. The method of claim 5, wherein determining the user specific level of difficulty based on the plurality of values comprises one of calculation a sum of the plurality of values, calculating an average of the plurality of values, and multiplying the sum of the plurality of values by the number of values in the plurality of values.

7. The method of claim 4, further comprising:

detecting that the language is being spoken with the accent at a later point in time;

monitoring user interactions of the user while the language is being spoken with the accent at the later point in time; and

updating, based on determining that the first plurality of user interactions are not being performed again, a second data structure, the second data structure associating a second value that is lower than the first value with the user specific level of difficulty.

8. The method of claim 1, wherein detecting that a media asset includes the language spoken with the accent includes determining an accent level, the accent level indicating how slight the accent is, the method further comprising:

comparing the accent level to the user specific level of difficulty to determine if the user specific level of difficulty is greater than the accent level; and

wherein automatically generating for display explanatory messages for the media asset is further based on determining that the user specific level of difficulty is greater than the accent level.

9. The method of claim 1, wherein detecting that a media asset includes the language spoken with the accent comprises:

receiving, from a media source, the media asset and information indicating a first portion of the media asset during which a specific actor appears;

identifying an accent level associated with the specific actor; and

wherein automatically generating for display explanatory messages for the media asset comprises automatically generating for display explanatory messages for the media asset when the user specific level of difficulty exceeds the accent level.

10. The method of claim 1, wherein receiving, from the remote source, information needed to populate the explanatory messages comprises:

generating, at the media device, a search query for the language;

transmitting the search query to the remote source; and

receiving, from the remote source, as a response to the search query, information needed to populate the explanatory messages.

11. A system for automatically displaying explanatory messages based on a user profile when a language is spoken with an accent in a media asset, the system comprising:

control circuitry configured to;

store, in a user profile, a first data structure indicating a list of one or more languages that the user understands;

determine, at a first point in time, that a language is being spoken in the media asset with an accent by;

performing natural language processing to detect an audio signature of the language;

determining, based on the audio signature, that the language is being spoken with an accent;

identifying the language spoken in the media asset; and

determining that the identified language spoken in the media asset matches one of the languages in the list of one or more languages that the user understands;

receive, from a remote source, information needed to populate the explanatory messages;

extract from the user profile a user specific level of difficulty being indicative of a measure of difficulty the user has in understanding the language when spoken with an accent; and

in response to determining that the language is being spoken in the media asset with the accent, automatically generate for display explanatory messages for the media asset based on the user specific level of difficulty.

12. The system of claim 11, wherein the information needed to populate the explanatory messages is received based on searching a dictionary of dialects.

13. The system of claim 11, wherein the information needed to populate the explanatory messages is received based on searching the Internet for an indication of alternate words for, or definitions of, a word.

14. The system of claim 11, wherein the user specific level of difficulty is determined by detecting a plurality of user interactions of the user while the language is being spoken with the accent.

15. The system of claim 14, wherein the control circuitry is further configured to:

retrieve, from the remote source, an information table associating user interactions with levels of difficulty being indicative of a measure of difficulty a plurality of users have in understanding accents;

determine a plurality of values by associating each user interaction of the plurality of user interactions with a value and;

determine the user specific level of difficulty based on the plurality of values.

16. The system of claim 15, wherein the control circuitry is configured to determine the user specific level of difficulty based on the plurality of values by one of calculation a sum of the plurality of values, calculating an average of the plurality of values, and multiplying the sum of the plurality of values by the number of values in the plurality of values.

17. The system of claim 14, wherein the control circuitry is further configured to:

detect that the language is being spoken with the accent at a later point in time;

monitor user interactions of the user while the language is being spoken with the accent at the later point in time; and

update, based on determining that the first plurality of user interactions are not being performed again, a second data structure, the second data structure associating a second value that is lower than the first value with the user specific level of difficulty.

18. The system of claim 11, wherein:

the control circuitry configured to detect that a media asset includes the language spoken with the accent is further configured to determine an accent level, the accent level indicating how slight the accent is;

the control circuitry is further configured to compare the accent level to the user specific level of difficulty to determine if the user specific level of difficulty is greater than the accent level; and

the control circuitry configured to automatically generate for display explanatory messages for the media asset is further configured to generate for display explanatory messages based on determining that the user specific level of difficulty is greater than the accent level.

19. The system of claim 11, wherein:

the control circuitry configured to detect that a media asset includes the language spoken with the accent is configured to;

receive, from a media source, the media asset and information indicating a first portion of the media asset during which a specific actor appears;

identify an accent level associated with the specific actor; and

the control circuitry configured to automatically generate for display explanatory messages for the media asset is further configured to automatically generate for display explanatory messages for the media asset when the user specific level of difficulty exceeds the accent level.

20. The system of claim 11, wherein the control circuitry configured to receive, from the remote source, information needed to populate the explanatory messages is further configured to:

generate, at the media device, a search query for the language;

transmit the search query to the remote source; and

receive, from the remote source, as a response to the search query, information needed to populate the explanatory messages.