Audiovisual user interface based on learned user preferences

• US 8,566,855 B2
• Filed: 12/02/2008
• Issued: 10/22/2013
• Est. Priority Date: 12/02/2008
• Status: Active Grant

• Current Assignee: Saturn Licensing LLC (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Wong, Ling Jun, Carvajal, Victor, Chee, Jaime
• Primary Examiner(s): Srivastava, Vivek
• Assistant Examiner(s): Le, Rong
• Application Number:
  US12/326,681
• Publication Number:
  US 20100138867A1
• Time in Patent Office: 1,785 Days
• Field of Search: None
• US Class Current: 725/9
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/637 Administration of user profiles, e.g. generation, initialization, adaptation or distribution
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/735 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/466 Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4668 for recommending content, e.g. movies
• CPC Class Codes: H04N 21/47 End-user applications

Abstract

A method and apparatus for providing an audiovisual user interface based on learned user preferences is described. In one implementation, the method involves collecting a set of user preferences for a user; providing a metadata stream associated with an audiovisual input; comparing elements within the metadata stream with the set of user preferences to form a comparison; and outputting a message to the user if the comparison indicates that the metadata stream matches one or more elements of the set of user preferences. The message may be output either acoustically or visually, and may relate to the audiovisual input, which may be audio, video, or both audio and video. The audiovisual input may be blocked based on a restricted message, and may be accessed upon a successful password query.

Claims

1. A method of interacting with a user, comprising:

collecting a set of user preferences for a user into a user preference database;

providing a metadata stream associated with a program, comprising metadata for the program;

comparing the metadata stream with the set of user preferences to form a comparison; and

outputting a message to the user if the comparison indicates that the metadata stream matches one or more elements of the set of user preferences;

wherein collecting the set of user preferences for the user comprises determining a percentage of time the program was perceived by the user prior to changing to a different audiovisual input;

wherein the percentage of time the audiovisual input was perceived is calculated by dividing an amount of time that the audiovisual input was perceived by the user by a total length of time the program was available; and

wherein the message is a verbal recommendation; and

executing instructions on a computer for performing user preference acquisition comprising;

(a) looping until the program is played, then unsetting a wait flag; and

(b) if the program has been played for a percentage of time <

30%, then;

(i) setting the wait flag;

(ii) queuing program metadata as a 0-Star entry in a user preference database;

(iii) looping until the wait flag is not set;

(iv) posting the queued metadata to the user preference database; and

(v) returning to step (a);

(c) if the program has been played for a percentage of time <

70%, then;

(i) setting the wait flag;

(ii) queuing program metadata as a 3-Star entry in the user preference database;

(iii) looping until the wait flag is not set;

(iv) posting the queued metadata to the user preference database; and

(v) returning to step (a);

(d) assuming that the program has been played to completion, then;

(i) if the user does not want to rate the program, then queuing program metadata as a 5-Star entry in the user preference database;

(ii) if the user rates the program as disliked, then queuing program metadata as the O-Star entry in the user preference database;

(iii) if the user rates the program as partially liked, then queuing program metadata as the 3-Star entry in the user preference database;

(iv) if the user rates the program as liked, then queuing program metadata as the 5-Star entry in the user preference database;

(v) posting the queued metadata to the user preference database; and

(vi) returning to step (a).

2. The method of claim 1, wherein the percentage of time is determined by a perception by the user that was physically present.

3. The method of claim 2, wherein the perception by the user comprises being available for perception by the user, but not necessarily a physically present perception by the user.

4. The method of claim 2, wherein the percentage of time the program was perceived is maintained for one or more concurrently available audiovisual inputs, thereby providing for partial viewing of the inputs.

5. The method of claim 2, wherein the perception by the user comprises viewing of an active window in a potentially multi-windowed device.

6. The method of claim 5, wherein the active window in the potentially multi-windowed device is a single window.

7. The method of interacting with a user of claim 2, comprising:

using the percentage of time the program was perceived by the user to select a binned rating attribute in the user preference database; and

inputting the metadata associated with the program into the user preference database as associated with the binned rating attribute.

8. The method of interacting with a user of claim 7, comprising allowing the user to override the binned rating attribute of the user preferences.

9. The method of claim 2, further comprising:

loading a list of assets available for user perception,

wherein each of the assets has an independent metadata stream; and

comparing at least one of the metadata streams of the assets, with a set of preferences determined from the user preference database for the user, to create one or more user recommendations.

10. The method of claim 9, further comprising:

ordering the one or more user recommendations; and

forming the message regarding the program corresponding to the user recommendation as a restriction message.

11. The method of claim 10, further comprising blocking the program that corresponds to the restriction message, thereby forming a blocked input.

12. The method of claim 11, further comprising releasing the blocked input upon a successful user response to a password query.

13. The method of claim 1, wherein the audiovisual input program is selected from a group of audiovisual inputs consisting of:

an audio input, a video input, a video input with an associated audio input, an audio input with a corresponding video display, and one or more of the foregoing.

14. The method of claim 1, wherein the metadata stream comprises a closed caption data stream for the program.

15. The method of claim 1, wherein the metadata stream comprises a speech recognition output stream for an audio component of the program.

16. The method of claim 1, wherein the metadata stream comprises a video recognition output stream for a video component of the program.

17. A non-transitory computer readable medium comprising the instructions for performing the method of claim 1.

18. A user preference apparatus comprising the computer and instructions for performing the method of claim 1.

19. An audiovisual device, comprising:

an audio, video, or audiovisual output device capable of receiving and processing a respective audio, video, or audiovisual input signal, the input signal comprising a program;

a computer; and

instructions executable on the computer for performing steps comprising;

processing a metadata stream associated with the respective audio, video, or audiovisual input signal, the metadata stream comprising metadata for the input signal;

generating user preferences; and

outputting to a user, one or more messages based on one or more comparisons of the user preferences and metadata stream;

wherein the one or more messages are verbal recommendations; and

wherein the generating user preferences step comprises;

collecting a set of user preferences for a user into a user preference database;

wherein collecting the set of user preferences for the user comprises determining a percentage of time the audiovisual input was perceived by the user prior to changing to a different program; and

wherein the percentage of time the audiovisual input was perceived is calculated by dividing an amount of time that the audiovisual input was perceived by the user by a total length of time the audiovisual input was available; and

performing user preference acquisition steps on the computer, the acquisition steps comprising;

(a) looping until the program is played, then unsetting a wait flag; and

(b) if the program has been played for the percentage of time <

30%, then;

(i) setting the wait flag;

(ii) queuing program metadata as a O-Star entry in a user preference database;

(iii) Looping until the wait flag is not set;

(iv) posting the queued metadata to the user preference database; and

(v) returning to step (a);

(c) if the program has been played for a percentage of time <

70%, then;

(i) setting the wait flag;

(ii) queuing program metadata as a 3-Star entry in the user preference database;

(iii) looping until the wait flag is not set;

(iv) posting the queued metadata to the user preference database; and

(v) returning to step (a);

(d) assuming that the program has been played to completion, then;

(i) if the user does not want to rate the program, then queuing program metadata as a 5-Star entry in the user preference database;

(ii) if the user rates the program as disliked, then queuing program metadata as the 0-Star entry in the user preference database;

(iii) if the user rates the program as partially liked, then queuing program metadata as the 3-Star entry in the user preference database;

(iv) if the user rates the program as liked, then queuing program metadata as the 5-Star entry in the user preference database;

(v) posting the queued metadata to the user preference database; and

(vi) returning to step (a).

20. The audiovisual device of claim 19, further comprising instructions executable on the computer for providing a wandering widget that fades into existence when one or more messages are to be communicated to the user.

21. The audiovisual device of claim 19, further comprising instructions executable on the computer for blocking the respective audio, video, or audiovisual input signal based on one or more of the messages.

22. The audiovisual device of claim 21, further comprising instructions executable on the computer for selectively unblocking the blocked respective audio, video, or audiovisual input signal based on a successful password query.