Digital data reproducing apparatus and corresponding method for reproducing content based on user characteristics

• US 8,634,701 B2
• Filed: 12/03/2010
• Issued: 01/21/2014
• Est. Priority Date: 12/04/2009
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kang, Mingoo, Kang, Haengjoon, Park, Jongsoon, Park, Jinyung, Kim, Jongcheol, Park, Junho, Hwang, Sunjung
• Primary Examiner(s): HARVEY, DAVID E
• Application Number:
  US12/959,543
• Publication Number:
  US 20110142413A1
• Time in Patent Office: 1,145 Days
• Field of Search: 386/241, 386/248, 386/344, 348/51
• US Class Current: 386/248
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06F 16/437 Administration of user profiles, e.g. generation, initialisation, adaptation, distribution
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/32 on separate auxiliary tracks of the same or an auxiliary record carrier
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44213 : Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

An digital data reproducing apparatus and a method for controlling the same are discussed. The method includes obtaining, by the digital data reproducing apparatus, emotion information associated with at least one of a user and a content, modifying at least one of audio and video characteristics of a content for reproduction based on at least the emotion information, and reproducing the content based on the modified at least one of audio and video characteristics of the content.

Claims

1. A method for controlling a digital data reproducing apparatus, the method comprising:

obtaining, by the digital data reproducing apparatus, emotional information associated with at least one of a user and a content;

displaying an object indicating whether to reproduce a content according to the emotional information;

entering into an emotional information application mode based on a user'"'"'s selection;

modifying at least one of audio and video characteristics of the content for reproduction based on at least the emotional information, wherein the modifying step includes modifying a depth of a 3D content according to the emotional information; and

reproducing the content based on the modified at least one of audio and video characteristics of the content; and

exchanging the emotional information with other users watching the same content over a network;

wherein obtaining emotional information further includes;

capturing an image of at least a part of a user'"'"'s body;

recording the user'"'"'s voice;

sensing, via a body signal sensing unit in a remote control unit, physical attributes of the users to generate physical information of the user; and

determining the emotional information of the user based on at least two of the captured image, the recorded voice, or the sensed body signal.

2. The method of claim 1, wherein the captured image, the recorded voice and the body signal are assigned different weights thereby obtaining the user'"'"'s emotional information.

3. The method of claim 1, further comprising:

generating an icon or avatar representing emotional information of the user; and

displaying the icon or avatar together with the content.

4. The method of claim 1, wherein obtaining the emotional information further includes at least two of the following:

obtaining representative emotional information associated with at least one of the user and the content;

obtaining emotional information for each scene or frame of the content;

or

obtaining emotional information for each time period associated with the content.

5. The method of claim 4, further comprising:

receiving the user'"'"'s selection of the emotional information among the at least two of the representative information, the information for each scene or frame, and the emotional information for each time period,

wherein the modifying step modifies the at least one of audio and video characteristics of the content based on the user-selected information.

6. The method of claim 1, further comprising:

displaying an object indicating total run time of the content and an object indicating a current play position;

displaying objects representing at least two pieces of obtained emotional information, respectively, when at least two pieces of emotional information are obtained;

selecting one piece of emotional information according to a user'"'"'s selection; and

moving from a position in the content corresponding to the selected piece of emotional information and reproducing the content from the position,

wherein each object representing different emotional information has a different size or color.

7. The method of claim 1, further comprising:

obtaining, by the digital data reproducing apparatus, at least one of attentiveness information of the user and emotional intensity information associated with at least one of the user and the content,

wherein the modifying step modifies the at least one of audio and video characteristics of the content based on the emotional information and based on at least one of the attentiveness information and the emotional intensity information.

8. The method of claim 1, further comprising:

modifying an ambient lighting for the digital data reproducing apparatus based on the emotional information.

9. The method of claim 1, wherein the capturing step includes obtaining, by the digital data reproducing apparatus, user image information of a user using at least two cameras, the at two cameras including a 3-D depth camera and an RGB camera.

10. The method of claim 1, further comprising:

receiving, by the digital data reproducing apparatus from a transmitting apparatus, (a) seamless reproduction information associated with a content displayed at the transmitting apparatus, and (b) emotional information associated with the content; and

seamlessly reproducing, by the digital data reproducing apparatus, the content based on the seamless reproduction information and based on the modified at least one of audio and video characteristics of the content.

11. The method of claim 10, wherein the seamless reproduction information includes time stamp information associated with the content being displayed, when an event for requesting the seamless reproduction at the digital data reproducing apparatus occurs.

12. The method of claim 1, further comprising:

collecting emotional information of the respective users watching the same content; and

determining the representative emotional state of the users for the same content based on the collected emotional information.

13. The method of claim 1, further comprising:

displaying objects representing at least two pieces of emotional information, respectively, when at least two pieces of emotional information are obtained; and

selecting one piece of emotional information according to a user'"'"'s selection,

wherein the modifying step modifies at least one of audio and video characteristics of the content based on the selected emotional information,

wherein each object representing different emotional information has a different size or color.

14. A digital data reproducing apparatus, comprising:

an image capturing unit configured to capture a video of a user to generate user image information;

an audio recording unit configured to record audio data of the user to generate voice information of the user;

a body signal sensing unit in a remote control unit, and configured to sense physical attributes of the user to generate physical information of the user;

a network interface unit configured to exchange data with another electronic device or an external network server;

at least one of a display unit and an audio output unit; and

a controller configured to;

obtain emotional information associated with at least one of the user and a content;

display an object indicating whether to reproduce a content according to the emotional information;

enter into an emotional information application mode based on a user'"'"'s selection;

modify at least one of audio and video characteristics of a content for reproduction based on at least the emotional information;

reproduce the content, using the at least one of the display unit and the audio output unit, based on the modified at least one of audio and video characteristics of the content; and

determine the emotional information of the user based on at least two of the captured image, the recorded voice, and the sensed body signal;

wherein the controller controls the network interface unit to exchange the emotional information with other users watching the same content over a network; and

wherein the controller modifies a depth of a 3D content according to the emotional information, when modifying the video characteristics of the content.

15. The apparatus of claim 14, wherein the captured image, the recorded voice and the body signal are assigned different weights thereby obtaining the user'"'"'s emotional information.

16. The apparatus of claim 14, wherein the controller generates an icon or avatar representing emotional information of the user, and controls the display to display the icon or avatar together with the content.

17. The apparatus of claim 14, wherein the controller is further configured to obtain at least two of the following:

representative emotional information associated with at least one of the user and the content;

emotional information for each scene or frame of the content;

or

emotional information for each time period associated with the content.

18. The apparatus of claim 17, wherein the controller is further configured to receive a user'"'"'s selection among the at least two of the representative emotional information, the emotional information for each scene or frame, and emotional information for each time period, and to modify the at least one of audio and video characteristics of the content based on the user-selected emotion information.

19. The apparatus of claim 14, wherein the controller controls the display unit to display an object indicating total run time of the content and an object indicating a current play position and the controller displays objects representing the at least two pieces of emotional information, respectively, when at least two pieces of emotion information are obtained;

wherein the controller selects one emotion information according to a user'"'"'s selection, and moves from a position in the content corresponding to the selected emotion and reproduces the content from the position; and

wherein each object representing different emotional information has a different size or color.

20. The apparatus of claim 14, wherein the controller is further configured to obtain at least one of attentiveness information of the user and emotional intensity information associated with at least one of the user and the content, and to modify the at least one of the audio and video characteristics of the content based on the emotional information and based on the attentiveness information and the emotional intensity information.

21. The apparatus of claim 14, wherein the controller is further configured to modify ambient lighting for the digital data reproducing apparatus based on the emotional information.

22. The apparatus of claim 15, wherein the controller collects the emotional information of the respective users watching the same content, and determines the representative emotional state of the users for the same content based on the collected emotional information.

23. The apparatus of claim 14, wherein the controller controls the display unit to display objects representing at least two pieces of emotional information, respectively, when at least two pieces of emotional information are obtained, selects one piece of emotional information according to a user'"'"'s selection, and modifies at least one of audio and video characteristics of the content based on the selected emotional information;

and

wherein each object representing different emotional information has a different size or color.