Social system and method used for bringing virtual social network into real life

• US 9,329,677 B2
• Filed: 03/15/2012
• Issued: 05/03/2016
• Est. Priority Date: 12/29/2011
• Status: Active Grant

• Current Assignee: National Taiwan University (Government of The Republic of China)
• Original Assignee: National Taiwan University (Government of The Republic of China)
• Inventors: Chien, Shao-Yi, Lai, Jui-Hsin, Lin, Jhe-Yi, Su, Min-Yian, Wu, Po-Chen, Kao, Chieh-Chi
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Nguyen, Vu
• Application Number:
  US13/420,966
• Publication Number:
  US 20130169680A1
• Time in Patent Office: 1,510 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

The present invention is related to a social system and process used for bringing virtual social network into real life, which is allowed for gathering and analyzing a social message of at least one interlocutor from virtual social network so as to generate at least one recommended topic, allowing a user to talk with the interlocutor through the utilization of the recommended topic, and then capturing and analyzing at least one speech and behavior and/or physiological response of the interlocutor during talking so as to generate an emotion state of the interlocutor. The user is allowed to determine whether the interlocutor is interested in the recommended topic through the emotion state of interlocutor. Thus, it is possible to bring the social message on virtual network into real life, so as to increase communication topics between persons in real life.

Claims

1. A social system used for bringing virtual social network into real life comprising:

an electronic eye-glass unit wearable on a user'"'"'s face, the electronic eye-glass unit including a processor and a capture module capturing an image, speech, facial expression, and body movement of at least one interlocutor, and an image of surroundings, in a real scene;

a social message processor portion identifying the at least one interlocutor according to the captured image, gathering a pre-configured social profile of the at least one interlocutor from a virtual social network based on the identification, and analyzing said social profile so as to generate at least one recommended conversation topic, said at least one recommended conversation topic being generated according to the analyzed social profile;

an augmented reality processor portion including a scene recognition unit generating a scene recognition result indicating a context of the interlocutor'"'"'s real scene, said scene recognition result generated at least in part according to the captured image of the surroundings of the real scene, said social message processor portion determining a suitability of said at least one recommended conversation topic adaptive to the scene recognition result, the selective generation of said at least one recommended conversation topic being based on said suitability determined therefor; and

an emotion analysis processor portion, analyzing said speech, facial expression, and body movement of the at least one interlocutor so as to determine an emotion state of the at least one interlocutor,

wherein said at least one recommended conversation topic and said emotion state are provided to the user, and said social message processor portion generates a different conversation topic upon determination of inconsistency of said at least one recommended conversation topic with said emotion state of said interlocutor.

2. The social system according to claim 1, wherein information corresponding to said at least one recommended conversation topic and said emotion state of the at least one interlocutor are displayed on an image window.

3. The social system according to claim 2, further comprising a camera, said camera capturing the real scene in which the user talks with the at least one interlocutor, a video content of said real scene being displayed on said image window.

4. The social system according to claim 3, wherein said augmented reality processor portion further includes an unsalient region recognition processor portion, said unsalient region recognition processor portion recognizing at least one unsalient region from said real scene, said unsalient region remaining visually offset from views of either the interlocutor or complex image contours, said augmented reality processor portion adaptively positioning the information corresponding to said at least one recommended conversation topic and said emotion state in said at least one unsalient region.

5. The social system according to claim 4, wherein the information corresponding to said at least one recommended conversation topic and said emotion state are drawn in said at least one unsalient region via an object type.

6. The social system according to claim 5, wherein said augmented reality processor portion further comprises:

a photorealistic rendering processor portion performing a photorealistic rendering of the information corresponding to said at least one recommended conversation topic and said emotion state of the object type drawn in said at least one unsalient region; and

a stable rendering processor portion rendering the information corresponding to said at least one recommended conversation topic and said emotion state of the object type in an appropriate place of said at least one unsalient region in stabilized manner, wherein when said at least one unsalient region within said real scene is moved as the shooting angle changes, said at least one recommended conversation topic and said emotion state of the object type are adjusted in placement by said stable rendering processor portion correspondingly.

7. The social system according to claim 3, wherein said scene recognition unit, in order to generate the scene recognition result, recognizes and analyzes at least one of:

the content of said real scene, location of said real scene, and shooting time of said real scene.

8. The social system according to claim 1, wherein said capture module comprises a face recognition processor portion, said face recognition processor portion executing to recognize the face feature of the at least one interlocutor, and then to compare the recognized face feature of the at least one interlocutor with at least one personal picture to which each identity account on the virtual social network corresponds, so as to ascertain an identity of the at least one interlocutor.

9. A social system used for bringing virtual social network into real life comprising:

an electronic eye-glass unit wearable on a user'"'"'s face, the electronic eye-glass unit including a processor and a capture module capturing an image and behavior of at least one interlocutor and an image of surroundings;

a camera module capturing a real scene of the at least one interlocutor;

a social message processor portion, identifying the at least one interlocutor according to the captured image, gathering, based on the identification, and analyzing a pre-configured social profile of at least one interlocutor from a virtual social network so as to generate at least one recommended conversation topic, said at least one recommended conversation topic being generated according to the analyzed social profile;

a physiology capture sensor operably coupled to the processor, capturing at least one physiological response of the at least one interlocutor;

an emotion analysis processor portion analyzing said at least one physiological response of the at least one interlocutor so as to determine an emotion state of the at least one interlocutor; and

an augmented reality processor portion including an unsalient region recognition unit selectively determining at least one unsalient region within the real scene, said unsalient region being variably located and visually offset from views of either the interlocutor or complex image contours, said augmented reality processor portion adaptively positioning information corresponding to the at least one recommended conversation topic and emotion state in the at least one unsalient region for concurrent unobstructive display with the interlocutor, and said social message processor portion generating a different conversation topic upon determination of inconsistency of said at least one recommended conversation topic with said emotion state of said interlocutor.

10. The social system according to claim 9, wherein the information corresponding to said at least one recommended conversation topic and said emotion state of the at least one interlocutor are displayed on an image window.

11. The social system according to claim 10, wherein a video content of said real scene is displayed on said image window.

12. The social system according to claim 11, wherein said augmented reality processor portion further includes:

a scene recognition processor portion, said scene recognition processor portion being used to recognize and analyze at least one of;

a content of said real scene, a location of said real scene, and a shooting time of said real scene so as to generate a scene recognition result; and

,

said social profile of the at least one interlocutor and said scene recognition result being analyzed by said social message processor portion so as to generate said at least one recommended conversation topic.

13. The social system according to claim 9, wherein said physiology capture sensor includes at least one of a body temperature sensor, a blood pressure monitor, a breath monitor, a brain waves inspector, a pupil detector and an equipment capable of inspecting the physiological response of a body.

14. The social system according to claim 9, further comprising a capture module having a face recognition processor portion, said face recognition unit recognizing a facial feature of the at least one interlocutor, and then comparing the recognized facial feature of the at least one interlocutor with at least one personal picture to which each identity account on the virtual social network corresponds, to ascertain an identity of the at least one interlocutor.

15. The social system according to claim 9, wherein the information corresponding to said at least one recommended conversation topic and said emotion state are drawn in said at least one unsalient region via the means of an object type.

16. The social system according to claim 15, wherein said augmented reality processor module further comprises:

a photorealistic rendering processor portion performing a photorealistic rendering of the information corresponding to said at least one recommended conversation topic and said emotion state of the object type drawn in said at least one unsalient region; and

a stable rendering processor portion rendering the information corresponding to said at least one recommended conversation topic and said emotion state of the object type in an appropriate place of said at least one unsalient region in stabilized manner;

wherein when said at least one unsalient region of said real scene is moved as the shooting angle changes, the information corresponding to said at least one recommended conversation topic and said emotion state of the object type are adjusted in placement by said stable rendering processor portion correspondingly.

17. A social method used for bringing virtual social network into real life comprising:

establishing an electronic eye-glass unit on a user'"'"'s face, the electronic eye-glass unit including a processor and a capture module;

executing the processor to capture an image of at least one interlocutor and surroundings in a real scene, identify the interlocutor according to the captured image, gather a pre-configured social profile of the at least one interlocutor from a virtual social network based on the identification, and analyze said social profile to generate at least one recommended conversation topic for conversation, said at least one recommended conversation topic being generated according to the analyzed social profile;

executing a scene recognition unit within an augmented reality processor portion to generate a scene recognition result indicating a context of the interlocutor'"'"'s real scene, said scene recognition result generated at least in part according to the captured image of the surroundings;

determining a suitability of said at least one recommended conversation topic adaptive to the scene recognition result, the selective generation of said at least one recommended conversation topic being based on said suitability determined therefor;

enabling a user to talk with the interlocutor through the utilization of said at least one recommended conversation topic;

capturing at least one of speech, facial expression, body movement, or physiological response of the at least one interlocutor;

analyzing said speech, facial expression, body movement, or physiological response of the interlocutor so as to determine an emotion state of the interlocutor;

indicating to the user said at least one recommended conversation topic and said emotion state of the at least one interlocutor; and

,

executing said processor to generate a different conversation topic upon determination of inconsistency of said at least one recommended conversation topic with said emotion state of said interlocutor.

18. The social method according to claim 17, further comprising:

recognizing either a content of said real scene, or a location of said real scene, or a shooting time of said real scene to generate said scene recognition result.

19. The social method according to claim 17, further comprising:

indicating to the user, during a conversation with the at least one interlocutor, a different conversation topic upon determination of said emotion state of said interlocutor being either one of boredom, lifelessness, or expressionlessness.