Method and system for intelligently mining data during communication streams to present context-sensitive advertisements using background substitution

• US 20120011454A1
• Filed: 04/30/2009
• Published: 01/12/2012
• Est. Priority Date: 04/30/2008
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Droz, Timothy, Acharya, Sunil, Bamji, Cyrus
• Application Number:
  US12/387,438
• Publication Number:
  US 20120011454A1
• Time in Patent Office: Days
• US Class Current: 715/758
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation

Abstract

The present invention mines or extracts data present during interaction between at least two participants, for example in a chat session, a video session, etc. via the Internet. The data, which can include participant web camera generated video, audio, keyboard typed information, handwriting recognized information, is analyzed. Based upon the analysis, content-dependent information is determined and may be displayed to one or more participants in the chat session. In one aspect, a video foreground based upon a participant'"'"'s generated video is combined with a customized computer generated background that is based upon data mined from the chat session. The customized background preferably is melded seamlessly with the participant'"'"'s foreground data, preferably via background substitution that combines RGB video with depth data that predicts what background may substituted with new imagery. Content-based targeted information can include advertisement(s).

Claims

1. A method to create at least one targeted content during a communications chat session between at least a first participant and a second participant, said first participant creating a first data stream that captures at least one characteristic of said first participant, the method including the following steps:

(a) extracting from said first data stream information to create a first representation of said at least one characteristic of said first participant;

(b) generating a first response appropriate to said at least one characteristic of said first participant, and

(c) communicating said first response to at least one of said first and said second participant,

wherein said first response includes at least one content targeted to be responsive to information obtained from said first data stream during said communications chat session.

2. The method of claim 1, wherein said first data stream includes at least one form of data selected from a group consisting of (i) a still video image of said first participant, (ii) a live dynamic video image of said first participant, (iii) an avatar created by said first participant, (iv) a sound made by said first participant during said communications chat session, (v) at least one word enunciated by said first participant during said communications chat session, (vi) at least one keyboard stroke entered by said first participant during said communications chat session, (vii) handwriting generated by said first participant during said communications chat session, and (viii) at least one video captured gesture made by said first participant during said communications chat session.

3. The method of claim 1, wherein said communications chat session includes a video chat session, said first data stream includes a first video stream, and said content targeted to be responsive includes at least one advertisement targeted to data mined during said video chat session.

4. The method of claim 1, wherein said communications chat session includes a video chat session in which at least part of said first data stream is captured in a manner selected from at least one of (i) using at least one camera system from which depth information is ascertainable, and (ii) using at least one time-of-flight camera system from which depth information is ascertainable.

5. The method of claim 3, wherein portions of said first video stream are captured using a RGB camera and a depth camera, and

step (a) includes using background substitution to extract from said first video stream said first foreground in said first scene;

and

step (c) includes displaying said first foreground on at least one display viewable by at least one of said first and said second participant, said at least one display showing said first foreground superimposed on a computer controlled first background, said first background generatable even in absence of a video stream from said first participant;

wherein said computer controlled first background includes at least one content responsive to information obtained from said first video stream during said video chat session.

6. The method of claim 3, wherein communicating at step (c) includes at least one feature selected from a group consisting of (i) communicating said first response to at least one display remote from said first participant, (ii) communicating from said first video stream an extracted first foreground for viewing by at least a third participant in said chat session, (iii) communicating at least said first response via an Internet, (iv) communicating at least said first response via a network, (v) communicating at least said first response wirelessly, and (v) communicating said first response in a domain differing from an acquisition of said first response.

7. The method of claim 5, wherein at least said first foreground has at least one characteristic selected from a group consisting of (i) a transparency level greater than 0%, (ii) said first foreground includes at least one aspect identified by at least one participant in said chat session, (iii) said first foreground is representative of a customer support function, and (iv) a display of said first foreground is scalable independently of size of said computer controlled first background.

8. The method of claim 5, wherein said computer controlled background includes at least one of (i) an existing display less said first foreground, (ii) a document being presented by one of said first participant and said second participant, (iii) a virtual whiteboard used by one of said first participant and said second participant to create at least one visual image, (iv) a displayed advertisement including at least one participant-operable virtual selection button;

(v) a static HTML page, (vi) a dynamic HTML page, (vii) a video stream, (viii) a database including at least one image, (ix) a database including social networking information, (x) a computer controlled background that is updatable in real-time responsive to at least one content of said chat session, (xi) said computer controlled background includes information of events occurring substantially contemporaneously with said chat session (xii) said computer controlled background includes information regarding at least one participant in said video chat, and (xxi) said computer controlled background is branded to display a brand of at least one of (I) a service provider, (II) an application provider, (III) a content provider, in which a branded said display is one of a static display and a dynamic display.

17. The system of claim 1, wherein at least one of said means for extracting, said means for generating, and said means for communicating is implemented using at least one of (i) hardware, and (ii) executable software.

9. A system to create at least one targeted content during a communications chat session between at least a first participant and a second participant, said first participant creating a first data stream that captures at least one characteristic of said first participant, said system including:

means for extracting from said first data stream information to create a first representation of said at least one characteristic of said first participant;

means for generating a first response appropriate to said at least one characteristic of said first participant, and

means for communicating said first response to at least one of said first and said second participant;

wherein said first response includes at least one content targeted to be responsive to information obtained from said first data stream during said communications chat session.

10. The system of claim 9, wherein said first data stream includes at least one form of data selected from a group consisting of (i) a still video image of said first participant, (ii) a live dynamic video image of said first participant, (iii) an avatar created by said first participant, (iv) a sound made by said first participant during said communications chat session, (v) at least one word enunciated by said first participant during said communications chat session, (vi) at least one keyboard stroke entered by said first participant during said communications chat session, (vii) handwriting generated by said first participant during said communications chat session, and (viii) at least one video captured gesture made by said first participant during said communications chat session.

11. The system of claim 9, wherein said communications chat session includes a video chat session, and said content targeted to be responsive includes at least one advertisement targeted to data mined during said video chat session.

12. The system of claim 9, wherein said communications chat session includes a video chat session in which at least part of said first data stream is captured in a manner selected from at least one of (i) using at least one camera system from which depth information is ascertainable, and (ii) using at least one time-of-flight camera system from which depth information is ascertainable.

13. The system of claim 10, wherein said first data stream includes a first video stream wherein portions of said first video stream are captured using a RGB camera and a depth camera, and

said means for extracting uses background substitution to extract from said first video stream a first foreground in said first scene;

and

said means for communicating includes displaying said first foreground on at least one display viewable by at least one of said first and said second participant, said at least one display showing said first foreground superimposed on a computer controlled first background, said first background generatable even in absence of a video stream from said first participant;

wherein said computer controlled first background includes at least one content responsive to information obtained from said first video stream during said video chat session.

14. The system of claim 9, wherein said means for communicating carries out least one feature selected from a group consisting of (i) communicating said first response to at least one display remote from said first participant, (ii) communicating said first foreground for viewing by at least a third participant in said chat session, (iii) communicating at least said first response via an Internet, (iv) communicating at least said first response via a network, (v) communicating at least said first response wirelessly, and (vi) communicating said first response in a domain differing from an acquisition of said first response.

15. The method of claim 13, wherein at least said first foreground has at least one characteristic selected from a group consisting of (i) a transparency level greater than 0%, (ii) said first foreground includes at least one aspect identified by at least one participant in said chat session, (iii) said first foreground is representative of a customer support function, and (iv) a display of said first foreground is scalable independently of size of said computer controlled first background.

16. The system of claim 13, wherein said computer controlled background includes at least one of (i) an existing display absent said first foreground, (ii) a document being presented by one of said first participant and said second participant, (iii) a virtual whiteboard used by one of said first participant and said second participant to create at least one visual image, (iv) a displayed advertisement including at least one participant-operable virtual selection button;

(v) a static HTML page, (vi) a dynamic HTML page, (vii) a video stream, (viii) a database including at least one image, (ix) a database including social networking information, (x) a computer controlled background that is updatable in real-time responsive to at least one content of said chat session, (xi) said computer controlled background includes information of events occurring substantially contemporaneously with said chat session (xii) said computer controlled background includes information regarding at least one participant in said video chat, and (xxi) said computer controlled background is branded to display a brand of at least one of (I) a service provider, (II) an application provider, (III) a content provider, in which a branded said display is one of a static display and a dynamic display.

18. A method to present an image that represents a participant in a video communications chat session that occurs between at least a first participant and a second participant, creating for said first participant using a three-dimensional camera system a first data stream that captures at least one video-derived characteristic of an imaged scene including said first participant, the method including the following steps:

(a) extracting from said first data stream a foreground image from said scene representing said first participant; and

(b) presenting the extracted said foreground image on a display of said second user, wherein said second user views said foreground image against a background this is a desktop for said second user.

19. The method of claim 18, wherein said foreground is less than 100% opaque.

20. The method of claim 18, wherein said foreground image is scalable relative to the desktop and includes at least one of (i) an actual image of said first participant, and (ii) an avatar representing said first participant.