Apparatus and method for multimedia content based manipulation

• US 7,714,878 B2
• Filed: 08/09/2004
• Issued: 05/11/2010
• Est. Priority Date: 08/09/2004
• Status: Active Grant

• Current Assignee: Nice Ltd
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Shermister, Shai, Gabay, Shay, Levi, Dafna, Sitbon, Doron
• Primary Examiner(s): Amini; Javid A
• Application Number:
  US10/507,582
• Publication Number:
  US 20060028488A1
• Time in Patent Office: 2,101 Days
• Field of Search: 345/626, 700/94
• US Class Current: 345/626
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/564 Enhancement of application control based on intercepted application data
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/845 Structuring of content, e.g. decomposing content into time segments

Abstract

An apparatus and method for multimedia content based manipulation. The method comprises the masking at least one part of the segment of an interaction, said masking can include the hiding, blurring, or alternatively, the enhancing or focusing on a region of interest.

Claims

1. A method for restricting access to call center multimedia information comprising an at least one audio and screen stream, the method comprising:

recording a first part of the at least one audio stream and a first part of the at least one screen stream comprising a screen event in a data format structure by a multimedia capture device, to obtain at least one recorded audio stream and at least one recorded screen stream, the at least one audio stream and at least one screen stream representing at least one interaction between a customer and an agent of the call center;

automatically marking locations in the at least one recorded audio stream and at least one recorded screen stream, the locations associated with an event, metadata, or content user data;

automatically masking at least two second parts of the at least one recorded audio stream and the at least one recorded screen stream, substantially at the locations, the at least two second parts containing a personal detail or a credit card detail or a medical detail of the customer; and

presenting to a user the at least one recorded audio stream or the at least one recorded screen stream as masked,

wherein said multimedia capture device is selected from the group consisting of;

a disk device, a tape device, a DAT device, a telephone, a cellular phone, a satellite phone, a wireless device, a microphone, a RF receiving device, a VOIP device, a facsimile machine, a computing device in which a session of data communications occurs, and any combination thereof.

2. The method of claim 1 further comprising obtaining metadata associated with the at least one interaction from a content input device.

3. The method of claim 2 further comprising masking the metadata obtained from the content input device and associated with the at least one interaction.

4. The method of claim 1 wherein the screen comprises any one of the following:

a display device, a computer screen, a video screen, a television screen, or a portable device screen.

5. The method of claim 4 wherein restricting access refers to restricting access to an event recorded from a screen.

6. The method for claim 1 wherein the masking is altered based on at least one user request.

7. The method of claim 1 further comprising analyzing the obtained metadata or the stream associated with the at least one interaction.

8. The method for claim 1 further comprising associating the metadata associated with the at least one interaction with the at least second part of the at least one recorded audio stream and screen stream.

9. The method of claim 1 further comprising selecting an at least one content masking to be performed on the at least one interaction to be masked.

10. The method of claim 1 further comprising altering at least a part of the at least second part of the at least one recorded, audio and screen stream.

11. The method of claim 1 further comprising inserting an at least one content masking element into the at least second part of the at least one recorded audio and screen stream.

12. The method of claim 1 wherein the masking comprises generating content masking from the at least one recorded audio and screen stream and associated with the at least one interaction.

13. The method of claim 1 wherein the masking comprises generating content masking from a first masked content and a second masked content, each said masked content is associated with an at least one interaction.

14. The method of claim 1 wherein the step of masking is associated with at least one previous masking or on a previously performed action by the user or an element of the apparatus of the present invention.

15. The method of claim 1 wherein the content is multimedia content.

16. The method of claim 1 wherein the content user data is multimedia content user data.

17. The method of claim 1 wherein masking is one of the following:

hiding of information;

replacing of information;

blurring of images or information;

preventing recording of information;

denying access;

enhancing a region of interest;

improving visibility of an image or information;

encrypting of information;

manipulation of multimedia content based on actions associated with rules.

18. The method of claim 1 wherein restricting access to the audio stream and screen stream comprises masking the at least one audio and screen stream except a first portion thereof, or masking only a second portion of the audio and screen streams.

19. The method of claim 1 wherein the metadata is Computer Telephony Interface data and wherein the Computer Telephony Interface data is captured by an external capture device.

20. The method of claim 19 wherein the interaction is between a customer and a machine or a computer application.

21. The method of claim 1 wherein the content user data is a product of analysis.

22. The method of claim 21 wherein the analysis is selected from the group consisting of:

audio analysis;

emotion analysis;

gender detection;

language detection; and

key stroke analysis.

23. The method of claim 1 wherein the machine is a computer.

24. The method of claim 1 wherein the personal detail or the credit card detail or the medical detail is identified according to a template.

25. The method of claim 24 wherein the template represents a credit card number.

26. The method of claim 1 wherein marking locations in the at least one recorded audio stream and at least one recorded screen stream is done by embedding information in the at least one recorded audio stream or the at least one recorded screen stream.

27. The method of claim 1 further comprising a step of storing the at least one recorded audio stream and the at least one recorded screen stream in masked/unmasked multimedia storage, after masking.

28. The method of claim 1, wherein said session of data communications is selected from the group consisting of email, a chat session, a keystroke, a use of a virtual keyboard or pen, touch screen use, mouse use, peripheral use, an interaction with a computer, and any combination thereof.

29. An apparatus for restricting access to call center multimedia information comprising an at least one audio stream and at least one screen stream, the stream comprising at least one interaction between a client and either an agent of the call center or a machine, the apparatus comprising:

a physical access device;

a multimedia capture device for capturing the at least one audio stream and at least one screen stream;

a content masking generator that automatically marks locations in the at least one audio stream and at least one recorded screen stream, the location associated with an event, metadata or content data, and that masks automatically at least a part of the at least one audio stream and at least a part of the at least one screen stream substantially at the locations and based on user access rights to obtain a masked audio stream an a masked screen stream, the at least a part of the at least one audio stream or at least a part of the at least one screen stream containing a personal detail or a credit card detail or a medical detail of the client; and

a user presentation generator for presenting to a user the masked audio stream an a masked screen stream.

30. The apparatus of claim 29 further comprising a metadata capture device for obtaining metadata associated with the at least one interaction.

31. The apparatus of claim 30 further comprising a metadata analysis device that analyzes the metadata obtained from the metadata capture device and associated with the at least one interaction.

32. The apparatus of claim 29 further comprising at least one capture device that captures audio and screen stream.

33. The apparatus of claim 29 wherein the screen comprises any one of the following:

a display device, a computer screen, a video screen, a television screen, or a portable device screen.

34. The apparatus of claim 33 wherein restricting access refers to restricting access to an event recorded from a screen.

35. The apparatus of claim 29 further comprising a masking methods device to provide an at least one content masking method to the content masking generator to be applied to content associated with the at least one interaction.

36. The apparatus of claim 29 wherein the masking of the at least one interaction is altered based on at least one user request.

37. The apparatus of claim 29 further comprising an event identification device that recognizes events in content associated with the at least one interaction and based on the results of analysis of the content, metadata or user data captured or obtained.

38. The apparatus of claim 37 wherein the event identification device identifies a compound event.

39. The apparatus of claim 37 wherein the event identification device identifies a single event.

40. The apparatus of claim 29 further comprising a request handler that generates displays and responds to commands associated with multimedia display requests, and a masked streaming device that generates selectively masked content associated with the at least one interaction.

41. The apparatus of claim 29 wherein the masking content generator alters at least a part of the interaction to be masked.

42. The apparatus of claim 29 wherein the masking content generator inserts an at least one content masking element into the at least one audio and screen stream associated with the at least one interaction to be masked.

43. The apparatus of claim 29 wherein the content to be masked is multimedia content.

44. The apparatus of claim 29 wherein the content user data is multimedia content user data.

45. The apparatus of claim 29 wherein masking is one of the following:

hiding of information;

replacing of information;

blurring of images or information;

preventing recording of information;

denying access;

enhancing a region of interest;

improving visibility of an image or information;

encrypting of information;

or manipulation of multimedia content based on actions associated with rules.

46. The apparatus of claim 29 further comprising a masked storage that stores selectively masked content.

47. The apparatus of claim 29 further comprising an unmasked storage that stores unmasked content.

48. The apparatus of claim 29 further comprising at least one triggering device that activates or control content capture, content recording, content storage, or content masking activation.

49. The apparatus of claim 29 wherein restricting access to the at least one audio stream and screen stream comprises masking the audio and screen streams except a first portion thereof, or masking only a second portion of the audio and screen streams.

50. The apparatus of claim 29 wherein the metadata is Computer Telephony Interface data, and wherein the Computer Telephone Interface data is captured by an external capture device.

51. The apparatus of claim 29 wherein the content user data is a product of analysis.

52. The apparatus of claim 51 wherein the analysis is selected from the group consisting of:

audio analysis;

emotion analysis;

gender detection;

language detection;

or key stroke analysis.

53. The apparatus of claim 29 wherein the machine is a computer.

54. The apparatus of claim 29 wherein the personal detail or the credit card detail or the medical detail is identified according to a template.

55. The apparatus of claim 54 wherein the template represents a credit card number.

56. The apparatus of claim 29 wherein the content masking generator marks locations in the at least one audio stream and at least one recorded screen stream by embedding information in the at least one audio stream and the at least one recorded screen stream.

57. A method for restricting access to call center multimedia information comprising an at least one audio stream and at least one screen stream comprising at least one interaction between a customer and either an agent of the call center or a machine, the method comprising:

capturing, with a multimedia capture device, the at least one audio stream and at least one screen stream;

extracting metadata from the call center multimedia information;

analyzing the metadata;

selecting a multimedia masking method according to the call center multimedia information or the metadata;

obtaining a security masking method from a security masking methods engine;

generating automatically, and based on user access rights, a multimedia masking for the call center multimedia information, using the security masking method;

automatically marking a location in the at least one audio and screen stream by embedding information in the at least one audio stream and at least one screen stream, the location associated with an event, metadata, or content user data;

automatically inserting the multimedia masking into the at least audio stream and screen stream at the location, to create masked multimedia restricting access to a personal detail or a credit card detail or a medical detail of the customer; and

displaying the masked multimedia,

wherein said multimedia capture device is selected from the group consisting of;

a disk device, a tape device, a DAT device, a telephone, a cellular phone, a satellite phone, a wireless device, a microphone, a RF receiving device, a VOIP device, a facsimile machine, a computing device in which a session of data communications occurs, and any combination thereof.

58. The method of claim 57, wherein said session of data communications is selected from the group consisting of email, a chat session, a keystroke, a use of a virtual keyboard or pen, touch screen use, mouse use, peripheral use, an interaction with a computer, and any combination thereof.

59. A method for restricting access to recorded multimedia information comprising an at least one audio stream and at least one screen stream comprising a screen event in a data format structure, the streams comprising at least one interaction between a customer and a call center agent, the method comprising:

capturing, with a multimedia capture device, the at least one audio stream and at least one screen stream;

automatically marking a location in the at least one audio stream and at least one screen stream, the location associated with an event, metadata, or content user data;

automatically masking at the location at least a first part of the at least one audio stream and at least a second part of the at least one screen stream, the first part or the second part comprising a personal detail or a credit card detail or a medical detail of the customer,

wherein the multimedia capture device records data streams comprising voice, screen events, key strokes, and physical access device data,

wherein said multimedia capture device is selected from the group consisting of;

a disk device, a tape device, a DAT device, a telephone, a cellular phone, a satellite phone, a wireless device, a microphone, a RF receiving device, a VOIP device, a facsimile machine, a computing device in which a session of data communications occurs, and any combination thereof.

60. The method of claim 59, wherein said session of data communications is selected from the group consisting of email, a chat session, a keystroke, a use of a virtual keyboard or pen, touch screen use, mouse use, peripheral use, an interaction with a computer, and any combination thereof.

61. An apparatus for restricting access to recorded call center multimedia information comprising an at least one audio and screen stream, the stream comprising an at least one audio and screen stream comprising a screen event in a data format structure, the stream comprising at least one interaction between a customer and a call center agent or between a customer and a machine, the apparatus comprising:

a multimedia capture device that captures the at least one audio, and at least one screen stream; and

a content masking generator that automatically marks a location in the at least one audio stream or at least one screen stream, and masks at least a part of the at least one audio stream or at least a part of the at least one screen stream thus restricting access to a personal detail or a credit card detail or a medical detail of the customer based on events, metadata or content user data associated with the at least one interaction, and based on user access rights,

wherein the multimedia capture device records data streams comprising voice, screen events, key strokes, and physical access device data.