Method and apparatus for sociological data analysis

US 20060271526A1
Filed: 07/31/2006
Published: 11/30/2006
Est. Priority Date: 02/04/2003
Status: Active Grant

First Claim

Patent Images

1. A method to enable improved analysis and use of sociological data, the method comprising identifying causal relationships between a plurality of documents;

identifying a plurality of characteristics of a communication, including a modality used, actors involved, proximate events of relevance; and

enabling a user to query based on all of the characteristics available.

View all claims

10 Assignments

Timeline View

Assignment View

0 Petitions

Accused Products

Abstract

A method to enable improved analysis and use of sociological data, the method comprising identifying causal relationships between a plurality of documents, identifying a plurality of characteristics of a communication, including a modality used, actors involved, proximate events of relevance, and enabling a user to query based on all of the characteristics available.

Citations

157 Claims

1. A method to enable improved analysis and use of sociological data, the method comprising identifying causal relationships between a plurality of documents;
- identifying a plurality of characteristics of a communication, including a modality used, actors involved, proximate events of relevance; and
  
  enabling a user to query based on all of the characteristics available.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157)
- - 2. The method of claim 1, wherein the characteristics further comprise a tone used by an actor in the communication.
  - 3. The method of claim 2, wherein the characteristics further comprise a predominant tone used in a discussion.
  - 4. The method of claim 2, further comprising:
    - determining a percentage of communication with respect to a topic which utilizes a tone; and
      
      tagging the communication with an appropriate indicator to illustrate a predominant tone.
  - 5. The method of claim 2, further comprising:
    - identifying a pattern of tonal usage on the part of an individual actor with respect to other actors; and
      
      identifying deviations from the pattern of tonal usage; and
      
      associating specific topics, actors, events of importance, or changes in communications patterns associated with the deviation.
  - 6. The method of claim 5, further comprising:
    - identifying unusual quotes or capitalization as indications of sarcasm.
  - 7. The method of claim 5, further comprising:
    - classifying an item as hot or worthy of special examination, based on a tone associated with the item.
  - 8. The method of claim 5, further comprising:
    - displaying overall tonal content per time period for an actor, to identify anomalies.
  - 9. The method of claim 8, wherein the display is a matrix, with years displayed along the Y-axis, to enable a comparison on a seasonal basis.
  - 10. The method of claim 2, further comprising:
    - identifying, for a particular actor, divergence in tones in communications with different actors on a particular topic within a given timeframe.
  - 11. The method of claim 2, further comprising:
    - illustrating a range of tones used in communications between two actors over time, for sending and receiving messages.
  - 12. The method of claim 1, further comprising:
    - determining a tone used in communication between actors; and
      
      adjusting the determination based on communication modality.
  - 13. The method of claim 1, further comprising:
    - identifying the modality of the communication when displaying a discussion.
  - 14. The method of claim 13, further comprising:
    - establishing an actor heartbeat based on an activity level, the activity level differentiated between the modalities of communication.
  - 15. The method of claim 14, further comprising:
    - utilizing the actor heartbeat to identify periods of vacation, illness, or business travel.
  - 16. The method of claim 14, further comprising:
    - utilizing the actor heartbeat to identify missing activities or abnormally low activities to identify deletions.
  - 17. The method of claim 14, wherein the modalities include:
    - telephone, instant messaging, calendar, in-person meetings, creation of documents, editing of documents.
  - 18. The method of claim 1, further comprising:
    - evaluating actor behavior over time, to create a threshold behavior for the actor; and
      
      identifying deviations from the threshold behavior.
  - 19. The method of claim 18, further comprising:
    - relating the deviation from the threshold behavior to relevant events.
  - 20. The method of claim 18, wherein the behavior includes one or more of the following:
    - a reply cycle to communications, tone used, communications modality used.
  - 21. The method of claim 18, further comprising:
    - displaying an average length and a total number of discussions for a topic selected by a user, as well as the number of discussions that ended in a meeting event.
  - 22. The method of claim 21, further comprising:
    - highlighting deviations from a normal number of such interactions for a particular occurrence of the topic.
  - 23. The method of claim 1, further comprising:
    - identifying potential meeting events which may or should have occurred based on data in the corpus of documents; and
      
      classifying the meeting event in one or more of the following categories;
      
      ad hoc, scheduled, cancelled, shortened, lengthened, calendared, rescheduled, requested, declined, attended remotely.
  - 24. The method of claim 23, further comprising:
    - determining if a particular actor was present at the meeting event.
  - 25. The method of claim 1, wherein identifying causal relationships further comprises:
    - associating text of a supportive document with a meeting event, ensuring that the meeting event is retrieved for queries including terms in the supportive document.
  - 26. The method of claim 1, further comprising:
    - identifying mixed type discussions which have at least one meeting event.
  - 27. The method of claim 1, further comprising:
    - calculating a probability of a meeting event occurring in a conversation, and depicting the probabilities for various discussions.
  - 28. The method of claim 1, further comprising:
    - analyzing communications to identify normal paths of communication between actors; and
      
      identifying a communication to which, according to the normal paths of communication, a particular actor should have been invited but was not.
  - 29. The method of claim 28, wherein the communication is a meeting event.
  - 30. The method of claim 1, further comprising:
    - using colorful indicators for relevant occurrences to ensure that the indicator is visible in a thumbnail image size.
  - 31. The method of claim 1, further comprising:
    - time slicing data by creating substantially equal increments of time within a time interval of interest specified by the user, and creating a snapshot image for each time increment to provide a depiction of data for the user.
  - 32. The method of claim 31, wherein the snapshots are presented as one of the following:
    - a set of thumbnail images, a matrix of images aligned vertically by time period, or in a sequential fashion as a movie.
  - 33. The method of claim 1, further comprising:
    - depicting communications between a plurality of actors using communication lines between the actors; and
      
      enabling graphical queries based on selecting the communication lines between actors.
  - 34. The method of claim 33, wherein the communications lines identify the communication as in-person, via telephone, via email, via IM, via fax, via pager, via conference call.
  - 35. The method of claim 33, wherein the communications depicted by communication lines include meeting events.
  - 36. The method of claim 33, further comprising:
    - enabling a user to access documents associated with a communication line via the communications graph.
  - 37. The method of claim 33, wherein the communications line reflects one or more separate communications, and further comprising:
    - itemizing the one or more communications on mouse over of the communications line.
  - 38. The method of claim 33, further comprising:
    - displaying the communications graph to illustrate communications between a set of actors, in response to a search query; and
      
      displaying a special icon for a meeting event between more than two actors.
  - 39. The method of claim 38, wherein the meeting event is selected from among:
    - an in-person meeting, and telephone conference.
  - 40. The method of claim 1, further comprising:
    - overlaying an organization chart over a communications graph displaying communication between actors, to enable a user to see unusual communications patterns.
  - 41. The method of claim 40, further comprising:
    - highlighting a missing link, indicating that a particular actor had a substantially different level of communication with another actor compared to equivalently positioned actors.
  - 42. The method of claim 40, further comprising:
    - highlighting an “
      
      extra”
      
      link, indicating that a particular actor that is outside of channels was included in the communication.
  - 43. The method of claim 40, further comprising:
    - identifying a boundary between groups within the organization; and
      
      highlighting communications which cross the boundary.
  - 44. The method of claim 40, further comprising:
    - enabling the overlaying for searches and graphical queries, to enable the identification of anomalies by topic, relevant event, time period.
  - 45. The method of claim 40, further comprising:
    - generating a graph of a spread of knowledge regarding an issue through actors in an organization, based on utilization of identifiers for the issue.
  - 46. The method of claim 1, further comprising:
    - depicting communications between a plurality of actors using communication lines between the actors in relationship with a structural relationship chart.
  - 47. The method of claim 46, further comprising:
    - highlighting abnormal levels of communication between actors.
  - 48. The method of claim 46, further comprising:
    - enabling a user to select actor characteristics for this depiction, wherein the characteristics may include;
      
      race, religion, gender, national origin, length of time at company.
  - 49. The method of claim 1, further comprising:
    - identifying instruction relaying by identifying instruction language in communication, and identifying an instruction giver and an interpreter;
      
      displaying a communication graph based on instruction relaying.
  - 50. The method of claim 49, further comprising:
    - differentiating between a real instruction and a mere forward.
  - 51. The method of claim 50, further comprising:
    - measuring actor proximity based on interactions, including the use of mere forwards.
  - 52. The method of claim 50, further comprising:
    - separately highlighting real instruction giver, mere forwarding instruction giver, interpreter of forwarding instruction, and interpreter of real instruction.
  - 53. The method of claim 49, further comprising:
    - in the communication graph, highlighting actors as givers and interpreters based on reaching a threshold level.
  - 54. The method of claim 1, further comprising:
    - extracting formulas from spreadsheets;
      
      identifying a basis for differences between the spreadsheets including differences in manually entered information and differences in formulas to assist in determining whether the spreadsheets are related.
  - 55. The method of claim 54, further comprising:
    - identifying whether the formulas are the same and identifying if the formulas are supersets, and if so, determining that the spreadsheet is related.
  - 56. The method of claim 54, further comprising:
    - identifying a number of cells that differ because of the formulas and because of the manually entered information; and
      
      if a number of non-formulaic changes is small, determining that the spreadsheets are related as families.
  - 57. The method of claim 1, further comprising:
    - enabling a user to select multiple matters, and perform queries across the multiple matters.
  - 58. The method of claim 57, further comprising:
    - enabling a user to conjoin information from the multiple matters to form a new discussion.
  - 59. The method of claim 1, further comprising:
    - a timeline view to depict one or more items related to a discussion in parallel along a timeline, to enable a user to see overlapping items, as well as the duration of each item.
  - 60. The method of claim 59, wherein the item shown in the timeline view comprises one or more of the following:
    - email, IM, SMS, phone call activity, journal entries, travel status of actors, notes, external events of importance, and calendar entries.
  - 61. The method of claim 59, wherein the timeline view further comprises indicators for a start time and stop time associated with a relevant subpoena.
  - 62. The method of claim 59, further comprising:
    - visually differentiating work and non-work times, relative to a determined work habit of an actor.
  - 63. The method of claim 62, further comprising:
    - in a discussion, visually differentiating the work and the non-work times based on a predominant actor'"'"'s work habits.
  - 64. The method of claim 1, further comprising:
    - global controls available to a user to code a group of documents at the same time.
  - 65. The method of claim 64, further comprising:
    - providing a control that promotes a state of an individual document to the global controls.
  - 66. The method of claim 64, further comprising:
    - local controls enabling a user to override the code applied with the global controls on a per document basis.
  - 67. The method of claim 66, wherein the local controls for an item are visible only when the item is coded differently from the global code.
  - 68. The method of claim 1, further comprising:
    - displaying a plurality of controls for a current user interface display, the controls being defined by a document type currently being displayed.
  - 69. The method of claim 68, further comprising:
    - adding the controls defined by the current document type to a list of available global controls if multiple items of the document type are part of a current collection of items.
  - 70. The method of claim 1, further comprising:
    - enabling a user to categorize items for production; and
      
      providing an interface mechanism to enable a user to select certain content in a document and indicate a category for the content; and
      
      enabling a user to indicate that the certain content places all other documents including that content in the same category.
  - 71. The method of claim 1, further comprising:
    - a duplicate indicator to indicate when documents are one of the following;
      
      true duplicates, near duplicates, or prior or subsequent versions, or different formats of the same document.
  - 72. The method of claim 71, further comprising:
    - when displaying a document for review, indicating a tally of prior reviews of the document and near duplicate documents.
  - 73. The method of claim 72, further comprising:
    - determining a level of unanimity of the prior reviews, and if the level of unanimity is above a threshold, labeling the item.
  - 74. The method of claim 1, further comprising:
    - when displaying responsive data exchanges, displaying the exchanges ascending or descending order.
  - 75. The method of claim 74, further comprising:
    - in communications containing quoted text, enabling a user to hide redundant quoted text.
  - 76. The method of claim 74, further comprising:
    - in communications containing quoted text, enabling a user to hide quoted text by author.
  - 77. The method of claim 1, further comprising:
    - associating one or more electronic identities with each author; and
      
      identifying human identified electronic identities separately from system-identified electronic identities.
  - 78. The method of claim 1, further comprising:
    - in a search query enabling a user to associate communications with an actor that have one or more of the following features;
      
      to an actor, from the actor, created by the actor, modified by the actor, and about the actor.
  - 79. The method of claim 1, further comprising:
    - analyzing an actor'"'"'s communications over a determined time period;
      
      displaying an actor information report, including a percentage communication devoted to a designated topic.
  - 80. The method of claim 1, further comprising:
    - enabling a user to configurably highlight actor names, and any text that is attributable to them, on any basis.
  - 81. The method of claim 80, further comprising:
    - enabling the user to select the actor names for highlighting based on any available actor characteristics.
  - 82. The method of claim 81, wherein the characteristics may include one or more of the following criteria:
    - privileged status, actor group, one or more actors returned by a particular query, and individually identified actor.
  - 83. The method of claim 1, further comprising:
    - identifying an item having an associated custodian identity which does not provide an exact match with a known actor name;
      
      identifying edit distance between the custodian identity and the known actor names; and
      
      ordering the available known actors based on an edit distance.
  - 84. The method of claim 1, wherein creating a sociological relationship comprises correctly identifying actors, and correctly identifying actors comprises:
    - enabling a user to correct automated actor identification by assigning an electronic identity to an actor or remove an electronic identity from an actor; and
      
      enabling a user to add, remove, and modify attributes associated with an actor, the attributes including one or more of the following;
      
      organizational membership, privileged status, and country of residence.
  - 85. The method of claim 84, further comprising:
    - verifying the correction made by the user based on one or more of the following;
      
      classification of actor-associated items by other users, linguistic fingerprinting, and number of documents already reviewed with the erroneous assignment.
  - 86. The method of claim 1, further comprising:
    - building a discussion including interrelated collections of items; and
      
      exploding the discussion when the discussion includes items that must be placed in different stages in a review workflow according to workflow rules, the exploding separating the discussion into discrete chunks.
  - 87. The method of claim 1, further comprising:
    - enabling addition of new items to a discussion after review has been initiated and processing the new items by doing one or more of the following;
      
      separating the new items into separate clusters and adding a notification to the discussion indicating the existence of the new items;
      
      join the new items to the discussion, but isolate the new items to enable easy examination, without re-examining the discussion;
      
      join new items to the discussion, but designate the discussions and individual new items with appropriate visual indicators;
      
      join new items to the discussion, and apply global settings to the new items if the new items are homogeneous with the discussion;
      
      join new items to the discussion, analyze the new items and system-calculated settings to the new items.
  - 88. The method of claim 1, further comprising:
    - determining a personal proximity and a professional proximity and overall proximity between actors.
  - 89. The method of claim 88, further comprising:
    - weighing one or more of the following to determine proximity between actors;
      
      multi-topic communications, communications regarding attendance at social events, use of a large range of tones, use of different communications channels, use of short format messages, and use of salutation.
  - 90. The method of claim 88 further comprising:
    - identifying divergence between the proximity between actors and a tone used in a communication between actors.
  - 91. The method of claim 88, further comprising:
    - illustrating communications between two actors with a tone and proximity component.
  - 92. The method of claim 1, further comprising:
    - analyzing sequential backups to identify old data which was also present in a previous backup, data that was in a previous backup and now is missing, data that was missing in a previous backup but is old data, and new data.
  - 93. The method of claim 1, further comprising:
    - identifying named entities using application metadata, selected from among data in custom dictionaries and address books on users'"'"' system; and
      
      determining common misspellings based on auto-correct settings in users'"'"' applications.
  - 94. The method of claim 1, further comprising:
    - identifying a baseline document lifecycle for a given type of document type, the document lifecycle is a set of stages that documents of the given type typically move through, from initial creation to ultimate abandonment.
  - 95. The method of claim 94, further comprising:
    - identifying bursts of activity with respect to a document, the bursts of activity involving multiple activities in close time proximity.
  - 96. The method of claim 95, further comprises:
    - assigning version numbers to a document for each lifecycle event, wherein major numbers are assigned by the burst of activity, and minor numbers by sequential user-saved changes.
  - 97. The method of claim 95, further comprising:
    - comparing the life cycle events and the bursts of activity for documents related to the event for multiple occurrences of the event over time.
  - 98. The method of claim 94, further comprising:
    - relating the lifecycle events for one or more documents related to an event to the event along a timeline.
  - 99. The method of claim 98, further comprising:
    - relating the lifecycle events for a document to a workflow associated with documents of a given type.
  - 100. The method of claim 1, further comprising:
    - providing an item information report to display a history of a particular item, including links to each actual version of the item available in the corpus.
  - 101. The method of claim 100, wherein the item information report includes one or more of the following:
    - a list of custodians;
      
      a list of actors that deleted the item;
      
      sequential list of all dates and times that something occurred with the item;
      
      document lifecycle events;
      
      actors who edited, sent, received the item; and
      
      a list of actors who should have received the item but did not.
  - 102. The method of claim 1, further comprising:
    - tools to enable an administrator to review logs of user actions and apply corrections to incorrect user actions.
  - 103. The method of claim 102, further comprising:
    - enabling an authorized user to globally replace document classifications.
  - 104. The method of claim 102, further comprising:
    - displaying clusters of documents based on classification, to flag heterogeneous clusters for additional review.
  - 105. The method of claim 1, further comprising correcting for email corruption based problems by:
    - identifying emails based on hashes of header information;
      
      create frequency occurrence table based on content of emails;
      
      compare frequency occurrence table, and identify emails as identical, corrupted, or compared as a result of hash collision.
  - 106. The method of claim 105, further comprising:
    - enabling a user to select a preferred copy of an email for future use.
  - 107. The method of claim 1, further comprising:
    - enabling a search for individuals including actors and system users to determine who has seen, received, or acted on a particular document.
  - 108. The method of claim 107, further comprising:
    - identifying system users who are also actors, and document review decisions made by system users upon their own work.
  - 109. The method of claim 107, further comprising:
    - creating a batch of data and automatically batch assigning the data based on current reviewer performance characteristics of similar data-types.
  - 110. The method of claim 109, further comprising:
    - projecting review completion time, based on a current number of documents, currently available reviewers, batch type assignments, and the current reviewer performance characteristics.
  - 111. The method of claim 109, further comprising:
    - projecting review completion time based on a hypothetical scenario input by the user.
  - 112. The method of claim 1, further comprising:
    - pre-tagging data based on determined topical content;
      
      performing a comparison between tagging and human reviewer decisions.
  - 113. The method of claim 112, further comprising:
    - generating a report showing a time-per document, and a time-per pre-flagged document.
  - 114. The method of claim 112, further comprising:
    - evaluating a disagreement between human reviewer decisions and pre-tagging decisions to correct pre-tagging mistakes and evaluate human reviewer performance.
  - 115. The method of claim 114, further comprising:
    - creating a report noting differences in reviewer decisions for similar documents that do not have objectively different classifications.
  - 116. The method of claim 1, further comprising:
    - enabling the creation of a hierarchy of folders to sort data from the corpus for research purposes.
  - 117. The method of claim 116, further comprising:
    - defining a findings folder containing only items whose content is determined by a user to relate to a specific topic, and to providing a definitive conclusion regarding the specific topic.
  - 118. The method of claim 117, wherein the definitive conclusion is:
    - true, false, or unknown, probably true, probably false.
  - 119. The method of claim 118, further comprising:
    - enabling a user to create a decision tree to reach the definitive conclusion.
  - 120. The method of claim 117, further comprising:
    - identifying multiple findings folders which contain significantly overlapping data sets, and alerting users of the overlap to ensure that duplication of work is minimized.
  - 121. The method of claim 117, further comprising:
    - enabling a user to construct a query by selecting a subset of the hierarchy of folders.
  - 122. The method of claim 117, further comprising:
    - enabling a user to assign priorities to findings folders; and
      
      enabling de-duplication of an item between findings folders, where the findings folder with the highest priority receives the item.
  - 123. The method of claim 122, further comprising:
    - enabling a user to highlight an item or data within a document as being of high interest relative to the issues of the findings folder; and
      
      enabling the addition of notes to the highlight; and
      
      creating a separate object from the highlighted data for storage in the folder.
  - 124. The method of claim 123, further comprising:
    - utilizing the separate object from the highlighted data for relevance searching.
  - 125. The method of claim 117, further comprising:
    - profiling the findings folder based on the attributes of the items in the findings folder.
  - 126. The method of claim 117, further comprising:
    - tagging each item that is in a findings folder with an attribute identifying it as “
      
      important”
      
      within a corpus, the attribute available for searches of the corpus elsewhere in the system.
  - 127. The method of claim 116, further comprising:
    - automatically organizing the hierarchy of folders based on a degree of overlap between the folders.
  - 128. The method of claim 127, further comprising:
    - generating a report illustrating the degree of overlap between findings folders.
  - 129. The method of claim 1, further comprising:
    - clustering content of folder creating reports on groups of items in the folder which share user-selected attributes.
  - 130. The method of claim 129, wherein the report includes a timeline illustrating a period of time that is covered by each group of items.
  - 131. The method of claim 1, further comprising:
    - a configurable query window to enable a user to specify a set of items to include in the query, including an actual query, and a presentation style for showing results.
  - 132. The method of claim 131, wherein the presentation styles comprise one of the following:
    - main view with query controls, main view, time elapsed style, thumbnail style, tabular.
  - 133. The method of claim 1, further comprising:
    - enabling a user in a query window to construct a query and save the query for reuse.
  - 134. The method of claim 133, further comprising:
    - providing fully-customizable controls that are part of a query specification, which may be customized through the query window itself.
  - 135. The method of claim 1, wherein the corpus includes documents including items not originally in a text file form.
  - 136. The method of claim 135, wherein items that were not originally in written form include audio data, video data, instant messaging (IM) data, calendar data, telephone record data, and database data.
  - 137. The method of claim 1, further comprising:
    - enabling an automatic selection of a subset of a corpus of documents for production.
  - 138. The method of claim 137, further comprising:
    - presenting a user interface through which the user may review the automatic selection of a subset of a corpus of documents for production.
  - 139. The method of claim 138, further comprising:
    - allowing the user to modify parameters used in the automatic selection.
  - 140. The method of claim 137, further comprising:
    - performing an initial classification of the subset of the corpus of documents into a relevant set of reviewable documents.
  - 141. The method of claim 140, further comprising:
    - partitioning the corpus of documents in at least two sets.
  - 142. The method of claim 141, further comprising:
    - one of the partitioned sets is a supra-responsive set designed to contain a majority of documents of interest from the corpus.
  - 143. The method of claim 137, further comprising:
    - enabling a user to review of a sample of the automatically selected documents to ascertain the validity of the automatic selection.
  - 144. The method of claim 143, wherein a sample is created by the use of a statistical sampling technique applied to a set of documents automatically selected.
  - 145. The method of claim 137, further comprising:
    - distilling the automatic selection into a basis involving categorization components.
  - 146. The method of claim 145, wherein a categorization component represents a set of documents grouped as a direct result of a categorization technique.
  - 147. The method of claim 137, further comprising:
    - refining the accuracy by supervised review of the subset of corpus documents as chosen by an automatic selection.
  - 148. The method of claim 1, further comprising:
    - utilizing a plurality of heterogeneous categorization mechanisms to automatically categorize documents in the corpus; and
      
      arranging the heterogeneous categorization mechanisms to a partially ordered rule set to reduce a search space of possible combinations of the plurality of heterogeneous categorization mechanisms.
  - 149. The method of claim 148, further comprising:
    - providing a user interface to allow the user to alter an order of the heterogeneous categorization mechanisms in the partially ordered rule set.
  - 150. The method of claim 1, further comprising:
    - utilizing a plurality of heterogeneous categorization mechanisms to automatically categorize documents in the corpus; and
      
      analyzing and visualizing a relative performance, overlaps, and inconsistencies among the plurality of heterogeneous categorization mechanisms.
  - 151. The method of claim 150, further comprising:
    - automatically breaking up poorly performing categorization mechanisms.
  - 152. The method of claim 1, further comprising:
    - utilizing a plurality categorization mechanisms to automatically categorize documents in the corpus;
      
      receiving categorization decisions from human reviewers; and
      
      determining a best fit of the categorization mechanisms to the categorization decisions of the human reviewers.
  - 153. The method of claim 1, further comprising:
    - utilizing a plurality categorization mechanisms to automatically categorize documents in the corpus; and
      
      utilizing a supervised sampling process, based on receiving categorization decisions from human reviewers, to increase accuracy of automated categorization of the documents.
  - 154. The method of claim 153, wherein non-randomized sets are used to increase human performance.
  - 155. The method of claim 153, wherein the categorization decisions from human reviewers may modify a category with an “
    - arguable”
      
      modifier for decisions that are not certain.
  - 156. The method of claim 153, further comprising:
    - projecting a final size of the documents in each of the categories, once categorized across the whole data set.
  - 157. The method of claim 156, further comprising:
    - assessing a possible overfitting of categorization mechanisms to a particular sample, based on the final size projection.

Specification

Resources

Litigation Campaign Assessment

Current Assignee
Adobe Inc.
Original Assignee
Cataphora Incorporated
Inventors
Roberts, Steven L., Charnock, Elizabeth, Thompson, Curtis, Schon, Keith, Espanol, Joanes C., Bothra, Arpit, Nystrom, Jonathan D., Brouk, Roman

Granted Patent

US 7,519,589 B2
Time in Patent Office

Days
Field of Search
US Class Current

1/1
CPC Class Codes

G06Q 30/02 Marketing; Price estimation...

Method and apparatus for sociological data analysis

First Claim

10 Assignments

0 Petitions

Accused Products

Abstract

Citations

157 Claims

Specification

Solutions

Use Cases

Quick Links

Method and apparatus for sociological data analysis

First Claim

10 Assignments

Subscription Required

Subscription Required

0 Petitions

Subscription Required

Accused Products

Subscription Required

Abstract

Citations

157 Claims

Specification

Subscription Required

Solutions

Use Cases

Quick Links