Method and apparatus for determining expertise based upon observed usage patterns

• US 7,546,295 B2
• Filed: 02/08/2006
• Issued: 06/09/2009
• Est. Priority Date: 12/27/2005
• Status: Active Grant

• Current Assignee: Monetate, Inc. (StartApp, Inc.)
• Original Assignee: Baynote Incorporated (StartApp, Inc.)
• Inventors: Brave, Scott, Bradshaw, Robert, Minson, Christopher, Jia, Jack
• Primary Examiner(s): Cottingham; John R
• Assistant Examiner(s): Bromell; Alexandria Y
• Application Number:
  US11/350,668
• Publication Number:
  US 20070150465A1
• Time in Patent Office: 1,217 Days
• Field of Search: 707/3, 707/7, 705/7, 705/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: Y10S 707/966 Distributed
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

The invention comprises a set of complementary techniques that dramatically improve enterprise search and navigation results. The core of the invention is an expertise or knowledge index, called UseRank that tracks the behavior of website visitors. The expertise-index is designed to focus on the four key discoveries of enterprise attributes: Subject Authority, Work Patterns, Content Freshness, and Group Know-how. The invention produces useful, timely, cross-application, expertise-based search and navigation results. In contrast, traditional Information Retrieval technologies such as inverted index, NLP, or taxonomy tackle the same problem with an opposite set of attributes than what the enterprise needs: Content Population, Word Patterns, Content Existence, and Statistical Trends. Overall, the invention emcompasses Baynote Search—a enhancement over existing IR searches, Baynote Guide—a set of community-driven navigations, and Baynote Insights—aggregated views of visitor interests and trends and content gaps.

Claims

1. A computer implemented method for automatically determining any of importance of an on-line asset and expertise that one or more members of an online community possess, without asking said community members directly identifying, extracting, capturing, and leveraging expertise and knowledge, comprising the processor executing the steps of:

observing usage by a community of peers and experts who show high affinity in connection with online assets one or more documents;

employing automatic techniques to extract patterns from said usage;

identifying usefulness of an online asset by observing user implicit behaviors in connection with said usage patterns of said online asset and by extracting behavioral patterns from said observations;

refining said identified online asset usefulness by context, wherein the context of each online asset is automatically detected based on observed terms/topics from individual and group user behaviors when said online asset is determined to be useful based upon said individual and group user behaviors;

assigning to each said online asset a document impact factor score for each possible topic/term, said document impact factor representing the importance of each said online asset to each topic;

assigning to each user an expert impact factor which is determined by aggregating identified topics of online assets each user has found useful, weighted by the document impact factor and by document rareness, wherein said expert impact factor and other observed patterns of behavior define a user'"'"'s identified expertise; and

using said identified expertise of each user to identify a community of experts given a specific topic/term of interest expressed by a user, and to identify a community of peers for a given user based upon a relationship between a target user'"'"'s identified expertise and all other users;

said observed usage patterns comprising user online search, navigation, and interaction behavior, said behavior including any of searches performed and position in user trail;

assets viewed and position in user trail;

dwell, range, scrolling, think time, and mouse movement on an asset;

anchors and lines used in asset text;

virtual bookmarks and virtual printing; and

explicit downloading, emailing, printing, saving, and removing to or from a hardware storage device.

2. The method of claim 1, further comprising the step of:

classifying usage of a given asset by a given user as successful or unsuccessful use based on a combination of factors comprising any of time spent on an online asset, explicit user actions taken on said asset, said actions comprising any of print, save, email, and bookmark, mouse and keyboard behavior, length and distribution of stable pauses between activity, navigation to and from said asset, and browser window state, said state comprising any of foreground, background, minimized, maximized, and closed.

3. The method of claim 1, further comprising the step of:

providing a significantly greater weighting to the importance of implicit observations over the weighting provided to explicit observations.

4. The method of claim 1, further comprising the steps of:

analyzing all observations; and

via said analysis, generating a set of recommendations comprising distilled experiences from a community of users;

wherein said recommendations age over time and are discarded if they have relatively little value; and

wherein recommendations which are most valuable based on repeated usage are stored into a long term memory.

5. The method of claim 1, further comprising the step of:

for a given user who may be anonymous, said user visiting a particular site, and a given context comprising any of what page the user is on and how the user got there, providing recommendations to said user that allow said user to navigate said site more efficiently.

6. The method of claim 1, further comprising the steps of:

generating a set a recommendations that may be applied to a search; and

for a given user who may be anonymous, and a given search query, using said recommendations to refine and augment a resulting search.

7. The method of claim 6, further comprising the step of:

driving said recommendations not just by individual uses, but by the use of communities, leveraging the wisdom of crowds and community emergent behavior.

8. The method of claim 1, further comprising the step of:

identifying communities comprising any of peer groups and expert groups based on an information context;

wherein communities are nested and defined by different levels of contexts.

9. The method of claim 1, further comprising the steps of:

building a target community of users, comprised of identified peers and experts for a topic expressed by a target user; and

analyzing said target community'"'"'s usage patterns to identify those online assets which have highest connection to a particular topic.

10. The method of claim 1, further comprising the steps of:

based upon observed contexts, segmenting said community into any of progressively smaller and nested sub-communities, and affinity groups of behaviorally defined peers; and

any of

storing said progressively smaller and nested sub-communities, and affinity groups of behaviorally defined peers per user; and

aggregating said progressively smaller and nested sub-communities, and

affinity groups of behaviorally defined peers across subgroups of users;

wherein said affinity groups identify users within the group based upon their behaviors, terms/topics (context) associated with the group, and based upon their patterns of associations among said affinity groups or subgroups.

11. An apparatus for automatically determining any of importance of an on-line asset and expertise that one or more members of an online community possess, without asking said community members directly, comprising:

means for observing usage by a community of peers and experts who show high affinity in connection with online assets;

a processor for employing automatic techniques to extract patterns from said usage;

said processor comprising a module for identifying usefulness of an online asset by observing user implicit behaviors in connection with said usage patterns of said online asset and by extracting behavioral patterns from said observations;

said processor comprising a module for refining said identified online asset usefulness by context, wherein the context of each online asset is automatically detected based on observed terms/topics from individual and group user behaviors when said online asset is determined to be useful based upon said individual and group user behaviors;

said processor comprising a module for assigning to each said online asset a document impact factor score for each possible topic/term, said document impact factor representing the importance of each said online asset to each topic;

said processor comprising a module for assigning to each user an expert impact factor which is determined by aggregating identified topics of online assets each user has found useful, weighted by the document impact factor and by document rareness, wherein said expert impact factor and other observed patterns of behavior define a user'"'"'s identified expertise;

said processor comprising a module for using said identified expertise of each user to identify a community of experts given a specific topic/term of interest expressed by a user, and to identify a community of peers for a given user based upon a relationship between a target user'"'"'s identified expertise and all other users; and

said observed usage patterns comprising user online search, navigation, and interaction behavior, said behavior including any of searches performed and position in user trail;

assets viewed and position in user trail;

dwell, range, scrolling, think time, and mouse movement on an asset;

anchors and lines used in asset text;

virtual bookmarks and virtual printing; and

explicit downloading, emailing, printing, saving, and removing.

12. The apparatus of claim 11, further comprising:

said processor classifying usage of a given asset by a given user as successful or unsuccessful use based on a combination of factors comprising any of time spent on an asset, explicit user actions taken on said asset, said actions comprising any of print, save, email, and bookmark, mouse and keyboard behavior, length and distribution of stable pauses between activity, navigation to and from said asset, and browser window state, said state comprising any of foreground, background, minimized, maximized, and closed.

13. The apparatus of claim 11, further comprising:

said processor providing a significantly greater weighting to the importance of implicit observations over the weighting provided to explicit observations.

14. The apparatus of claim 11, further comprising:

said processor analyzing all observations; and

via said analysis, said processor generating a set of recommendations comprising distilled experiences from a community of users;

wherein said recommendations age over time and are discarded if they have relatively little value; and

wherein recommendations which are most valuable based on repeated usage are stored into a long term memory.

15. The apparatus of claim 11, further comprising:

for a given user who may be anonymous, said user visiting a particular site, and a given context comprising any of what page the user is on and how the user got there, said processor providing recommendations to said user that allow said user to navigate said site more efficiently.

16. The apparatus of claim 11, further comprising:

said processor generating a set a recommendations that may be applied to a search; and

for a given user who may be anonymous, and a given search query, said processor using said recommendations to refine and augment a resulting search.

17. The apparatus of claim 16, further comprising:

said processor driving said recommendations not just by individual uses, but by the use of communities, leveraging the wisdom of crowds and community emergent behavior.

18. The apparatus of claim 11, further comprising:

said processor identifying communities comprising any of peer groups and expert groups based on an information context;

wherein communities are nested and defined by different levels of contexts.

19. The apparatus of claim 11, said processor further comprising:

a module for building a target community of users, comprised of identified peers and experts for a topic expressed by a target user; and

a module for analyzing said target community'"'"'s usage patterns to identify those online assets which have highest connection to a particular topic.

20. The apparatus of claim 11, said processor further comprising:

a module for, based upon observed contexts, segmenting said community into any of progressively smaller and nested sub-communities, and affinity groups of behaviorally defined peers; and

for

any of

storing said progressively smaller and nested sub-communities, and affinity groups of behaviorally defined peers per user; and

aggregating said progressively smaller and nested sub-communities, and

affinity groups of behaviorally defined peers across subgroups of users;

wherein said affinity groups identify users within the group based upon their behaviors, terms/topics (context) associated with the group, and based upon their patterns of associations among said affinity groups or subgroups.