Expertise finding

What is expertise?

The Oxford English Dictionary defines expertise as

a. Expert opinion or knowledge, often obtained through the action of submitting a matter to, and its consideration by, experts; an expert's appraisal, valuation, or report. b. The quality or state of being expert; skill or expertness in a particular branch of study or sport. Oxford English Dictionary, second edition, 1989.

At the risk of seeming pedantic, this definition is valuable because it emphasizes the notions that expertise is tightly associated with individuals. Rephrased, one could say that expertise is the quality exhibited by people whom we believe demonstrate an above-average ability to perform a non-trivial task. Furthermore, because one may not be an expert in a given field, expertise is often what other people say a given person demonstrates. People often accept such claims whether they can adequately verify this claim or not, one of many problems associated with assessing and quantifying human expertise.

Locating and assessing expertise, and why it matters

It can be argued that human expertise is the most valuable resource in the universe, more valuable than capital, means of production or intellectual property. Why? Because contrary to expertise, all other aspects of capitalism are now relatively generic: access to capital is global, as are means of the production for most areas of manufacturing, and intellectual property can be licensed. However, finding and “licensing” expertise is much harder, starting with the very first step: finding expertise that you can trust.

Until very recently, finding the expertise suitable for solving non-trivial problems was a haphazard process at best, requiring a mix of individual, social and collaborative practices. Mostly, it involves contacting individuals one trusts and asking for referrals, while hoping that one’s judgment in those individuals is justified and that their answers are thoughtful.

In the last fifteen years, a class of knowledge management software has emerged to facilitate and improve the quality of expertise finding, termed “expertise locating systems”. These programs range from social networking systems to knowledge bases. Some softwares, like those in the social networking realm, rely on users to connect each other, thus using social filtering to act as “recommendation systems”.

At the other end of the spectrum are specialized knowledge bases that rely on experts to populate the database with their self-determined areas of expertise and contributions, and do not rely on user recommendations. Of course, hybrids that feature expert-populated content in conjunction with user recommendations, exist and are arguably more valuable for doing so (e.g., LinkedIn).

Still other expertise knowledge bases rely strictly on external manifestations of expertise, herein termed “gated objects”, e.g., citation impacts for scientific papers or data mining approaches wherein many of the work products of an expert are collated. Such systems are free of user-introduced biases (e.g., ResearchScorecard), though the use of computational methods can introduce other biases.

Examples of the systems outlined above are listed in Table 1.

Type	Application domain	Data source	Examples
Social networking	Professional networking	LinkedIn [1] (LinkedIn Corporation)	User-generated
Scientific literature	Identifying publications with strongest research impact	Third-party generated	Science Citation Index [2] (Thomson Reuters)
Knowledge base	Private expertise database	User-generated	MIT ExpertFinder
Knowledge base	Publicly-accessible expertise database	User-generated	COS Expertise [3](Community Of Science); ResearchID http://www.thomsonreuters.com/products_services/scientific/ResearcherID](ThomsonReuters)
Knowledge base	Publicly-accessible expertise database	Third party-generated	ResearchScorecard [4] (ResearchScorecard Inc.); BiomedExperts [5] (Collexis Holdings Inc.)
Blog search engines		Third party-generated	Technorati [6](Technorati, Inc.)

Technical problems associated with expertise finding

A number of interesting problems follow from the use of expertise finding systems:

• The matching of questions from non-expert to the database of existing expertise is inherently difficult, especially when the database does not store the requisite expertise. This problem grows even more acute with increasing ignorance on the part of the non-expert due to typical search problems involving use of keywords to search unstructured data that are not semantically normalized, as well as variability in how well an expert has set up their descriptive content pages. Improved question matching is one reason why third-party semantically normalized systems such as ResearchScorecard and BiomedExperts should be able to provide better answers to user queries.
• Avoiding expert-fatigue due to too many questions/requests from non-experts (Maybury, D'Amore et al. 2002)
• Mitigating the social or professional stigma associated with the use of an authority ranking (used in Technorati and ResearchScorecard).
• Finding ways to avoid “gaming” of the system to reap unjustified expertise credibility.

Beyond expertise finding: Expertise ranking

Means of classifying and ranking expertise become rapidly crucial if the number of experts returned by a query is greater than a handful, which raises the following social problems associated with such systems:

• How can expertise be assessed objectively? Is that even possible? See definition of expertise for why this may not be possible
• What are the consequences of relying on unstructured social assessments of expertise, such as user recommendations?
• How does one distinguish authoritativeness as a proxy metric of expertise from simple popularity, which is often a function of one's ability to express oneself coupled with a good social sense?

Sources of data for assessing expertise

Below is a list of source of data that have been used to assess expertise, in no particular ranking order:

• user recommendations
• help desk tickets: what the problem was and who fixed it
• e-mails between users
• published documents, whether private or on the web
• user-maintained web pages
• patents
• scientific publications
• reports (technical, marketing, etc)
• issued grants
• clinical trials
• product launches

Interesting expertise systems over the years

In no particular order...

• Tacit Knowledge Systems
• MIT’s ExpertFinder (Viavacqua 1999)
• MITRE’s Expert Finder (Mattox, Maybury et al. 1999) http://www.mitre.org/news/the_edge/june_98/third.html
• MITRE’s XpertNet
• Dataware II Knowledge Directory
• Autonomy
• Thomson’s tool
• Hewlett-Packard’s CONNEX
• Microsoft’s SPUD project

References

Maybury, M., R. D'Amore, et al. (2002). "Awareness of Organizational Expertise." International Journal of Human-Computer Interaction 14(2): 199-217.

Mattox, D., M. Maybury, et al. (1999). Enterprise expert and knowledge discovery. Proceeings of the 8th International Conference on Human-Computer Interactions (HCI International 99), Munich, Germany.

Viavacqua, A. (1999). Agents for expertise location. Proceedings of the 1999 AAAI Spring Symposium on Intelligent Agents in Cyberspace, Stanford, CA.