IdeaWazaWiki - User contributions [en]

Expertise finding

2015-09-29T13:13:03Z

77.174.22.70: /* Interesting expertise systems over the years */

{{multiple issues|
{{original research|date=June 2015}}
{{refimprove|date=June 2015}}
{{cleanup|date=November 2010}}{{External links|date=January 2012}}
}}
'''Expertise finding''' is the use of tools for finding and assessing individual [[expertise]], with particular focus on scientific expertise.

== Importance of expertise ==

It can be argued that human expertise is more valuable than capital, means of production or intellectual property. Contrary to expertise, all other aspects of capitalism are now relatively generic: access to capital is global, as is access to means of production for many areas of manufacturing. [[Intellectual property]] can be similarly licensed. Furthermore, expertise finding is also a key aspect of [[institutional memory]], as without its experts an institution is effectively decapitated. However, finding and “licensing” expertise, the key to the effective use of these resources, remain much harder, starting with the very first step: finding expertise that you can trust.

Until very recently, finding expertise required a mix of individual, social and collaborative practices, a haphazard process at best. Mostly, it involved contacting individuals one trusts and asking them for referrals, while hoping that one’s judgment about those individuals is justified and that their answers are thoughtful.

In the last fifteen years, a class of [[knowledge management]] software has emerged to facilitate and improve the quality of expertise finding, termed “expertise locating systems”. These software range from [[Social network service|social networking systems]] to [[knowledge base]]s. Some software, like those in the social networking realm, rely on users to connect each other, thus using social filtering to act as [[Recommender system|“recommender systems”]].

At the other end of the spectrum are specialized [[knowledge base]]s that rely on experts to populate a specialized type of [[database]] with their self-determined areas of expertise and contributions, and do not rely on user recommendations. Hybrids that feature expert-populated content in conjunction with user recommendations also exist, and are arguably more valuable for doing so.

Still other expertise knowledge bases rely strictly on external manifestations of expertise, herein termed “gated objects”, e.g., [[citation impact]]s for scientific papers or [[data mining]] approaches wherein many of the work products of an expert are collated. Such systems are more likely to be free of user-introduced biases (e.g., [http://researchscorecard.com/ ResearchScorecard] ), though the use of computational methods can introduce other biases.

Examples of the systems outlined above are listed in Table 1.

'''Table 1: A classification of expertise location systems'''

{| class="wikitable" border="1"
|-
! Type
! Application domain
! Data source
! Examples
|-
| Social networking
| Professional networking
| User-generated
|
* [[LinkedIn]]
|-
| [[Scientific literature]]
| Identifying publications with strongest research impact
| Third-party generated
|
* [[Science Citation Index]] (Thomson Reuters)[http://www.thomsonreuters.com/products_services/science/science_products/a-z/science_citation_index]
|-
| [[Scientific literature]]
| Expertise search
| Software
|
* [[Arnetminer]][http://arnetminer.org]
|-
| Knowledge base
| Private expertise database
| User-Generated
|
* [http://www.mitre.org/news/the_edge/june_98/third.html MITRE Expert Finder] (MITRE Corporation)
* MIT ExpertFinder (ref. 3)
* Decisiv Search Matters & Expertise ([[Recommind (software company)|Recommind]], Inc.)
* [https://skillhive.com Skillhive] (Intunex)
* [[Tacit Software]] (Oracle Corporation)
* [http://www.guruscan.nl GuruScan] (GuruScan Social Expert Guide)
|-
| Knowledge base
| Publicly accessible expertise database
| User-generated
|
* [http://expertisefinder.com/ Expertise Finder]<ref>http://expertisefinder.com/</ref>
* [[Community of Science]] Expertise [http://expertise.cos.com]
* [[ResearcherID]] (Thomson Reuters)[http://www.thomsonreuters.com/products_services/scientific/ResearcherID]
|-
| Knowledge base
| Private expertise database
| Third party-generated
|
* [http://www.mitre.org/news/the_edge/june_98/third.html MITRE Expert Finder] (MITRE Corporation)
* MIT ExpertFinder (ref. 3)
* MindServer Expertise ([[Recommind]], Inc.)
* Tacit Software
|-
| Knowledge base
| Publicly accessible expertise database
| Third party-generated
|
* [http://researchscorecard.com ResearchScorecard] (ResearchScorecard Inc.)
* [http://authoratory.com/ authoratory.com]
* [http://biomedexperts.com BiomedExperts] (Collexis Holdings Inc.)
* [http://www.hcarknowledgemesh.com/ KnowledgeMesh] (Hershey Center for Applied Research)
* [http://med.stanford.edu/profiles/ Community Academic Profiles] (Stanford School of Medicine)
* [http://researchcrossroads.org ResearchCrossroads.org] (Innolyst, Inc.)
|-
| Blog [[search engine]]s
|
| Third party-generated
|
* [[Technorati]] [http://technorati.com/]
|}

== Technical problems ==
A number of interesting problems follow from the use of expertise finding systems:

* The matching of questions from non-expert to the database of existing expertise is inherently difficult, especially when the database does not store the requisite expertise. This problem grows even more acute with increasing ignorance on the part of the non-expert due to typical search problems involving use of keywords to search unstructured data that are not semantically normalized, as well as variability in how well an expert has set up their descriptive content pages. Improved question matching is one reason why third-party semantically normalized systems such as [http://researchscorecard.com ResearchScorecard] and [[BiomedExperts]] should be able to provide better answers to queries from non-expert users.
* Avoiding expert-fatigue due to too many questions/requests from users of the system (ref. 1).
* Finding ways to avoid “gaming” of the system to reap unjustified expertise [[credibility]].

== Expertise ranking ==

Means of classifying and ranking expertise (and therefore experts) become essential if the number of experts returned by a query is greater than a handful. This raises the following social problems associated with such systems:

* How can expertise be assessed objectively? Is that even possible?
* What are the consequences of relying on unstructured social assessments of expertise, such as user recommendations?
* How does one distinguish [[Authority|''authoritativeness'']] as a proxy metric of expertise from simple ''popularity'', which is often a function of one's ability to express oneself coupled with a good social sense?
* What are the potential consequences of the social or professional stigma associated with the use of an authority ranking, such as used in [http://technorati.com Technorati] and [http://researchscorecard.com ResearchScorecard])?

== Sources of data for assessing expertise ==
Many types of data sources have been used to infer expertise. They can be broadly categorized based on whether they measure "raw" contributions provided by the expert, or whether some sort of filter is applied to these contributions.

Unfiltered data sources that have been used to assess expertise, in no particular ranking order:

* user recommendations
* help desk tickets: what the problem was and who fixed it
* e-mail traffic between users
* documents, whether private or on the web, particularly publications
* user-maintained web pages
* reports (technical, marketing, etc.)

Filtered data sources, that is, contributions that require approval by third parties (grant committees, referees, patent office, etc.) are particularly valuable for measuring expertise in a way that minimizes biases that follow from popularity or other social factors:

* [[patent]]s, particularly if issued
* scientific publications
* issued grants (failed grant proposals are rarely know beyond the authors)
* [[clinical trial]]s
* product launches
* pharmaceutical drugs

== Approaches for creating expertise content ==
* Manual, either by experts themselves (e.g., LinkedIn and [https://skillhive.com Skillhive]) or by a curator ([http://expertisefinder.com/ Expertise Finder])
* Automated, e.g., using [[software agent]]s (e.g., MIT's [http://web.media.mit.edu/~lieber/Lieberary/Expert-Finder/Expert-Finder-Intro.html ExpertFinder] and the [http://wiki.foaf-project.org/ExpertFinder ExpertFinder] initiative) or a combination of agents and human curation (e.g., [http://researchscorecard.com/ ResearchScorecard] )

== Interesting expertise systems over the years ==
In no particular order...

* [http://www.guruscan.nl/ GuruScan]
* Autonomy's IDOL
* AskMe
* [http://expertisefinder.com/ Expertise Finder]
* Tacit Knowledge Systems' ActiveNet
* Triviumsoft's SEE-K
* MIT’s [http://web.media.mit.edu/~lieber/Lieberary/Expert-Finder/Expert-Finder-Intro.html ExpertFinder] (ref 3)
* MITRE’s (ref 1) [http://www.mitre.org/news/the_edge/june_98/third.html Expert Finder]
* MITRE’s XpertNet
* Arnetminer (ref 2)
* Dataware II Knowledge Directory
* Thomson’s tool
* Hewlett-Packard’s CONNEX
* Microsoft’s SPUD project
* [http://www.xperscore.com Xperscore]
* [http://intunex.fi/skillhive/ Skillhive]

== Conferences ==
# [http://expertfinder.info/pickme2008 The ExpertFinder Initiative]

== References ==
{{Reflist}}

# Ackerman, Mark and McDonald, David (1998) "Just Talk to Me: A Field Study of Expertise Location" ''Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work''.
# Hughes, Gareth and Crowder, Richard (2003) "Experiences in designing highly adaptable expertise finder systems" ''Proceedings of the DETC Conference 2003''.
# Maybury, M., D’Amore, R., House, D. (2002). "Awareness of organizational expertise." ''International Journal of Human-Computer Interaction'' '''14'''(2): 199-217.
# Maybury, M., D’Amore, R., House, D. (2000). Automating Expert Finding. ''International Journal of Technology Research Management.'' 43(6): 12-15.
# Maybury, M., D’Amore, R, and House, D. December (2001). Expert Finding for Collaborative Virtual Environments. ''Communications of the ACM 14''(12): 55-56. In Ragusa, J. and Bochenek, G. (eds). Special Section on Collaboration Virtual Design Environments.
# Maybury, M., D’Amore, R. and House, D. (2002). Automated Discovery and Mapping of Expertise. In Ackerman, M., Cohen, A., Pipek, V. and Wulf, V. (eds.). ''Beyond Knowledge Management: Sharing Expertise.'' Cambridge: MIT Press.
# Mattox, D., M. Maybury, ''et al.'' (1999). "Enterprise expert and knowledge discovery". ''Proceedings of the 8th International Conference on Human-Computer Interactions (HCI International 99)'', Munich, Germany.
# Tang, J., Zhang J., Yao L., Li J., Zhang L. and Su Z.(2008) "ArnetMiner: extraction and mining of academic social networks" ''Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining''.
# Viavacqua, A. (1999). "Agents for expertise location". ''Proceedings of the 1999 AAAI Spring Symposium on Intelligent Agents in Cyberspace'', Stanford, CA.

[[Category:Evaluation methods]]
[[Category:Metrics]]
[[Category:Analysis]]
[[Category:Impact assessment]]
[[Category:Intellectual works]]
[[Category:Knowledge sharing]]
[[Category:Library science]]
[[Category:Information retrieval genres]]
[[Category:Science studies]]

Expertise finding

2015-09-29T13:10:11Z

77.174.22.70: Guruscan added in Private Expertise database / User generated

{{multiple issues|
{{original research|date=June 2015}}
{{refimprove|date=June 2015}}
{{cleanup|date=November 2010}}{{External links|date=January 2012}}
}}
'''Expertise finding''' is the use of tools for finding and assessing individual [[expertise]], with particular focus on scientific expertise.

== Importance of expertise ==

It can be argued that human expertise is more valuable than capital, means of production or intellectual property. Contrary to expertise, all other aspects of capitalism are now relatively generic: access to capital is global, as is access to means of production for many areas of manufacturing. [[Intellectual property]] can be similarly licensed. Furthermore, expertise finding is also a key aspect of [[institutional memory]], as without its experts an institution is effectively decapitated. However, finding and “licensing” expertise, the key to the effective use of these resources, remain much harder, starting with the very first step: finding expertise that you can trust.

Until very recently, finding expertise required a mix of individual, social and collaborative practices, a haphazard process at best. Mostly, it involved contacting individuals one trusts and asking them for referrals, while hoping that one’s judgment about those individuals is justified and that their answers are thoughtful.

In the last fifteen years, a class of [[knowledge management]] software has emerged to facilitate and improve the quality of expertise finding, termed “expertise locating systems”. These software range from [[Social network service|social networking systems]] to [[knowledge base]]s. Some software, like those in the social networking realm, rely on users to connect each other, thus using social filtering to act as [[Recommender system|“recommender systems”]].

At the other end of the spectrum are specialized [[knowledge base]]s that rely on experts to populate a specialized type of [[database]] with their self-determined areas of expertise and contributions, and do not rely on user recommendations. Hybrids that feature expert-populated content in conjunction with user recommendations also exist, and are arguably more valuable for doing so.

Still other expertise knowledge bases rely strictly on external manifestations of expertise, herein termed “gated objects”, e.g., [[citation impact]]s for scientific papers or [[data mining]] approaches wherein many of the work products of an expert are collated. Such systems are more likely to be free of user-introduced biases (e.g., [http://researchscorecard.com/ ResearchScorecard] ), though the use of computational methods can introduce other biases.

Examples of the systems outlined above are listed in Table 1.

'''Table 1: A classification of expertise location systems'''

{| class="wikitable" border="1"
|-
! Type
! Application domain
! Data source
! Examples
|-
| Social networking
| Professional networking
| User-generated
|
* [[LinkedIn]]
|-
| [[Scientific literature]]
| Identifying publications with strongest research impact
| Third-party generated
|
* [[Science Citation Index]] (Thomson Reuters)[http://www.thomsonreuters.com/products_services/science/science_products/a-z/science_citation_index]
|-
| [[Scientific literature]]
| Expertise search
| Software
|
* [[Arnetminer]][http://arnetminer.org]
|-
| Knowledge base
| Private expertise database
| User-Generated
|
* [http://www.mitre.org/news/the_edge/june_98/third.html MITRE Expert Finder] (MITRE Corporation)
* MIT ExpertFinder (ref. 3)
* Decisiv Search Matters & Expertise ([[Recommind (software company)|Recommind]], Inc.)
* [https://skillhive.com Skillhive] (Intunex)
* [[Tacit Software]] (Oracle Corporation)
* [http://www.guruscan.nl GuruScan] (GuruScan Social Expert Guide)
|-
| Knowledge base
| Publicly accessible expertise database
| User-generated
|
* [http://expertisefinder.com/ Expertise Finder]<ref>http://expertisefinder.com/</ref>
* [[Community of Science]] Expertise [http://expertise.cos.com]
* [[ResearcherID]] (Thomson Reuters)[http://www.thomsonreuters.com/products_services/scientific/ResearcherID]
|-
| Knowledge base
| Private expertise database
| Third party-generated
|
* [http://www.mitre.org/news/the_edge/june_98/third.html MITRE Expert Finder] (MITRE Corporation)
* MIT ExpertFinder (ref. 3)
* MindServer Expertise ([[Recommind]], Inc.)
* Tacit Software
|-
| Knowledge base
| Publicly accessible expertise database
| Third party-generated
|
* [http://researchscorecard.com ResearchScorecard] (ResearchScorecard Inc.)
* [http://authoratory.com/ authoratory.com]
* [http://biomedexperts.com BiomedExperts] (Collexis Holdings Inc.)
* [http://www.hcarknowledgemesh.com/ KnowledgeMesh] (Hershey Center for Applied Research)
* [http://med.stanford.edu/profiles/ Community Academic Profiles] (Stanford School of Medicine)
* [http://researchcrossroads.org ResearchCrossroads.org] (Innolyst, Inc.)
|-
| Blog [[search engine]]s
|
| Third party-generated
|
* [[Technorati]] [http://technorati.com/]
|}

== Technical problems ==
A number of interesting problems follow from the use of expertise finding systems:

* The matching of questions from non-expert to the database of existing expertise is inherently difficult, especially when the database does not store the requisite expertise. This problem grows even more acute with increasing ignorance on the part of the non-expert due to typical search problems involving use of keywords to search unstructured data that are not semantically normalized, as well as variability in how well an expert has set up their descriptive content pages. Improved question matching is one reason why third-party semantically normalized systems such as [http://researchscorecard.com ResearchScorecard] and [[BiomedExperts]] should be able to provide better answers to queries from non-expert users.
* Avoiding expert-fatigue due to too many questions/requests from users of the system (ref. 1).
* Finding ways to avoid “gaming” of the system to reap unjustified expertise [[credibility]].

== Expertise ranking ==

Means of classifying and ranking expertise (and therefore experts) become essential if the number of experts returned by a query is greater than a handful. This raises the following social problems associated with such systems:

* How can expertise be assessed objectively? Is that even possible?
* What are the consequences of relying on unstructured social assessments of expertise, such as user recommendations?
* How does one distinguish [[Authority|''authoritativeness'']] as a proxy metric of expertise from simple ''popularity'', which is often a function of one's ability to express oneself coupled with a good social sense?
* What are the potential consequences of the social or professional stigma associated with the use of an authority ranking, such as used in [http://technorati.com Technorati] and [http://researchscorecard.com ResearchScorecard])?

== Sources of data for assessing expertise ==
Many types of data sources have been used to infer expertise. They can be broadly categorized based on whether they measure "raw" contributions provided by the expert, or whether some sort of filter is applied to these contributions.

Unfiltered data sources that have been used to assess expertise, in no particular ranking order:

* user recommendations
* help desk tickets: what the problem was and who fixed it
* e-mail traffic between users
* documents, whether private or on the web, particularly publications
* user-maintained web pages
* reports (technical, marketing, etc.)

Filtered data sources, that is, contributions that require approval by third parties (grant committees, referees, patent office, etc.) are particularly valuable for measuring expertise in a way that minimizes biases that follow from popularity or other social factors:

* [[patent]]s, particularly if issued
* scientific publications
* issued grants (failed grant proposals are rarely know beyond the authors)
* [[clinical trial]]s
* product launches
* pharmaceutical drugs

== Approaches for creating expertise content ==
* Manual, either by experts themselves (e.g., LinkedIn and [https://skillhive.com Skillhive]) or by a curator ([http://expertisefinder.com/ Expertise Finder])
* Automated, e.g., using [[software agent]]s (e.g., MIT's [http://web.media.mit.edu/~lieber/Lieberary/Expert-Finder/Expert-Finder-Intro.html ExpertFinder] and the [http://wiki.foaf-project.org/ExpertFinder ExpertFinder] initiative) or a combination of agents and human curation (e.g., [http://researchscorecard.com/ ResearchScorecard] )

== Interesting expertise systems over the years ==
In no particular order...

* Autonomy's IDOL
* AskMe
* [http://expertisefinder.com/ Expertise Finder]
* Tacit Knowledge Systems' ActiveNet
* Triviumsoft's SEE-K
* MIT’s [http://web.media.mit.edu/~lieber/Lieberary/Expert-Finder/Expert-Finder-Intro.html ExpertFinder] (ref 3)
* MITRE’s (ref 1) [http://www.mitre.org/news/the_edge/june_98/third.html Expert Finder]
* MITRE’s XpertNet
* Arnetminer (ref 2)
* Dataware II Knowledge Directory
* Thomson’s tool
* Hewlett-Packard’s CONNEX
* Microsoft’s SPUD project
* [http://www.xperscore.com Xperscore]
* [http://intunex.fi/skillhive/ Skillhive]

== Conferences ==
# [http://expertfinder.info/pickme2008 The ExpertFinder Initiative]

== References ==
{{Reflist}}

# Ackerman, Mark and McDonald, David (1998) "Just Talk to Me: A Field Study of Expertise Location" ''Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work''.
# Hughes, Gareth and Crowder, Richard (2003) "Experiences in designing highly adaptable expertise finder systems" ''Proceedings of the DETC Conference 2003''.
# Maybury, M., D’Amore, R., House, D. (2002). "Awareness of organizational expertise." ''International Journal of Human-Computer Interaction'' '''14'''(2): 199-217.
# Maybury, M., D’Amore, R., House, D. (2000). Automating Expert Finding. ''International Journal of Technology Research Management.'' 43(6): 12-15.
# Maybury, M., D’Amore, R, and House, D. December (2001). Expert Finding for Collaborative Virtual Environments. ''Communications of the ACM 14''(12): 55-56. In Ragusa, J. and Bochenek, G. (eds). Special Section on Collaboration Virtual Design Environments.
# Maybury, M., D’Amore, R. and House, D. (2002). Automated Discovery and Mapping of Expertise. In Ackerman, M., Cohen, A., Pipek, V. and Wulf, V. (eds.). ''Beyond Knowledge Management: Sharing Expertise.'' Cambridge: MIT Press.
# Mattox, D., M. Maybury, ''et al.'' (1999). "Enterprise expert and knowledge discovery". ''Proceedings of the 8th International Conference on Human-Computer Interactions (HCI International 99)'', Munich, Germany.
# Tang, J., Zhang J., Yao L., Li J., Zhang L. and Su Z.(2008) "ArnetMiner: extraction and mining of academic social networks" ''Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining''.
# Viavacqua, A. (1999). "Agents for expertise location". ''Proceedings of the 1999 AAAI Spring Symposium on Intelligent Agents in Cyberspace'', Stanford, CA.

[[Category:Evaluation methods]]
[[Category:Metrics]]
[[Category:Analysis]]
[[Category:Impact assessment]]
[[Category:Intellectual works]]
[[Category:Knowledge sharing]]
[[Category:Library science]]
[[Category:Information retrieval genres]]
[[Category:Science studies]]