wikademia>Eme at 04:57, 11 August 2010

2010-08-11T04:57:51Z

← Older revision		Revision as of 04:57, 11 August 2010
Line 1:		Line 1:
	~~{{linkfarm}}~~
	'''Named entity recognition''' (NER) (also known as '''entity identification''' and '''entity extraction''') is a subtask of [[information extraction]] that seeks to locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.		'''Named entity recognition''' (NER) (also known as '''entity identification''' and '''entity extraction''') is a subtask of [[information extraction]] that seeks to locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.

wikademia>Eme: /* References */

2010-05-18T19:24:33Z

References

← Older revision		Revision as of 19:24, 18 May 2010
Line 150:		Line 150:
	[[Category:Computational linguistics]]		[[Category:Computational linguistics]]
	[[Category:Tasks of Natural language processing]]		[[Category:Tasks of Natural language processing]]

	~~[[es:Reconocimiento de nombres de entidades]]~~
	~~[[fr:Entités nommées]]~~
	~~[[ja:固有表現抽出]]~~

wikademia>Dr. Eme: http://en.wikipedia.org/w/index.php?title=Named_entity_recognition&action=edit

2009-09-26T15:34:35Z

http://en.wikipedia.org/w/index.php?title=Named_entity_recognition&action=edit

New page

{{linkfarm}}
'''Named entity recognition''' (NER) (also known as '''entity identification''' and '''entity extraction''') is a subtask of [[information extraction]] that seeks to locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.

Most research on NER systems has been structured as taking an unannotated block of text, such as this one:

:''Jim bought 300 shares of Acme Corp. in 2006.''

And producing an annotated block of text, such as this one:

:'''''<ENAMEX TYPE="PERSON">'''Jim'''</ENAMEX>''' bought '''<NUMEX TYPE="QUANTITY">'''300'''</NUMEX>''' shares of '''<ENAMEX TYPE="ORGANIZATION">'''Acme Corp.'''</ENAMEX>''' in '''<TIMEX TYPE="DATE">'''2006'''</TIMEX>'''''.

In this example, the annotations have been done using so-called '''ENAMEX''' tags that were developed for the [[Message Understanding Conference]] in the 1990s.

[http://aclweb.org/aclwiki/index.php?title=Named_Entity_Recognition_%28State_of_the_art%29 State-of-the-art NER systems] produce near-human performance. For example, the best system entering [http://www.itl.nist.gov/iad/894.02/related_projects/muc/proceedings/muc_7_toc.html MUC-7] scored 93.39% of [[Information_retrieval#F-measure|f-measure]] while human annotators scored 97.60% and 96.95%. These results indicate the algorithms had roughly twice the error rate (6.61%) as human annotators (2.40% and 3.05%).

==Approaches==
NER systems have been created that use linguistic [[formal grammar|grammar]]-based techniques as well as [[statistical model]]s. Hand-crafted grammar-based systems typically obtain better results, but at the cost of months of work by experienced [[Computational linguistics|computational linguists]]. Statistical NER systems typically require a large amount of manually [[annotation|annotated]] training data.

==Problem Domains==
Research indicates that NER systems developed for one domain do not typically perform well on other domains.<ref>Poibeau, Thierry and Kosseim, L. (2001) Proper Name Extraction from Non-Journalistic Texts. Proc. Computational Linguistics in the Netherlands. </ref>.

Early work in NER systems in the 1990s was aimed primarily at extraction from journalistic articles. Attention then turned to processing of military dispatches and reports. Since about 1998, there has been a great deal of interest in entity identification in the [[molecular biology]], [[bioinformatics]], and medical [[natural language processing]] communities. The most common entity of interest in that domain has been names of genes and gene products.

==Named Entity types==

In the expression ''named entity'', the word ''named'' restricts the task to those entities for which one or many [[rigid designator]]s, as defined by [[Saul Kripke|Kripke]], stands for the referent. For instance, the ''automotive company created by Henry Ford in 1903'' is referred to as ''Ford'' or ''Ford Motor Company''. Rigid designators include proper names as well as certain natural kind terms like biological species and substances.

There is a general agreement to include [[temporal expressions]] and some numerical expressions (i.e., money, percentages, etc.) as instances of named entities in the context of the NER task. While some instances of these types are good examples of rigid designators (e.g., the year 2001) there are also many invalid ones (e.g., I take my vacations in “June”). In the first case, the year ''2001'' refers to the ''2001st year of the Gregorian calendar''. In the second case, the month ''June'' may refer to the month of an undefined year (''past June'', ''next June'', ''June 2020'', etc.). It is arguable that the named entity definition is loosened in such cases for practical reasons.

At least two [[Hierarchy|hierarchies]] of named entity types have been proposed in the literature. [[BBN Technologies|BBN]] categories [http://www.ldc.upenn.edu/Catalog/docs/LDC2005T33/BBN-Types-Subtypes.html], proposed in 2002, is used for [[Question Answering]] and consists of 29 types and 64 subtypes. Sekine's extended hierarchy [http://nlp.cs.nyu.edu/ene/], proposed in 2002, is made of 200 subtypes.

==NER Evaluation Forums==
Evaluation of NER systems is critical to scientific progress of this field.

Most evaluation of these systems has been performed at conferences or contests put on by government organizations, sometimes acting in concert with contractors or academics.
{| border="0" cellpadding="2" cellspacing="2" align="top"
|-style="background:#bfbfbf; font-weight:bold"
!width="350"|Conference
!Acronym
!Language(s)
!Year(s)
!Sponsor
!Archive Site
|-
|[[Message Understanding Conference]]
|MUC
|English
|1987-1999
|[[DARPA]]
| [http://www.itl.nist.gov/iaui/894.02/related_projects/muc/index.html]
|-
|[[Multilingual Entity Task Conference]]
|MET
|Chinese and Japanese
|1998
|US
|[http://www-nlpir.nist.gov/related_projects/tipster/met.htm]
|-
|[[Automatic Content Extraction Program]]
|ACE
|English
|2000-
|NIST
|[http://www.nist.gov/speech/tests/ace/]
|-
|Evaluation contest for named entity recognizers in Portuguese
|HAREM
|Portuguese
|2004-2008
|[http://www.linguateca.pt Linguateca]
|[http://www.linguateca.pt/HAREM/]
|-
| Information Retrieval and Extraction Exercise
| IREX
| Japanese
| 1998-1999
|
| [http://portal.acm.org/citation.cfm?id=992814&dl=acm&coll=&CFID=15151515&CFTOKEN=6184618]
|-
| ACL Special Interest Group in Chinese
| SIGHan
| Chinese
| 2006
|
|[http://sighan.cs.uchicago.edu/bakeoff2006/]
|}

==External links==
===Conferences===
*[http://www.cnts.ua.ac.be/conll/ Conference on Computational Natural Language Learning]
* [[Text Retrieval Conference|Text REtrieval Conference (TREC)]]
*[http://www.lrec-conf.org/ International Conference on Language Resources and Evaluation (LREC)]

===Datasets and hierarchies===
*[http://www.cs.technion.ac.il/~gabr/resources/data/ne_datasets.html Tagged datasets for named entity recognition tasks]
*[http://www.ldc.upenn.edu/Catalog/docs/LDC2005T33/BBN-Types-Subtypes.html BBN named entity type hierarchy]
*[http://nlp.cs.nyu.edu/ene/ Sekine's extended named entity hierarchy]

===NER Software===
====Open source====
*[ftp://ftp.ncbi.nlm.nih.gov/pub/tanabe/AbGene AbGene] Biomedical named entity recognizer.
*[http://www.cs.wisc.edu/~bsettles/abner/ ABNER] Biomedical named entity recognizer.
*[http://bcsp1.iis.sinica.edu.tw/aiiagmt/ AIIAGMT] Biomedical named entity recognizer.
*[http://gate.ac.uk/ie/annie.html ANNIE] Information extraction package (a [http://gate.ac.uk/ GATE] component) with NER capabilities.
*[http://balie.sourceforge.net/ Balie] Baseline implementation of named entity recognition.
*[http://kmi.open.ac.uk/people/jianhan/ESpotter/ ESpotter] A domain and user adaptation approach for named entity recognition on the Web.
*[http://garraf.epsevg.upc.es/freeling/ FreeLing] An open source language analysis tool suite. See the [http://garraf.epsevg.upc.es/freeling/demo.php online demo].
*[http://www.hgc.ims.u-tokyo.ac.jp/service/tooldoc/KeX/intro.html KeX] A simple Knowledge EXtraction tool.
*[http://minorthird.sourceforge.net/ MinorThird] Collection of Java classes for storing text, annotating text, and learning to extract entities and categorize text.
*[http://bionlp.sourceforge.net MutationFinder] An information extraction system for extracting descriptions of point mutations from free text.
*[http://isoft.postech.ac.kr/Research/Bio/bio.html#Requirements POSBIOTM/W] NER client tool that enables users to automatically annotate biomedical-related entities.
*[http://www.digitalsonata.com/demo.aspx?component=morphoLogic Carabao MorphoLogic] Mixed dictionary-based and heuristics-based named entity recognition for single words only.
*[http://mallet.cs.umass.edu/ Mallet] Java-based package. Mainly interest by [[Conditional random field|CRF]] implementation. Also contains classification and topic models.
*[http://l2r.cs.uiuc.edu/~cogcomp/software.php LBJ Named Entity Tagger] State of the art Named Entity Recognizer from the Cognitive Computation Group at the University of Illinois Urbana Champaign.
*[http://sourceforge.net/projects/oscar3-chem/ OSCAR] Chemical Entity Recogniser .
*[http://xldb.di.fc.ul.pt/Rembrandt/ Rembrandt], a Named Entity Recognition tool and web service for Portuguese and English.
*[http://www.nada.kth.se/iplab/hlt/swenam/index-eng.html SweNam - a Named Entity Recognizer for Swedish online].
*Apache [[UIMA]] is an architecture that includes entity extraction in its components.
*[[OpenPipeline]] is an open source framework for processing documents with entity extraction as an available stage.

====Dual license (free and commercial version)====
*[http://www.alchemyapi.com AlchemyAPI] Named entity extraction & disambiguation, text categorization, and text mining service for multiple languages.
*[http://www.alias-i.com/lingpipe LingPipe] Java Natural Language Processing software that includes a trainable named-entity extraction framework with first-best, n-best and confidence-ranked-by-entity output. Models available for various languages and genres. See the [http://www.alias-i.com/lingpipe/web/demos.html online demos].
*[[Calais (Reuters Product)|Calais]] Named entity, fact and event extraction web service provided by [[Reuters]]
*[[Cypher transcoder|Cypher]] A NLP framework which includes a named-entity processor which converts NE's into [[FOAF]] instances. Demo available [http://demo.monrai.com online].
*[http://nlp.stanford.edu/ner/index.shtml Stanford NER] NER client tool based on Java. Uses CRF algorithm.

====Commercial====
*[http://www.lockheedmartin.com/products/AeroText/index.html AeroText (TM)| Lockheed Martin] An extensible, commercial natural language processing toolkit for entity, relationship, and event extraction.
*[http://www.alethes.it/ Alethes] Commercial Text Analytics Solution, entity extraction, information extraction, categorization, clustering, sentiment analysis for 8 different language.
*[http://www.basistech.com Basis Technology]'s Rosette Entity Extractor (REX)
*[http://www.bbn.com BBN Technologies]'s IdentiFinder and IdentiFinder Text Suite
*[http://www.clearforest.com/ ClearForest] Commercial natural language processing toolkit that includes NER.
*[http://www.cortex-intelligence.com/english Cortex Intelligence] Commercial competitive intelligence web software that use entity extraction technology.
*[http://www.expertsystem.net/ Expert System] Commercial natural language processing, entity extraction, categorization rules and domain construction tool sets.
*[[Inxight]]: Natural language processing, entity extraction and fact extraction in 32 languages.
*[http://www.isys-search.com ISYS] An enterprise search product which includes automatic entity recognition
*[http://www.janyainc.com Janya, Inc.] Provide of text analytics for English and Chinese.
*[[Language Computer Corporation|LCC]] [http://www.languagecomputer.com/index.php?page=cicerolite CiceroLite] Commercial and state-of-the-art extraction suite which includes entity extraction for English, Chinese and Arabic.
*[http://www.megaputer.com/ PolyAnalyst] Commercial natural language processing suite with entity extraction tools
*[http://www.netowl.com/ SRA NetOwl] Commercial and state-of-the-art recognizer in its class (rule and statistical based) covering many scripts and including highly inflected languages such as Arabic.
*[http://www.nogacom.com Nogacom] Commercial mulilingual business entity extraction, NLP and classification for information management and information governance.
*[[Teragram]] multilingual entity extraction
*[http://www.trifeed.com/ Trifeed Ltd.] Trifeed is a research and development software company operating in the field of text analysis and information extraction.
*[http://www.alethes.it/ OpenEyes] Commercial NLP suite with entity and information extraction engine and resource

==References==
{{reflist}}

[[Category:Computational linguistics]]
[[Category:Tasks of Natural language processing]]

[[es:Reconocimiento de nombres de entidades]]
[[fr:Entités nommées]]
[[ja:固有表現抽出]]

Named entity recognition - Revision history

wikademia>Eme at 04:57, 11 August 2010

wikademia>Eme: /* References */

wikademia>Dr. Eme: http://en.wikipedia.org/w/index.php?title=Named_entity_recognition&action=edit