Hypnotherapy For Smoking Cessation

From Wikipedia, the free encyclopedia

Natural Language processing (NLP) is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages. [1] In theory, natural-language processing is a very attractive method of human-computer interaction. Natural-language understanding is sometimes referred to as an AI-complete problem, because natural-language recognition seems to require extensive knowledge about the outside world and the ability to manipulate it.

NLP has significant overlap with the field of computational linguistics, and is often considered a sub-field of artificial intelligence.

• 1 History
• 2 Tasks and limitations
• 3 Subproblems
• 4 Statistical NLP
• 5 Major tasks in NLP
• 6 Concrete problems
  • 6.1 Part of speech tagging and Word sense disambiguation
  • 6.2 Parse tree disambiguation
  • 6.3 Anaphora resolution
  • 6.4 Intonation
• 7 Evaluation of natural language processing
  • 7.1 Objectives
  • 7.2 Short history of evaluation in NLP
  • 7.3 Different types of evaluation
  • 7.4 Shared tasks (Campaigns)
• 8 Standardization in NLP
• 9 Journals
• 10 Organizations and conferences
  • 10.1 Associations
  • 10.2 Conferences
• 11 Software tools
• 12 See also
• 13 References
  • 13.1 Related academic articles
• 14 External links
  • 14.1 Resources
  • 14.2 Organizations

[edit] History

The history of NLP generally starts in the 1950s, although work can be found from earlier periods. In 1950, Alan Turing published his famous article “Computing Machinery and Intelligence”[2] which proposed what is now called the Turing test as a criterion of intelligence. This criterion depends on the ability of a computer program to impersonate a human in a real-time written conversation with a human judge, sufficiently well that the judge is unable to distinguish reliably – on the basis of the conversational content alone – between the program and a real human. The Georgetown experiment in 1954 involved fully automatic translation of more than sixty Russian sentences into English. The authors claimed that within three or five years, machine translation would be a solved problem.[3]

In the 60′s, SHRDLU, a natural language system working in restricted “blocks worlds” with restricted vocabularies, worked extremely well, leading researchers to great optimism.

However, the real progress was much slower, and after the ALPAC report in 1966, which found that ten years long research had failed to fulfill the expectations, the funding was dramatically reduced.

ELIZA was a simulation of a Rogerian psychotherapist, was written by Joseph Weizenbaum between 1964 to 1966. Using almost no information about human thought or emotion, ELIZA sometimes provided a startlingly human-like interaction. When the “patient” exceeded the very small knowledge base, ELIZA might provide a generic response, for example, responding to “My head hurts” with “Why do you say your head hurts?”.

During the 70′s many programmers began to write ‘conceptual ontologies’, which structured real-word information into computer-understandable data: MARGIE (Schank, 1975), SAM (Cullingford, 1978), PAM (Wilensky, 1978), TaleSpin (Meehan, 1976), QUALM (Lehnert, 1977), Politics (Carbonell, 1979), Plot Units (Lehnert 1981).

During this time, many chatterbots were written including PARRY, Racter, and Jabberwacky.

Starting in the late 1980s, as computational power increased and became less expensive, more interest began to be shown in statistical models for machine translation.

[edit] Tasks and limitations

Although NLP may encompass both text and speech, work on speech processing has evolved into a separate field. Natural language generation systems convert information from computer databases into readable human language. Natural language understanding systems convert samples of human language into more formal representations such as parse trees or first-order logic structures that are easier for computer programs to manipulate. Many problems within NLP apply to both generation and understanding; for example, a computer must be able to model morphology (the structure of words) in order to understand an English sentence, and a model of morphology is also needed for producing a grammatically correct English sentence.

[edit] Subproblems Speech segmentation In most spoken languages, the sounds representing successive letters blend into each other, so the conversion of the analog signal to discrete characters can be a very difficult process. Also, in natural speech there are hardly any pauses between successive words; the location of those boundaries usually must take into account grammatical and semantic constraints, as well as the context. Text segmentation Some written languages like Chinese, Japanese and Thai do not have single-word boundaries either, so any significant text parsing usually requires the identification of word boundaries, which is often a non-trivial task. Part-of-speech tagging Word sense disambiguation Many words have more than one meaning; we have to select the meaning which makes the most sense in context. Syntactic ambiguity The grammar for natural languages is ambiguous, i.e. there are often multiple possible parse trees for a given sentence. Choosing the most appropriate one usually requires semantic and contextual information. Specific problem components of syntactic ambiguity include sentence boundary disambiguation. Imperfect or irregular input Foreign or regional accents and vocal impediments in speech; typing or grammatical errors, OCR errors in texts. Speech acts and plans A sentence can often be considered an action by the speaker. The sentence structure alone may not contain enough information to define this action. For instance, a question is sometimes the speaker requesting some sort of response from the listener. The desired response may be verbal, physical, or some combination. For example, “Can you pass the class?” is a request for a simple yes-or-no answer, while “Can you pass the salt?” is requesting a physical action to be performed. It is not appropriate to respond with “Yes, I can pass the salt,” without the accompanying action (although “No” or “I can’t reach the salt” would explain a lack of action). [edit] Statistical NLP

Statistical natural-language processing uses stochastic, probabilistic and statistical methods to resolve some of the difficulties discussed above, especially those which arise because longer sentences are highly ambiguous when processed with realistic grammars, yielding thousands or millions of possible analyses. Methods for disambiguation often involve the use of corpora and Markov models. Statistical NLP comprises all quantitative approaches to automated language processing, including probabilistic modeling, information theory, and linear algebra[4]. The technology for statistical NLP comes mainly from machine learning and data mining, both of which are fields of artificial intelligence that involve learning from data.

[edit] Major tasks in NLP

• Automatic summarization
• Foreign language reading aid
• Foreign language writing aid
• Information extraction
• Information retrieval (IR) – IR is concerned with storing, searching and retrieving information. It is a separate field within computer science (closer to databases), but IR relies on some NLP methods (for example, stemming). Some current research and applications seek to bridge the gap between IR and NLP.
• Machine translation – Automatically translating from one human language to another.
• Named entity recognition (NER) – Given a stream of text, determining which items in the text map to proper names, such as people or places. Although in English, named entities are marked with capitalized words, many other languages do not use capitalization to distinguish named entities.
• Natural language generation
• Natural language search
• Natural language understanding
• Optical character recognition
• anaphora resolution
• Query expansion
• Question answering – Given a human language question, the task of producing a human-language answer. The question may be a closed-ended (such as “What is the capital of Canada?”) or open-ended (such as “What is the meaning of life?”).
• Speech recognition – Given a sound clip of a person or people speaking, the task of producing a text dictation of the speaker(s). (The opposite of text to speech.)
• Spoken dialogue system
• Stemming
• Text simplification
• Text-to-speech
• Text-proofing

[edit] Concrete problems

Some concrete problems existing in the field include part-of-speech tag disambiguation (or tagging), word sense disambiguation, parse tree disambiguation, and Anaphora Resolution. While there are typically attempts to treat such problems individually, the problems can be shown to be highly intertwined. This section attempts to illustrate the complexities involved in some of these problems.

[edit] Part of speech tagging and Word sense disambiguation

An early AI goal was to give a computer the ability to parse natural language sentences into the type of sentence diagrams that grade-school children learn. One of the first such systems, developed in 1963 by Susumu Kuno of Harvard, was interesting in its revelation of the depth of ambiguity in the English language. Kuno asked his computerized parser what the sentence “Time flies like an arrow” means. In what has become a famous response[5], the computer replied that it was not quite sure. It might mean;

• • The common simile: time moves quickly just like an arrow does;
  • measure the speed of flies like you would measure that of an arrow (‘time’ being an imperative verb and ‘flies’ being the insects) – i.e. (You should) time flies as you would (time) an arrow;
  • measure the speed of flies like an arrow would – i.e. Time flies in the same way that an arrow would (time them);
  • measure the speed of flies that are like arrows – i.e. Time those flies that are like arrows;
  • A type of flying insect, “time-flies,” enjoys a single arrow (compare Fruit flies like a banana);

[edit] Parse tree disambiguation

English and several other languages don’t specify which word an adjective applies to. For example, in the string “pretty little girls’ school”.

• • Does the school look little?
  • Do the girls look little?
  • Do the girls look pretty?
  • Does the school look pretty?
  • Does the school look pretty little? (“pretty” here meaning “quite” as in the phrase “pretty ugly”)
  • Do the girls look pretty little? (same comparison applies)

This is essentially a problem of how to structure the sentence into a parse tree, and many factors may influence which is the correct tree.

[edit] Anaphora resolution

The sentences “We gave the monkeys the bananas because they were hungry” and “We gave the monkeys the bananas because they were over-ripe” have the same surface grammatical structure. However, the pronoun they refers to monkeys in one sentence and bananas in the other, and it is impossible to tell which without semantic knowledge (i.e., knowledge of the real-world properties of monkeys and bananas).

[edit] Intonation

NLP is often done as a form of text processing. Even speech input is typically transformed into a text string by a speech recognizer. However, there is much information included in the prosodic, or intonational, properties of an utterance.

An example of this is that a speaker will often imply additional information in spoken language by the placement of emphasis on individual words. The sentence “I never said she stole my money” demonstrates the importance emphasis can play in a sentence, and thus the inherent difficulty a natural language processor can have in parsing it. Depending on which word the speaker places the stress, this sentence could have several distinct meanings:

• • “I never said she stole my money” – Someone else said it, but I didn’t.
  • “I never said she stole my money” – I simply didn’t ever say it.
  • “I never said she stole my money” – I might have implied it in some way, but I never explicitly said it.
  • “I never said she stole my money” – I said someone took it; I didn’t say it was she.
  • “I never said she stole my money” – I just said she probably borrowed it.
  • “I never said she stole my money” – I said she stole someone else’s money.
  • “I never said she stole my money” – I said she stole something of mine, but not my money.

[edit] Evaluation of natural language processing [edit] Objectives

The goal of NLP evaluation is to measure one or more qualities of an algorithm or a system, in order to determine whether (or to what extent) the system answers the goals of its designers, or meets the needs of its users. Research in NLP evaluation has received considerable attention, because the definition of proper evaluation criteria is one way to specify precisely an NLP problem, going thus beyond the vagueness of tasks defined only as language understanding or language generation. A precise set of evaluation criteria, which includes mainly evaluation data and evaluation metrics, enables several teams to compare their solutions to a given NLP problem.

[edit] Short history of evaluation in NLP

The first evaluation campaign on written texts seems to be a campaign dedicated to message understanding in 1987 (Pallet 1998). Then, the Parseval/GEIG project compared phrase-structure grammars (Black 1991). A series of campaigns within Tipster project were realized on tasks like summarization, translation and searching (Hirschman 1998). In 1994, in Germany, the Morpholympics compared German taggers. Then, the Senseval and Romanseval campaigns were conducted with the objectives of semantic disambiguation. In 1996, the Sparkle campaign compared syntactic parsers in four different languages (English, French, German and Italian). In France, the Grace project compared a set of 21 taggers for French in 1997 (Adda 1999). In 2004, during the Technolangue/Easy project, 13 parsers for French were compared. Large-scale evaluation of dependency parsers were performed in the context of the CoNLL shared tasks in 2006 and 2007. In Italy, the evalita campaign was conducted in 2007 to compare various tools for Italian evalita web site. In France, within the ANR-Passage project (end of 2007), 10 parsers for French were compared passage web site.

Adda G., Mariani J., Paroubek P., Rajman M. 1999 L’action GRACE d’à valuation de l’assignation des parties du discours pour le franà ais. Langues vol-2
Black E., Abney S., Flickinger D., Gdaniec C., Grishman R., Harrison P., Hindle D., Ingria R., Jelinek F., Klavans J., Liberman M., Marcus M., Reukos S., Santoni B., Strzalkowski T. 1991 A procedure for quantitatively comparing the syntactic coverage of English grammars. DARPA Speech and Natural Language Workshop
Hirschman L. 1998 Language understanding evaluation: lessons learned from MUC and ATIS. LREC Granada
Pallet D.S. 1998 The NIST role in automatic speech recognition benchmark tests. LREC Granada

[edit] Different types of evaluation

Depending on the evaluation procedures, a number of distinctions are traditionally made in NLP evaluation.

• Intrinsic vs. extrinsic evaluation

Intrinsic evaluation considers an isolated NLP system and characterizes its performance mainly with respect to a gold standard result, pre-defined by the evaluators. Extrinsic evaluation, also called evaluation in use considers the NLP system in a more complex setting, either as an embedded system or serving a precise function for a human user. The extrinsic performance of the system is then characterized in terms of its utility with respect to the overall task of the complex system or the human user. For example, consider a syntactic parser that is based on the output of some new part of speech (POS) tagger. An intrinsic evaluation would run the POS tagger on some labelled data, and compare the system output of the POS tagger to the gold standard (correct) output. An extrinsic evaluation would run the parser with some other POS tagger, and then with the new POS tagger, and compare the parsing accuracy.

• Black-box vs. glass-box evaluation

Black-box evaluation requires one to run an NLP system on a given data set and to measure a number of parameters related to the quality of the process (speed, reliability, resource consumption) and, most importantly, to the quality of the result (e.g. the accuracy of data annotation or the fidelity of a translation). Glass-box evaluation looks at the design of the system, the algorithms that are implemented, the linguistic resources it uses (e.g. vocabulary size), etc. Given the complexity of NLP problems, it is often difficult to predict performance only on the basis of glass-box evaluation, but this type of evaluation is more informative with respect to error analysis or future developments of a system.

• Automatic vs. manual evaluation

In many cases, automatic procedures can be defined to evaluate an NLP system by comparing its output with the gold standard (or desired) one. Although the cost of producing the gold standard can be quite high, automatic evaluation can be repeated as often as needed without much additional costs (on the same input data). However, for many NLP problems, the definition of a gold standard is a complex task, and can prove impossible when inter-annotator agreement is insufficient. Manual evaluation is performed by human judges, which are instructed to estimate the quality of a system, or most often of a sample of its output, based on a number of criteria. Although, thanks to their linguistic competence, human judges can be considered as the reference for a number of language processing tasks, there is also considerable variation across their ratings. This is why automatic evaluation is sometimes referred to as objective evaluation, while the human kind appears to be more subjective.

[edit] Shared tasks (Campaigns)

• BioCreative
• Message Understanding Conference
• Technolangue/Easy
• Text Retrieval Conference

[edit] Standardization in NLP

An ISO sub-committee is working in order to ease interoperability between Lexical resources and NLP programs. The sub-committee is part of ISO/TC37 and is called ISO/TC37/SC4. Some ISO standards are already published but most of them are under construction, mainly on lexicon representation (see LMF), annotation and data category registry.

[edit] Journals

• Computational Linguistics
• International Conference on Language Resources and Evaluation
• Linguistic Issues in Language Technology

[edit] Organizations and conferences [edit] Associations

• Association for Computational Linguistics
• Association for Machine Translation in the Americas
• AFNLP – Asian Federation of Natural Language Processing Associations
• Australasian Language Technology Association (ALTA)
• Spanish Society of Natural Language Processing (SEPLN)
• Mexican Association of Natural Language Processing (AMPLN)

[edit] Conferences

Major conferences include:

• ACL
• COLING
• International Conference on Language Resources and Evaluation
• CICLing
• Empirical Methods on Natural Language Processing (EMNLP)

[edit] Software tools

• General Architecture for Text Engineering (GATE)
• Modular Audio Recognition Framework
• MontyLingua
• Natural Language Toolkit (NLTK): a Python library suite

[edit] See also

• Biomedical text mining
• Chatterbot
• Compound term processing
• Computational linguistics
• Computer-assisted reviewing
• Controlled natural language
• Information retrieval
• Language technology
• Latent semantic indexing
• Lexical Markup Framework
• Loglan / Lojban
• Natural language generation
• Natural language understanding
• Transderivational search
• Reification (linguistics)
• Speech recognition
• Data mining

[edit] References [edit] Related academic articles

• Bates, M. (1995). Models of natural language understanding. Proceedings of the National Academy of Sciences of the United States of America, Vol. 92, No. 22 (Oct. 24, 1995), pp. 9977-9982.

[edit] External links [edit] Resources

• Computational Linguistics Publications
• Computational Linguistics Resources
• Resources for Text, Speech and Language Processing
• A comprehensive list of resources, classified by category
• CICLing annual conferences on Intelligent Text Processing and Computational Linguistics
• Computation Linguistics: Models, Resources, Applications (online text)
• Language Technology Documentation Centre in Finland (FiLT)
• Some simple examples of NLP-hard utterances.

[edit] Organizations

• MedLEE Natural Language Processing by Columbia University and available commercially through NLP International
• The Stanford Natural Language Processing Group
• The Cognitive Computation Group
• NLP group of CIC-IPN; many publications available online
• CLaC — Computation Linguistics at Concordia, Concordia University

Incoming search terms for the article:

• examples of nlp language to help smoking cessation