The use of computer mediated communication has resulted in a new form of written text--Microtext--which is very different from well-written text. Tweets and SMS messages, which have limited length and may contain misspellings, slang, or abbreviations, are two typical examples of microtext. Microtext poses new challenges to standard natural language processing tools which are usually designed for well-written text. The objective of this work is to normalize microtext, in order to produce text that could be suitable for further treatment. We propose a normalization approach based on the source channel model, which incorporates four factors, namely an orthographic factor, a phonetic factor, a contextual factor and acronym expansion. Experiments show that our approach can normalize Twitter messages reasonably well, and it outperforms existing algorithms on a public SMS data set.

While microblogging has gained popularity on the Internet, analyzing and processing short messages has become a challenging task in natural language processing. This paper analyzes the differences between Internet short messages (or “microtext”) and general articles by comparing the Plurk Corpus and the Sinica Balanced Corpus. Likelihood ratio and the tóngyìcícílín thesaurus are adopted to analyze the lexical semantics of frequent terms in each corpus. Furthermore, the NTUSD sentiment dictionary is used to compare the sentiment distribution of the two corpora. The result is also applied to sentiment transition analysis.

We present in this paper, the application of a novel approach to computational modeling, understanding and detection of social phenomena in online multi-party discourse. A two-tiered approach was developed to detect a collection of social phenomena deployed by participants, such as topic control, task control, disagreement and involvement. We discuss how the mid-level social phenomena can be reliably detected in discourse and these measures can be used to differentiate participants of online discourse. Our approach works across different types of online chat and we show results on two specific data sets.

A content analysis of Wikipedia barnstars personalized tokens of appreciation given to participants reveals a wide range of valued work extending beyond simple editing to include social support, administrative actions, and types of articulation work. Barnstars are examples of short semi-structured text characterized by informal grammar and language. We propose a method to classify these barnstars which contain items of interest into various work type categories.We evaluate several multilabel text categorization classifiers and show that significant performance can be achieved by simple classifiers using features which carry context extracted from barnstars. Although this study focused specifically on work categorization via barnstar content for Wikipedia, we believe that the findings are applicable to other similar collaborative systems

Analyzing chat traffic has important applications for both the military and the civilian world. This paper presents a case study of a real-world application of chat analysis in support of team training exercise in the military. It compares the results of an unsupervised learning approach with those of a supervised classification approach. The paper also discusses some of the specific challenges presented by this domain.

How widespread is the phenomenon of commenting or editing a tweet in the practice of retweeting by members of political communities in Twitter? What is the nature of comments(agree/disagree), or of edits (change audience, change meaning, curate content). Being able to answer these questions will provide knowledge that will help answering other questions such as: what are the topics, events, people that attract more discussion (in forms of commenting) or controversy (agree/disagree)? Who are the users who engage in the processing of curating content by inserting hashtags or adding links? Which political community shows more enthusiasm for an issue and how broad is the base of engaged users? How can detection of agreement/disagreement in conversations inform sentiment analysis - the technique used to make predictions (who will win an election) or support insightful analytics (which policy issue resonates more with constituents). We argue that is necessary to go beyond the much-adopted aggregate text analysis of the volume of tweets, in order to discover and understand phenomena at the level of single tweets. This becomes important in the light of the increase in the number of human-mimicking bots in Twitter. Genuine interaction and engagement can be better measured by analyzing tweets that display signs of human intervention. Editing the text of an original tweet before it is retweeted, could reveal mindful user engagement with the content, and therefore, would allow us to perform sampling among real human users. This paper presents work in progress that deals with the challenges of discovering retweets that contain comments or edits, and outlines a machine-learning based strategy for classifying the nature of such comments.

We build a mechanism to form an ontology of entities which improves a relevance of matching and searching microtext. Ontology construction starts from the seed entities and mines the web for new entities associated with them. To form these new entities, machine learning of syntactic parse trees (syntactic generalization) is applied to form commonalities between various search results for existing entities on the web. Ontology and syntactic generalization are applied to relevance improvement in search and text similarity assessment in commercial setting; evaluation results show substantial contribution of both sources to microtext processing.

Rather, the goal is to Microtext media (Ellen, 2011), such as SMS, IM, Twitter, gather relevant messages, organize them, and extract some and text chat, have in common that they use short strings other kind of useful information from them, such as how for immediate communication or broadcast. Microtext can well a team is performing or what people are talking about be construed as one form of micro-messaging (e.g., and when. However, micro-messages do not exist in a Milstein, et al., 2008) which we extend here to include any vacuum; they are contextually oriented and may be part of of a number of other modalities (e.g., telephone calls, a larger network of communications which includes email, face-to-face interaction) used for short, immediate and telephone and other media, including "macro-text." Given (potentially) persistent message passing among this, we have found that natural language processing of the coordinating agents. In this paper, we describe several microtext must be paired with temporal or network recent attempts to study micro-messaging military and analysis of the context. To demonstrate this process, we related organizational contexts.

In a separate study, we were interested in understanding people's Q&A habits on Twitter. Finding questions within Twitter turned out to be a difficult challenge, so we considered applying some traditional NLP approaches to the problem. On the one hand, Twitter is full of idiosyncrasies, which make processing it difficult. On the other it is very restricted in length and tends to employ simple syntactic constructions, which could help the performance of NLP processing. In order to find out the viability of NLP and Twitter, we built a pipeline of tools to work specifically with Twitter input for the task of finding questions in tweets. This work is still preliminary, but in this paper we discuss the techniques we used and the lessons we learned.

In recent years, online chat has become a dominant mode of communication. This text-based medium has the potential of improving information awareness within an organization, but only if the critical information within messages can be identified and directed to where it is most needed. Such a goal has many challenges that traditional Information Extraction (IE) approaches have rarely addressed: the text is “dirty” (containing typos, misspellings, sparse punctuation, etc.), messages are fragmented and refer implicitly to previous messages and shared knowledge, messages from multiple topics are interleaved, etc. Past work in conversation analysis has included in-depth discussions of dialog acts, i.e., the individual utterances that comprise conversations. This paper describes how dialog acts within online chat differ from those within two-person voice conversations. It then presents methods for identifying dialog acts and the role that dialog acts play in identifying individual conversations within a chat stream. Identifying conversations is a necessary step for extracting actionable information, such as identifying individuals with specific expertise, recognizing reports of offline activities, and alerting decision makers to critical developments. Finally, we describe Chat-IE, a prototype software system that performs live dialog identification on chat streams.