Identifying Social Deliberative Behavior from Online Communication — A Cross-Domain Study

In this paper we describe automatic systems for identifying whether participants demonstrate social deliberative behavior within their online conversations. We test 3 corpora containing 2617 annotated segments. With machine learning models using linguistic features, we identify social deliberative behavior with up to 68.09% in-domain accuracy (com- pared to 50% baseline), 62.17% in-domain precision, and 84% in-domain recall. In cross-domain identification tasks, we achieve up to 55.56% cross-domain accuracy, 59.84% cross-domain precision, and 86.58% cross-domain recall. We also discover linguistic characteristics of social deliberative behavior. In the context of identifying social deliberative be- havior, we offer insights into why certain machine learning models generalize well across domains and why certain domains pose great challenges to machine learning models.