On the Stability of Online Language Features: How Much Text do you Need to know a Person?

In recent years, numerous studies have inferred personality and other traits from people's online writing. While these studies are encouraging, more information is needed in order to use these techniques with confidence. How do linguistic features vary across different online media, and how much text is required to have a representative sample for a person? In this paper, we examine several large sets of online, user-generated text, drawn from Twitter, email, blogs, and online discussion forums. We examine and compare population-wide results for the linguistic measure LIWC, and the inferred traits of Big5 Personality and Basic Human Values. We also empirically measure the stability of these traits across different sized samples for each individual. Our results highlight the importance of tuning models to each online medium, and include guidelines for the minimum amount of text required for a representative result.