Measuring Twitter Use: Validating Survey-Based Measures

An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare surve...

Full description

Saved in:
Bibliographic Details
Published in:Social science computer review 2021-12, Vol.39 (6), p.1121-1141
Main Authors: Henderson, Michael, Jiang, Ke, Johnson, Martin, Porter, Lance
Format: Article
Language:English
Subjects:
Behavior
Digital media
Error analysis
Mass media
Measurement errors
Polls & surveys
Researcher subject relations
Social media
Social networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare survey responses to observed behavior in order to assess the validity of self-reported frequency of posting to Twitter, retweeting content, sharing photos, sharing videos, and sending direct messages. Additionally, we examine correlations between self-reported and observed behavior across a range of time frames, from 1 month to 114 months before the survey. We find variation in the quality of self-reports across types of Twitter activity. We also find that self-reports about posting and retweeting tend to reflect recent activity, while self-reports about other activities tend to reflect behavior over a longer span. Furthermore, we find that two characteristics of experience with the platform—the length of time that a person has been active on Twitter and how much their activity on the platform changes over time—predict individual-level discrepancies between survey response and observed behavior, but these discrepancies cancel out when averaged across individuals. Nevertheless, other sources of bias remain. Taken together, our results indicate that while surveys are quite useful for collecting characteristics of social media users, relying on self-reported social media behavior distorts inferential results from what is found when relying on observed social media behavior.
ISSN:0894-4393
1552-8286
DOI:10.1177/0894439319896244