Measuring Twitter Use: Validating Survey-Based Measures

An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare surve...

Full description

Saved in:
Bibliographic Details
Published in:Social science computer review 2021-12, Vol.39 (6), p.1121-1141
Main Authors: Henderson, Michael, Jiang, Ke, Johnson, Martin, Porter, Lance
Format: Article
Language:English
Subjects:
Behavior
Digital media
Error analysis
Mass media
Measurement errors
Polls & surveys
Researcher subject relations
Social media
Social networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3
cites cdi_FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3
container_end_page 1141
container_issue 6
container_start_page 1121
container_title Social science computer review
container_volume 39
creator Henderson, Michael
Jiang, Ke
Johnson, Martin
Porter, Lance
description An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare survey responses to observed behavior in order to assess the validity of self-reported frequency of posting to Twitter, retweeting content, sharing photos, sharing videos, and sending direct messages. Additionally, we examine correlations between self-reported and observed behavior across a range of time frames, from 1 month to 114 months before the survey. We find variation in the quality of self-reports across types of Twitter activity. We also find that self-reports about posting and retweeting tend to reflect recent activity, while self-reports about other activities tend to reflect behavior over a longer span. Furthermore, we find that two characteristics of experience with the platform—the length of time that a person has been active on Twitter and how much their activity on the platform changes over time—predict individual-level discrepancies between survey response and observed behavior, but these discrepancies cancel out when averaged across individuals. Nevertheless, other sources of bias remain. Taken together, our results indicate that while surveys are quite useful for collecting characteristics of social media users, relying on self-reported social media behavior distorts inferential results from what is found when relying on observed social media behavior.
doi_str_mv 10.1177/0894439319896244
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2607605843</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sage_id>10.1177_0894439319896244</sage_id><sourcerecordid>2607605843</sourcerecordid><originalsourceid>FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3</originalsourceid><addsrcrecordid>eNp1kEtLw0AUhQdRsFb3LgOuR-f9cKfFF1Rc2LodZjI3JaU2dSZR-u9NiCAIri6c851z4SB0TsklpVpfEWOF4JZTa6xiQhygCZWSYcOMOkSTwcaDf4xOcl4TQpkmZIL0M_jcpXq7KhZfddtCKpYZros3v6mjbwf9tUufsMe3PkMsRhzyKTqq_CbD2c-douX93WL2iOcvD0-zmzkuObEtZiRQGSofFTMeNNdaMU9pqYKUIVgulQGvVcVMkIpKa0FYXhoeKxOigMin6GLs3aXmo4PcunXTpW3_0jFFtCLSCN5TZKTK1OScoHK7VL_7tHeUuGEe93eePoLHSPYr-C39l_8GD_pjLw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2607605843</pqid></control><display><type>article</type><title>Measuring Twitter Use: Validating Survey-Based Measures</title><source>Sage Journals Online</source><source>Sociological Abstracts</source><creator>Henderson, Michael ; Jiang, Ke ; Johnson, Martin ; Porter, Lance</creator><creatorcontrib>Henderson, Michael ; Jiang, Ke ; Johnson, Martin ; Porter, Lance</creatorcontrib><description>An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare survey responses to observed behavior in order to assess the validity of self-reported frequency of posting to Twitter, retweeting content, sharing photos, sharing videos, and sending direct messages. Additionally, we examine correlations between self-reported and observed behavior across a range of time frames, from 1 month to 114 months before the survey. We find variation in the quality of self-reports across types of Twitter activity. We also find that self-reports about posting and retweeting tend to reflect recent activity, while self-reports about other activities tend to reflect behavior over a longer span. Furthermore, we find that two characteristics of experience with the platform—the length of time that a person has been active on Twitter and how much their activity on the platform changes over time—predict individual-level discrepancies between survey response and observed behavior, but these discrepancies cancel out when averaged across individuals. Nevertheless, other sources of bias remain. Taken together, our results indicate that while surveys are quite useful for collecting characteristics of social media users, relying on self-reported social media behavior distorts inferential results from what is found when relying on observed social media behavior.</description><identifier>ISSN: 0894-4393</identifier><identifier>EISSN: 1552-8286</identifier><identifier>DOI: 10.1177/0894439319896244</identifier><language>eng</language><publisher>Los Angeles, CA: SAGE Publications</publisher><subject>Behavior ; Digital media ; Error analysis ; Mass media ; Measurement errors ; Polls &amp; surveys ; Researcher subject relations ; Social media ; Social networks</subject><ispartof>Social science computer review, 2021-12, Vol.39 (6), p.1121-1141</ispartof><rights>The Author(s) 2019</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3</citedby><cites>FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>315,786,790,27957,27958,33809</link.rule.ids></links><search><creatorcontrib>Henderson, Michael</creatorcontrib><creatorcontrib>Jiang, Ke</creatorcontrib><creatorcontrib>Johnson, Martin</creatorcontrib><creatorcontrib>Porter, Lance</creatorcontrib><title>Measuring Twitter Use: Validating Survey-Based Measures</title><title>Social science computer review</title><description>An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare survey responses to observed behavior in order to assess the validity of self-reported frequency of posting to Twitter, retweeting content, sharing photos, sharing videos, and sending direct messages. Additionally, we examine correlations between self-reported and observed behavior across a range of time frames, from 1 month to 114 months before the survey. We find variation in the quality of self-reports across types of Twitter activity. We also find that self-reports about posting and retweeting tend to reflect recent activity, while self-reports about other activities tend to reflect behavior over a longer span. Furthermore, we find that two characteristics of experience with the platform—the length of time that a person has been active on Twitter and how much their activity on the platform changes over time—predict individual-level discrepancies between survey response and observed behavior, but these discrepancies cancel out when averaged across individuals. Nevertheless, other sources of bias remain. Taken together, our results indicate that while surveys are quite useful for collecting characteristics of social media users, relying on self-reported social media behavior distorts inferential results from what is found when relying on observed social media behavior.</description><subject>Behavior</subject><subject>Digital media</subject><subject>Error analysis</subject><subject>Mass media</subject><subject>Measurement errors</subject><subject>Polls &amp; surveys</subject><subject>Researcher subject relations</subject><subject>Social media</subject><subject>Social networks</subject><issn>0894-4393</issn><issn>1552-8286</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>BHHNA</sourceid><recordid>eNp1kEtLw0AUhQdRsFb3LgOuR-f9cKfFF1Rc2LodZjI3JaU2dSZR-u9NiCAIri6c851z4SB0TsklpVpfEWOF4JZTa6xiQhygCZWSYcOMOkSTwcaDf4xOcl4TQpkmZIL0M_jcpXq7KhZfddtCKpYZros3v6mjbwf9tUufsMe3PkMsRhzyKTqq_CbD2c-douX93WL2iOcvD0-zmzkuObEtZiRQGSofFTMeNNdaMU9pqYKUIVgulQGvVcVMkIpKa0FYXhoeKxOigMin6GLs3aXmo4PcunXTpW3_0jFFtCLSCN5TZKTK1OScoHK7VL_7tHeUuGEe93eePoLHSPYr-C39l_8GD_pjLw</recordid><startdate>202112</startdate><enddate>202112</enddate><creator>Henderson, Michael</creator><creator>Jiang, Ke</creator><creator>Johnson, Martin</creator><creator>Porter, Lance</creator><general>SAGE Publications</general><general>SAGE PUBLICATIONS, INC</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7U4</scope><scope>8FD</scope><scope>BHHNA</scope><scope>DWI</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>WZK</scope></search><sort><creationdate>202112</creationdate><title>Measuring Twitter Use: Validating Survey-Based Measures</title><author>Henderson, Michael ; Jiang, Ke ; Johnson, Martin ; Porter, Lance</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Behavior</topic><topic>Digital media</topic><topic>Error analysis</topic><topic>Mass media</topic><topic>Measurement errors</topic><topic>Polls &amp; surveys</topic><topic>Researcher subject relations</topic><topic>Social media</topic><topic>Social networks</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Henderson, Michael</creatorcontrib><creatorcontrib>Jiang, Ke</creatorcontrib><creatorcontrib>Johnson, Martin</creatorcontrib><creatorcontrib>Porter, Lance</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Sociological Abstracts (pre-2017)</collection><collection>Technology Research Database</collection><collection>Sociological Abstracts</collection><collection>Sociological Abstracts</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Sociological Abstracts (Ovid)</collection><jtitle>Social science computer review</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Henderson, Michael</au><au>Jiang, Ke</au><au>Johnson, Martin</au><au>Porter, Lance</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Measuring Twitter Use: Validating Survey-Based Measures</atitle><jtitle>Social science computer review</jtitle><date>2021-12</date><risdate>2021</risdate><volume>39</volume><issue>6</issue><spage>1121</spage><epage>1141</epage><pages>1121-1141</pages><issn>0894-4393</issn><eissn>1552-8286</eissn><abstract>An important challenge for research on social media use is to relate users’ activity on these platforms to user characteristics such as demographics. Surveys allow researchers to measure these characteristics but may be subject to measurement error in self-reported social media use. We compare survey responses to observed behavior in order to assess the validity of self-reported frequency of posting to Twitter, retweeting content, sharing photos, sharing videos, and sending direct messages. Additionally, we examine correlations between self-reported and observed behavior across a range of time frames, from 1 month to 114 months before the survey. We find variation in the quality of self-reports across types of Twitter activity. We also find that self-reports about posting and retweeting tend to reflect recent activity, while self-reports about other activities tend to reflect behavior over a longer span. Furthermore, we find that two characteristics of experience with the platform—the length of time that a person has been active on Twitter and how much their activity on the platform changes over time—predict individual-level discrepancies between survey response and observed behavior, but these discrepancies cancel out when averaged across individuals. Nevertheless, other sources of bias remain. Taken together, our results indicate that while surveys are quite useful for collecting characteristics of social media users, relying on self-reported social media behavior distorts inferential results from what is found when relying on observed social media behavior.</abstract><cop>Los Angeles, CA</cop><pub>SAGE Publications</pub><doi>10.1177/0894439319896244</doi><tpages>21</tpages></addata></record>
fulltext fulltext
identifier ISSN: 0894-4393
ispartof Social science computer review, 2021-12, Vol.39 (6), p.1121-1141
issn 0894-4393
1552-8286
language eng
recordid cdi_proquest_journals_2607605843
source Sage Journals Online; Sociological Abstracts
subjects Behavior
Digital media
Error analysis
Mass media
Measurement errors
Polls & surveys
Researcher subject relations
Social media
Social networks
title Measuring Twitter Use: Validating Survey-Based Measures
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-09-23T00%3A17%3A57IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Measuring%20Twitter%20Use:%20Validating%20Survey-Based%20Measures&rft.jtitle=Social%20science%20computer%20review&rft.au=Henderson,%20Michael&rft.date=2021-12&rft.volume=39&rft.issue=6&rft.spage=1121&rft.epage=1141&rft.pages=1121-1141&rft.issn=0894-4393&rft.eissn=1552-8286&rft_id=info:doi/10.1177/0894439319896244&rft_dat=%3Cproquest_cross%3E2607605843%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c309t-20b15bfad628ae737762a11c6b55bb93568ea76f28b561599e493c83df8bd4ed3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2607605843&rft_id=info:pmid/&rft_sage_id=10.1177_0894439319896244&rfr_iscdi=true