TY - JOUR
T1 - The Turker Blues
T2 - Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers
AU - Ophir, Yaakov
AU - Sisso, Itay
AU - Asterhan, Christa S.C.
AU - Tikochinski, Refael
AU - Reichart, Roi
N1 - Publisher Copyright:
© The Author(s) 2019.
PY - 2020/1/1
Y1 - 2020/1/1
N2 - Data collection from online platforms, such as Amazon’s Mechanical Turk (MTurk), has become popular in clinical research. However, there are also concerns about the representativeness and the quality of these data for clinical studies. The present work explores these issues in the specific case of major depression. Analyses of two large data sets gathered from MTurk (Sample 1: N = 2,692; Sample 2: N = 2,354) revealed two major findings: First, failing to screen for inattentive and fake respondents inflates the rates of major depression artificially and significantly (by 18.5%–27.5%). Second, after cleaning the data sets, depression in MTurk is still 1.6 to 3.6 times higher than general population estimates. Approximately half of this difference can be attributed to differences in the composition of MTurk samples and the general population (i.e., sociodemographics, health, and physical activity lifestyle). Several explanations for the other half are proposed, and practical data-quality tools are provided.
AB - Data collection from online platforms, such as Amazon’s Mechanical Turk (MTurk), has become popular in clinical research. However, there are also concerns about the representativeness and the quality of these data for clinical studies. The present work explores these issues in the specific case of major depression. Analyses of two large data sets gathered from MTurk (Sample 1: N = 2,692; Sample 2: N = 2,354) revealed two major findings: First, failing to screen for inattentive and fake respondents inflates the rates of major depression artificially and significantly (by 18.5%–27.5%). Second, after cleaning the data sets, depression in MTurk is still 1.6 to 3.6 times higher than general population estimates. Approximately half of this difference can be attributed to differences in the composition of MTurk samples and the general population (i.e., sociodemographics, health, and physical activity lifestyle). Several explanations for the other half are proposed, and practical data-quality tools are provided.
KW - Mechanical Turk
KW - crowdsourcing
KW - data quality measures
KW - depression
KW - open data
KW - open materials
KW - prevalence of depression
UR - http://www.scopus.com/inward/record.url?scp=85074608412&partnerID=8YFLogxK
U2 - 10.1177/2167702619865973
DO - 10.1177/2167702619865973
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:85074608412
SN - 2167-7026
VL - 8
SP - 65
EP - 83
JO - Clinical Psychological Science
JF - Clinical Psychological Science
IS - 1
ER -