Examining Implications of Search Query Construction on Sentiment Analysis: The Case of Brandwatch

Consistent with other fields, advertising research is currently facing a replication crisis, evoking methodological concerns regarding the reproducibility of research findings. To address this crisis, we systematically reviewed seven computational advertising studies that collected and analyzed the sentiment of tweets. Upon identifying five studies that reported replicable data collection processes, we utilized the original search queries to recollect tweets using a third-party platform, Brandwatch. We also replicated data collection using iterative complex Boolean (ICB) operators, which sought to enhance the resulting dataset precision. Then, we juxtaposed the effects of search query design (original vs. ICB) using sentiment analysis. Together, the findings suggest that how a query is written can affect measures of precision, engagement, sentiment, and recall. Tests of precision are recommended to mitigate the effects of variance in retrieval relevance.